Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 141–240 of 1343. Source text is displayed literally.

141controls how evaluations transform. Descent determines when local142evaluations and parameter presentations assemble across a base.143Theorem~\ref{thm:relative-representation} gives the general comparison.144Theorem~\ref{thm:harmonic} computes the moving-parameter part in a145compact linear setting.146 147Parametrized composition and supervised learning have categorical148formulations through learners, lenses, and reverse differentiation149\cite{fong-spivak-tuyeras,cruttwell}. Markov categories organize channels150and sufficient statistics \cite{fritz-2020}, while equivariant151convolution on homogeneous spaces is described by induced152representations \cite{cohen-geiger-weiler}. Here we apply representation153constraints to the evaluation families themselves. This permits the same154typing of a manifold layer, a statistical decoder, and a locally presented155system of layers.156 157\section{Relative parametrized models}158\label{sec:relative}159 160Let $\E$ be an $\infty$-topos and $B$ an object of $\E$.161Possible choices are the $\infty$-category $\Spc$ of spaces and the162$\infty$-topos of sheaves of spaces on a smooth site. Work first163in $\overB$. For $X,Y$ over $B$, write $[X,Y]_B$ for their164exponential in that slice. Its evaluation map is165$[X,Y]_B\times_B X\to Y$ over $B$. A choice of site specifies which166families of sections are admitted. Smooth manifolds will be embedded167by their smooth functors of points.168 169\begin{definition}[A parametrized layer]170A layer from $X$ to $Y$ over $B$ consists of $P\to B$ and an171evaluation $f:P\times_B X\to Y$. A map to $(Q,g)$ is a morphism172$u:P\to Q$ over $B$ with a specified homotopy173$f\simeq g(u\times_B\id_X)$ over $B$. Higher morphisms are the174compatibilities in the mapping space of the resulting slice.175\end{definition}176 177Write $r_f:P\to[X,Y]_B$ for the transpose of $f$. Given a second178layer $(Q,g):Y\to Z$, the series composite has parameter179$P\times_B Q$ and evaluation $(p,q,x)\mapsto g(q,f(p,x))$.180Here and below parametric formulas describe generalized points in the181slice. A shared parameter $P$ can be imposed by reparametrization along182the diagonal $P\to P\times_B P$. More general tying is expressed by a183map $P\to\prod_{i,B}P_i$ into the product of layer parameters.184 185\begin{theorem}[Relative representation and descent]186\label{thm:relative-representation}187The $\infty$-category of layers from $X$ to $Y$ is188$\E_{/[X,Y]_B}$. On these categories, series composition is189induced by internal composition of exponentials and respects190reparametrizations. The construction commutes with pullback in $B$.191For an effective epimorphism $U\to B$ with \v{C}ech nerve $U_\bullet$,192the global layer category is equivalent to193\begin{equation}194 \label{eq:layerdescent}195 \lim_{[n]\in\boldsymbol\Delta}196   \E_{/[X_n,Y_n]_{U_n}},197 \qquad X_n=X\times_B U_n,\quad Y_n=Y\times_B U_n .198\end{equation}199\end{theorem}200 201\begin{proof}202Currying yields the equivalence of mapping spaces203\[204 \Map_B(P\times_B X,Y)\simeq\Map_B(P,[X,Y]_B).205\]206It is natural in $P$. The two definitions of a morphism of207layers therefore give the same homotopy fiber of a map of mapping208spaces, including its higher simplices. The relative double slice209$(\overB)_{/[X,Y]_B}$ is $\E_{/[X,Y]_B}$.210 211Evaluation in series transposes to a morphism212\[213 [X,Y]_B\times_B[Y,Z]_B\longrightarrow [X,Z]_B.214\]215It sends the classifying maps of the two layers to the classifying216map of their composite. The identity is classified by the transpose217of $\id_X$ with parameter $B$. The associativity and unit218compatibilities follow by uncurrying the iterated compositions.219Both ways to combine a string yield evaluation in the same order220on the product of its parameters. All product comparisons come221from their limit universal properties.222 223Let $a:C\to B$. If $T\to C$, the left adjoint $a_!$ to $a^*$224is composition of structure maps. Using the pullback identification225$a_!(T\times_C a^*X)\simeq (a_!T)\times_B X$ gives226\begin{align*}227 \Map_C(T,a^*[X,Y]_B)228 &\simeq \Map_B(a_!T,[X,Y]_B)\\229 &\simeq \Map_B(a_!T\times_B X,Y)\\230 &\simeq \Map_C(T\times_C a^*X,a^*Y).231\end{align*}232By Yoneda these equivalences identify $a^*[X,Y]_B$233with $[a^*X,a^*Y]_C$, preserving evaluation. Pullback also234preserves the products that define series composition.235 236Finally, the effective epimorphism is the colimit of its237\v{C}ech nerve. Colimits in an $\infty$-topos satisfy descent for238slices \cite[\S6.1.3]{lurie-htt}. Hence $\overB$ is the limit of the239categories $\E_{/U_n}$. Taking the slice at the compatible240exponential commutes with this limit, since a slice is the