Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 203–302 of 1343. Source text is displayed literally.

203\[204 \Map_B(P\times_B X,Y)\simeq\Map_B(P,[X,Y]_B).205\]206It is natural in $P$. The two definitions of a morphism of207layers therefore give the same homotopy fiber of a map of mapping208spaces, including its higher simplices. The relative double slice209$(\overB)_{/[X,Y]_B}$ is $\E_{/[X,Y]_B}$.210 211Evaluation in series transposes to a morphism212\[213 [X,Y]_B\times_B[Y,Z]_B\longrightarrow [X,Z]_B.214\]215It sends the classifying maps of the two layers to the classifying216map of their composite. The identity is classified by the transpose217of $\id_X$ with parameter $B$. The associativity and unit218compatibilities follow by uncurrying the iterated compositions.219Both ways to combine a string yield evaluation in the same order220on the product of its parameters. All product comparisons come221from their limit universal properties.222 223Let $a:C\to B$. If $T\to C$, the left adjoint $a_!$ to $a^*$224is composition of structure maps. Using the pullback identification225$a_!(T\times_C a^*X)\simeq (a_!T)\times_B X$ gives226\begin{align*}227 \Map_C(T,a^*[X,Y]_B)228 &\simeq \Map_B(a_!T,[X,Y]_B)\\229 &\simeq \Map_B(a_!T\times_B X,Y)\\230 &\simeq \Map_C(T\times_C a^*X,a^*Y).231\end{align*}232By Yoneda these equivalences identify $a^*[X,Y]_B$233with $[a^*X,a^*Y]_C$, preserving evaluation. Pullback also234preserves the products that define series composition.235 236Finally, the effective epimorphism is the colimit of its237\v{C}ech nerve. Colimits in an $\infty$-topos satisfy descent for238slices \cite[\S6.1.3]{lurie-htt}. Hence $\overB$ is the limit of the239categories $\E_{/U_n}$. Taking the slice at the compatible240exponential commutes with this limit, since a slice is the241category of arrows with fixed target. Base change identifies242the local targets with the ones in \eqref{eq:layerdescent}.243\end{proof}244 245\begin{figure}[tbp]246\centering247\(248\begin{tikzcd}[column sep=2.4em,row sep=2.0em]249  P\times_B X \arrow[r,"f"] \arrow[d,"u\times_B\id_X"'] &250  Y \arrow[d,equal] \\251  Q\times_B X \arrow[r,"g"'] & Y252\end{tikzcd}253\)254\caption{A reparametrization includes a commuting comparison of255evaluations. In an $\infty$-topos the comparison is a specified homotopy.}256\label{fig:reparam}257\end{figure}258 259A point of $[X,Y]_B$ over $b$ is one evaluation on the fibers.260The classifying map retains a distinction between a space of these261evaluations and a space of weights mapping into it. An architecture262selects a classifying map, for instance by composing affine maps,263an equivariant convolution, and a nonlinearity. The ambient264exponential also includes maps that the selected architecture265cannot realize.266 267\subsection{Transport and actions}268 269In spaces, straightening identifies $\Spc_{/B}$ with270$\Fun(B,\Spc)$, where the base $B$ is regarded as an271$\infty$-groupoid. Transport acts by equivalences and the slice272exponential is computed fiberwise \cite[\S3.2]{lurie-htt}. For273a path $\gamma:b\to c$ with transports $t_X,t_Y$, exponentials274transport by $h\mapsto t_Y h t_X^{-1}$. A classifying map275is consequently a coherent family of maps276$P_b\to [X_b,Y_b]$ commuting with transport. For an ordinary277action of an abstract group $G$ on $P,X,Y$, the resulting278comparison reads279\begin{equation}280 \label{eq:equiv-eval}281 f(gp,gx)=g f(p,x).282\end{equation}283It is a condition on the entire family. At a fixed $p$, it284forces equivariance under the stabilizer $G_p$. In a smooth285site one can instead use the classifying stack of a Lie group.286Sections of that stack are torsors with smooth local287trivializations, and \eqref{eq:equiv-eval} then holds for288smooth test families. The homotopy type of a smooth base289retains path transport and may identify smoothly distinct data.290 291\subsection{Realization on a quotient}292 293Suppose a group object $G$ acts on $P$ and a realization294$r:P\to M$ is coherently invariant with trivial action on295$M$. Here $M$ can itself be a space of functions or probability296laws. The action groupoid has simplicial levels $P\times G^n$.297Its colimit is $P//G$, and the colimit universal property gives298\[299 \Map(P//G,M)\simeq \Tot\,\Map(P\times G^\bullet,M).300\]301Thus $r$ factors through $P//G$ exactly when equipped with302descent, whose first comparison identifies $r(p)$ and $r(gp)$.