Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 447–546 of 1207. Source text is displayed literally.
447dimensions, for which the incidence equation still448makes sense and there need not be transport along449groupoid arrows.450 451\subsection{From global groups to local bisections}452 453Global group actions determine the shared weights in454group-equivariant convolutions \cite{cohen-welling}.455Cohen, Geiger, and Weiler characterize equivariant456linear maps between fields on homogeneous spaces by457their convolution kernels \cite{cohen-geiger-weiler}.458On meshes, de Haan et al.\ use parallel transport to459define orientation-sensitive convolutions that respect460changes of local frame \cite{dehaan-mesh}.461 462For a finite data set carrying a group action, let463$K_{x,y}:V_y\to W_x$ be the blocks of a linear464layer and let $\rho_V(g;y)$ and $\rho_W(g;x)$ be the465feature transports. Equivariance is equivalent to466\[467 K_{gx,gy}468 =\rho_W(g;x)K_{x,y}\rho_V(g;y)^{-1}.469\]470It follows by comparing coefficients of a feature471supported at $y$. Each diagonal pair orbit has one472free block in the intertwiner space of its joint473stabilizer, since two transports from a representative474to the same pair differ by a stabilizer element.475Conversely an intertwiner at each representative476propagates to a well-defined equivariant kernel.477 478Ibort, Jimenez-Vazquez, and Perez-Pardo extend this type479of kernel analysis to groupoid representations with a480chosen pseudogroup of local bisections, measure, and481input and output bundles. Their preprint treats bounded482domains, where a geometric transformation can be483defined on part of the data space and pair types can484depend on the boundary \cite{ibort-groupoid-cnn}. The485choice of which bisections to act by and the integration486and support conditions enter the analytical layer487construction. In the groupoid descent picture, the488representation bundles supply transport data on the489features and on the kernel's source and target490variables. The kernel then acts on spaces of sections491and can mix distinct sites. A morphism of feature492representations over the identity base, such as $A$493above, acts fiberwise. This distinction separates useful494precedents for learning with groupoid descent from495studies of a monoidal Grothendieck topos acting on496another topos by geometric morphisms.497\par498\section{Compositional training and inference}499\label{sec:categorical-ml}500 501The categorical literature on training and probability supplies two502ingredients for learning in a setting with topos actions. Parametrized503maps provide a compositional description of local models, and504stochastic kernels provide a description of local inference.505To obtain a model over a groupoid of domains or in an action topos,506one supplies additional transport for the parameters, inputs, and507outputs. Compatibility is a condition on both the local maps and508the operations applied to them.509 510\subsection{Reverse differentiation and descent of updates}511 512Fong, Spivak, and Tuy\'eras make a learner into a compositional513morphism with a forward evaluation, a parameter update, and a514backward request. Under their choices of error function and fixed515learning rate, gradient descent gives a monoidal functor from516parametrized functions to learners \cite{fong-backprop}.517Cruttwell et al.\ separate the optimizer and loss from the architecture518using parametrized maps, lenses, and reverse derivative categories519\cite{cruttwell-gradient}. These constructions address how a training520rule composes along a network. To use them with an action on a topos,521the relevant categories and rules must also support transport along522changes of domain. The transport condition is what allows the523compositional training rule to respect the chosen action.524 525For example, in smooth models the cotangent pullback of a loss526covector is canonical. Given layers \(f:X\to Y\) and \(g:Y\to Z\),527the cotangent maps satisfy528\[529 D(g\circ f)_x^*=(Df_x)^*(Dg_{f(x)})^* .530\]531The same identity applied to a commutative square of local realizations532describes the transformation of the reverse pass. Converting a533cotangent vector into a parameter update adds geometric data. In534particular, a matrix transpose relative to a Euclidean metric in one535coordinate system implements a different conversion when a536nonisometric change of coordinates is made and the same Euclidean537formula is used in the new coordinates.538 539An explicit sufficient condition for a global gradient is available540in the smooth setting. Suppose a groupoid presents a parameter model541by local manifolds \(P_i\) with transition diffeomorphisms542\(\phi_{ij}\) on the corresponding overlaps. Equip the local543manifolds with losses \(\ell_i\) and positive-definite metrics \(g_i\)544satisfying545\[546 \ell_i=\ell_j\circ\phi_{ij},\qquad