Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 167–266 of 1207. Source text is displayed literally.
167 168\subsection{External symmetry and spaces of layers}169 170Let an ordinary group $H$ act by geometric autoequivalences on171a Grothendieck topos $\mathcal E$. Write $T_h$ for the inverse172image indexed by the inverse element, so that173$T_hT_k\simeq T_{hk}$. An equivariant object $X$ has structure174$\theta_{h,X}:T_h X\simeq X$. The equivariant topos175$\mathcal E^H$ is the geometric quotient. For equivariant176objects $X,Y$, the underlying exponential $Y^X$ acquires177the conjugation action. The equivalence178$T_h(Y^X)\simeq(T_hY)^{T_hX}$ is followed by precomposition179with $\theta_{h,X}^{-1}$ and postcomposition with $\theta_{h,Y}$.180The two coherence laws follow by composing these maps on181either side of an evaluation. Consequently a layer182$P\times X\to Y$ is equivariant precisely when its transpose183$P\to Y^X$ is equivariant. The usual equation184$F(hp,hx)=hF(p,x)$ is its case in sets. It is an equation185about the family. At a fixed parameter the equation applies186to its stabilizer.187 188This elementary result makes the applications testable.189An architecture specifies a parameter object, a realization190map, and which symmetries it respects. A loss and a learning191rule require their own compatibility with those data.192Sections~\ref{sec:direct}--\ref{sec:categorical-ml} review193relevant work. The later sections develop descent for layer194evaluation and compute two specializations where the195symmetry changes what can be learned or represented.196\par197\section{Topos-theoretic precedents in learning}198\label{sec:direct}199 200The closest precedents use three different levels of structure.201Some place the states and maps of a learner in a topos of equivariant202objects. Others use a classifying topos to organize possible203interpretations and changes of context. A literal action in the204sense considered here gives a monoidal Grothendieck205topos \(\mathcal T\) a geometric action206\(\mathcal T\times\mathcal X\to\mathcal X\) on another topos. One207can test this last formulation by looking for the parameter topos,208the geometric action map, and the coherent multiplication and209unit comparisons. The existing overview of Jia, Peng, Yang, and210Chen \cite{jia-topos-survey} collects examples of topos-based211learning in the broader sense. We examined its references,212then checked primary sources for geometric actions, classifying213topoi, equivariant sheaves, and isotropy in relation to learning.214The sources give a basis for a focused survey at the first215two levels. They leave the design of learning procedures using216literal monoidal-topos actions or general localic isotropy as a217separate question.218 219\subsection{Neural networks in equivariant topoi}220 221A direct precedent relating topoi and neural networks is222Belfiore and Bennequin's preprint \cite[\S2.1--2.2]{belfiore-bennequin}.223A network has an indexing category \(C\) that records its224architecture. In their constant-symmetry example one replaces225set-valued presheaves by presheaves taking values in226\(\mathbf{B}G\), the topos of left \(G\)-sets for a discrete227group \(G\). A feature diagram is then228\[229 F:C^{\mathrm{op}}\longrightarrow \mathbf{B}G .230\]231Arrows of \(C\) induce restriction maps on feature data and232the maps of \(\mathbf{B}G\) require equivariance of each233transition. Curried evaluation gives234\[235\begin{aligned}236 \operatorname{Fun}(C^{\mathrm{op}},\mathbf{B}G)237 &\simeq \operatorname{Fun}(C^{\mathrm{op}}\times BG,\mathbf{Set})\\238 &\simeq \widehat{C\times(BG)^{\mathrm{op}}}.239\end{aligned}240\]241The last term is the presheaf topos on \(C\times (BG)^{\mathrm{op}}\). Here \(BG\)242is the one-object category with arrow group \(G\). In243equivariantization terminology the first expression is the244topos of equivariant objects for the trivial external action of245\(G\) on \(\widehat C\). This is a concrete use of the group246classifying topos as a target for feature data. The geometric247theory of arbitrary \(G\)-actions has a classifier of its own.248Its points describe \(G\)-sets, whereas points of \(\mathbf{B}G\)249describe \(G\)-torsors \cite[slide 22]{lafforgue-eth-ai}.250 251Belfiore and Bennequin also let the categories in the252invariance structure vary over the architecture and use253a fibration whose fibers carry those categories. Their relative254action is given by contravariant functors between fibers255commuting with reindexing \cite[Eq.~(2.1)]{belfiore-bennequin}.256This specifies compatible diagrams of behaviors in the257fibers. A geometric action of a parameter topos would further258specify a multiplication on the parameter topos and a coherent259geometric morphism out of a product. The preprint proposes that260invariance explains aspects of extrapolation. The topos261description supplies a type of invariance constraint.262Statistical generalization still requires an input distribution,263a loss, and hypotheses about a training procedure.264 265A related result gives a topos from the selected interface of266a learner. Spivak \cite{spivak-learners-languages} represents