Independent scholarship · Open accessThe monthly digest

Indexed text attachment

topos-actions-in-machine-learning-and-ai.tex

Source for Topos Actions in Machine Learning and Artificial Intelligence

View paper · Download original file

Lines 187–286 of 1207. Source text is displayed literally.

187 188This elementary result makes the applications testable.189An architecture specifies a parameter object, a realization190map, and which symmetries it respects. A loss and a learning191rule require their own compatibility with those data.192Sections~\ref{sec:direct}--\ref{sec:categorical-ml} review193relevant work. The later sections develop descent for layer194evaluation and compute two specializations where the195symmetry changes what can be learned or represented.196\par197\section{Topos-theoretic precedents in learning}198\label{sec:direct}199 200The closest precedents use three different levels of structure.201Some place the states and maps of a learner in a topos of equivariant202objects. Others use a classifying topos to organize possible203interpretations and changes of context. A literal action in the204sense considered here gives a monoidal Grothendieck205topos \(\mathcal T\) a geometric action206\(\mathcal T\times\mathcal X\to\mathcal X\) on another topos. One207can test this last formulation by looking for the parameter topos,208the geometric action map, and the coherent multiplication and209unit comparisons. The existing overview of Jia, Peng, Yang, and210Chen \cite{jia-topos-survey} collects examples of topos-based211learning in the broader sense. We examined its references,212then checked primary sources for geometric actions, classifying213topoi, equivariant sheaves, and isotropy in relation to learning.214The sources give a basis for a focused survey at the first215two levels. They leave the design of learning procedures using216literal monoidal-topos actions or general localic isotropy as a217separate question.218 219\subsection{Neural networks in equivariant topoi}220 221A direct precedent relating topoi and neural networks is222Belfiore and Bennequin's preprint \cite[\S2.1--2.2]{belfiore-bennequin}.223A network has an indexing category \(C\) that records its224architecture. In their constant-symmetry example one replaces225set-valued presheaves by presheaves taking values in226\(\mathbf{B}G\), the topos of left \(G\)-sets for a discrete227group \(G\). A feature diagram is then228\[229  F:C^{\mathrm{op}}\longrightarrow \mathbf{B}G .230\]231Arrows of \(C\) induce restriction maps on feature data and232the maps of \(\mathbf{B}G\) require equivariance of each233transition. Curried evaluation gives234\[235\begin{aligned}236 \operatorname{Fun}(C^{\mathrm{op}},\mathbf{B}G)237   &\simeq \operatorname{Fun}(C^{\mathrm{op}}\times BG,\mathbf{Set})\\238   &\simeq \widehat{C\times(BG)^{\mathrm{op}}}.239\end{aligned}240\]241The last term is the presheaf topos on \(C\times (BG)^{\mathrm{op}}\). Here \(BG\)242is the one-object category with arrow group \(G\). In243equivariantization terminology the first expression is the244topos of equivariant objects for the trivial external action of245\(G\) on \(\widehat C\). This is a concrete use of the group246classifying topos as a target for feature data. The geometric247theory of arbitrary \(G\)-actions has a classifier of its own.248Its points describe \(G\)-sets, whereas points of \(\mathbf{B}G\)249describe \(G\)-torsors \cite[slide 22]{lafforgue-eth-ai}.250 251Belfiore and Bennequin also let the categories in the252invariance structure vary over the architecture and use253a fibration whose fibers carry those categories. Their relative254action is given by contravariant functors between fibers255commuting with reindexing \cite[Eq.~(2.1)]{belfiore-bennequin}.256This specifies compatible diagrams of behaviors in the257fibers. A geometric action of a parameter topos would further258specify a multiplication on the parameter topos and a coherent259geometric morphism out of a product. The preprint proposes that260invariance explains aspects of extrapolation. The topos261description supplies a type of invariance constraint.262Statistical generalization still requires an input distribution,263a loss, and hypotheses about a training procedure.264 265A related result gives a topos from the selected interface of266a learner. Spivak \cite{spivak-learners-languages} represents267parametrized learners as generalized Moore machines using268polynomial interfaces. For a fixed polynomial \(p\), the269category of \(p\)-coalgebras forms a topos. Its internal language270can express propositions about the dynamics, with gradient271descent as a worked case. This makes the topos a setting for272specifying learner state and behavior. The monoidal structure273used to build the interfaces belongs to the category of274polynomials. The more recent survey also discusses275\emph{The Topos of Transformer Networks}. The canonical record276for that preprint is marked withdrawn277\cite{villani-transformers-withdrawn}.278 279\subsection{Relative classifying topoi for symbolic AI}280 281Lafforgue \cite[slides 20--25]{lafforgue-eth-ai} proposes282organizing a data language by a geometric theory \(T\).283A map from a test topos to its classifying topos classifies a284\(T\)-model in that test topos. Taking the test topos to vary285therefore models families of interpretations and their286change of base. His proposed networks of topoi attach