Independent scholarship · Open accessThe monthly digest

Indexed text attachment

topos-actions-in-machine-learning-and-ai.tex

Source for Topos Actions in Machine Learning and Artificial Intelligence

View paper · Download original file

Lines 221–320 of 1207. Source text is displayed literally.

221A direct precedent relating topoi and neural networks is222Belfiore and Bennequin's preprint \cite[\S2.1--2.2]{belfiore-bennequin}.223A network has an indexing category \(C\) that records its224architecture. In their constant-symmetry example one replaces225set-valued presheaves by presheaves taking values in226\(\mathbf{B}G\), the topos of left \(G\)-sets for a discrete227group \(G\). A feature diagram is then228\[229  F:C^{\mathrm{op}}\longrightarrow \mathbf{B}G .230\]231Arrows of \(C\) induce restriction maps on feature data and232the maps of \(\mathbf{B}G\) require equivariance of each233transition. Curried evaluation gives234\[235\begin{aligned}236 \operatorname{Fun}(C^{\mathrm{op}},\mathbf{B}G)237   &\simeq \operatorname{Fun}(C^{\mathrm{op}}\times BG,\mathbf{Set})\\238   &\simeq \widehat{C\times(BG)^{\mathrm{op}}}.239\end{aligned}240\]241The last term is the presheaf topos on \(C\times (BG)^{\mathrm{op}}\). Here \(BG\)242is the one-object category with arrow group \(G\). In243equivariantization terminology the first expression is the244topos of equivariant objects for the trivial external action of245\(G\) on \(\widehat C\). This is a concrete use of the group246classifying topos as a target for feature data. The geometric247theory of arbitrary \(G\)-actions has a classifier of its own.248Its points describe \(G\)-sets, whereas points of \(\mathbf{B}G\)249describe \(G\)-torsors \cite[slide 22]{lafforgue-eth-ai}.250 251Belfiore and Bennequin also let the categories in the252invariance structure vary over the architecture and use253a fibration whose fibers carry those categories. Their relative254action is given by contravariant functors between fibers255commuting with reindexing \cite[Eq.~(2.1)]{belfiore-bennequin}.256This specifies compatible diagrams of behaviors in the257fibers. A geometric action of a parameter topos would further258specify a multiplication on the parameter topos and a coherent259geometric morphism out of a product. The preprint proposes that260invariance explains aspects of extrapolation. The topos261description supplies a type of invariance constraint.262Statistical generalization still requires an input distribution,263a loss, and hypotheses about a training procedure.264 265A related result gives a topos from the selected interface of266a learner. Spivak \cite{spivak-learners-languages} represents267parametrized learners as generalized Moore machines using268polynomial interfaces. For a fixed polynomial \(p\), the269category of \(p\)-coalgebras forms a topos. Its internal language270can express propositions about the dynamics, with gradient271descent as a worked case. This makes the topos a setting for272specifying learner state and behavior. The monoidal structure273used to build the interfaces belongs to the category of274polynomials. The more recent survey also discusses275\emph{The Topos of Transformer Networks}. The canonical record276for that preprint is marked withdrawn277\cite{villani-transformers-withdrawn}.278 279\subsection{Relative classifying topoi for symbolic AI}280 281Lafforgue \cite[slides 20--25]{lafforgue-eth-ai} proposes282organizing a data language by a geometric theory \(T\).283A map from a test topos to its classifying topos classifies a284\(T\)-model in that test topos. Taking the test topos to vary285therefore models families of interpretations and their286change of base. His proposed networks of topoi attach287geometric object categories and classifiers to different288processing stages. A transformation natural in whole289families is a candidate for a geometric morphism between290classifiers. This provides a way to formulate the transfer of291structured representations. The talk gives an architecture292proposal rather than an optimization or accuracy result.293Caramello's course on relative topoi \cite{caramello-raven}294makes the symbolic application more concrete. It writes295invariance constraints for Raven matrices in a geometric296language, including an example where an arithmetic rule on297positions is expressed by a formula indexed by the graph of298that rule. Restricted vocabularies and specified relations299make candidate interpretations comparable by proof. This is300a formal model of a class of reasoning problems and a distinct301application from training an equivariant numerical predictor.302 303\subsection{An algorithmic precedent via nominal sets}304 305Active automata learning gives an implemented example of306learning in a symmetry-sensitive setting. The nominal topos307is the category of discrete sets with continuous action of the308permutation group of countably many names, equipped with309the topology of pointwise convergence. Continuity of an orbit310map means that an element has a finite support: its stabilizer311contains the subgroup fixing some finite set of names312pointwise. The set of name pairs, for example, has two orbits,313equal and unequal pairs. Such orbit-finite structure permits314a finite orbit description even for machines with infinitely315many individually named inputs. Iwaniack316\cite{iwaniack-automata-toposes} uses the nominal topos as317an instance of topos-enriched automata and derives318Myhill--Nerode results there.319 320Moerman and collaborators \cite{moerman-nominal-learning}