Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 527–626 of 1207. Source text is displayed literally.
527the cotangent maps satisfy528\[529 D(g\circ f)_x^*=(Df_x)^*(Dg_{f(x)})^* .530\]531The same identity applied to a commutative square of local realizations532describes the transformation of the reverse pass. Converting a533cotangent vector into a parameter update adds geometric data. In534particular, a matrix transpose relative to a Euclidean metric in one535coordinate system implements a different conversion when a536nonisometric change of coordinates is made and the same Euclidean537formula is used in the new coordinates.538 539An explicit sufficient condition for a global gradient is available540in the smooth setting. Suppose a groupoid presents a parameter model541by local manifolds \(P_i\) with transition diffeomorphisms542\(\phi_{ij}\) on the corresponding overlaps. Equip the local543manifolds with losses \(\ell_i\) and positive-definite metrics \(g_i\)544satisfying545\[546 \ell_i=\ell_j\circ\phi_{ij},\qquad547 g_i=\phi_{ij}^{*}g_j .548\]549Then their gradients are compatible on the overlaps:550\[551 D\phi_{ij}(\operatorname{grad}_{g_i}\ell_i)552 =(\operatorname{grad}_{g_j}\ell_j)\circ\phi_{ij}.553\]554Indeed, pairing the left-hand side with \(D\phi_{ij}(v)\) in \(g_j\)555gives \(g_i(\operatorname{grad}_{g_i}\ell_i,v)=d\ell_i(v)\).556Differentiating the loss identity gives the same pairing for the557right-hand side. Surjectivity of the differential of the transition558and nondegeneracy of the metric give equality. Local gradient flows559therefore respect the transitions wherever defined, by uniqueness of560solutions. A discrete training rule requires its own compatibility.561For instance, a step made using the Riemannian exponential inherits562the compatibility on the domain where it is defined, because the563transitions are local isometries.564 565Natural gradient supplies a relevant source of metrics566\cite{amari-natural}. For a smooth family of strictly positive567probability laws \(p_\theta(y)\) on a finite outcome set, its Fisher568form is569\[570 I_\theta(v,w)571 =\sum_y \frac{d p_\theta(y)(v)\,d p_\theta(y)(w)}{p_\theta(y)} .572\]573If a transition reparametrizes the family and permutes the outcome574labels, the form transforms by metric pullback, by the chain rule575and reindexing the sum. When the likelihood map has injective576differential this form is positive definite and the descent argument577applies. In overparametrized models its degeneracy can require578passing to identifiable directions. Invariance of the resulting579natural gradient depends on the chosen model or pushforward580\cite{vanoostrum-natural}.581 582In a supervised problem, compatibility of the losses can itself583be tested. Let \(\mu_i\) be a finite distribution on local labelled584examples, let \(F_i\) be the prediction map, and let \(\lambda_i\)585be the example loss. The risk586\[587 \ell_i(\theta)=\sum_{x,y}\mu_i(x,y)588 \lambda_i(F_i(\theta,x),y)589\]590is preserved if the transition transports the example distribution,591intertwines the predictions, and preserves the example loss.592The claim follows by substituting the transported examples in the593sum. This gives an intrinsic risk for data described in several594frames. Distinct datasets require explicitly specifying the risk595to be shared across frames.596 597\subsection{Probabilistic models under transport}598 599Markov categories give an abstract semantics for stochastic kernels,600conditional independence, and disintegration \cite{fritz-markov}.601Quasi-Borel spaces supply cartesian closed structure with probability602distributions, permitting higher-order probabilistic programs such603as random functions \cite{heunen-qbs}. Bayesian lenses describe604composition of prior-dependent inversions when the corresponding605conditionals exist \cite{smithe-bayes}. These frameworks address606probabilistic modeling and inference. Applying them to a data-domain607groupoid requires specifying what a domain change does to outcomes,608states, and probability laws.609 610In the finite case the extra condition can be checked directly.611On domains \(i,j\), take state and observation sets \(P_i,Y_i\) and612\(P_j,Y_j\), normalized priors \(q_i,q_j\), and likelihood kernels613\(K_i,K_j\). A transition consists of bijections \(u:P_i\to P_j\)614and \(v:Y_i\to Y_j\). If \(q_j(u\theta)=q_i(\theta)\) and615\(K_j(vy\mid u\theta)=K_i(y\mid\theta)\), then Bayes' rule commutes616with the transition whenever the observed evidence has positive617probability:618\[619 q_j(u\theta\mid vy)620 =\frac{K_i(y\mid\theta)q_i(\theta)}621 {\sum_{\theta'}K_i(y\mid\theta')q_i(\theta')}622 =q_i(\theta\mid y).623\]624The denominator follows by reindexing the evidence sum along \(u\).625Thus coherent transitions for priors and likelihoods induce coherent626transitions for posteriors wherever these are defined. The positivity