Independent scholarship · Open accessThe monthly digest

Indexed text attachment

topos-actions-in-machine-learning-and-ai.tex

Source for Topos Actions in Machine Learning and Artificial Intelligence

View paper · Download original file

Lines 547–646 of 1207. Source text is displayed literally.

547   g_i=\phi_{ij}^{*}g_j .548\]549Then their gradients are compatible on the overlaps:550\[551  D\phi_{ij}(\operatorname{grad}_{g_i}\ell_i)552     =(\operatorname{grad}_{g_j}\ell_j)\circ\phi_{ij}.553\]554Indeed, pairing the left-hand side with \(D\phi_{ij}(v)\) in \(g_j\)555gives \(g_i(\operatorname{grad}_{g_i}\ell_i,v)=d\ell_i(v)\).556Differentiating the loss identity gives the same pairing for the557right-hand side. Surjectivity of the differential of the transition558and nondegeneracy of the metric give equality. Local gradient flows559therefore respect the transitions wherever defined, by uniqueness of560solutions. A discrete training rule requires its own compatibility.561For instance, a step made using the Riemannian exponential inherits562the compatibility on the domain where it is defined, because the563transitions are local isometries.564 565Natural gradient supplies a relevant source of metrics566\cite{amari-natural}. For a smooth family of strictly positive567probability laws \(p_\theta(y)\) on a finite outcome set, its Fisher568form is569\[570 I_\theta(v,w)571  =\sum_y \frac{d p_\theta(y)(v)\,d p_\theta(y)(w)}{p_\theta(y)} .572\]573If a transition reparametrizes the family and permutes the outcome574labels, the form transforms by metric pullback, by the chain rule575and reindexing the sum. When the likelihood map has injective576differential this form is positive definite and the descent argument577applies. In overparametrized models its degeneracy can require578passing to identifiable directions. Invariance of the resulting579natural gradient depends on the chosen model or pushforward580\cite{vanoostrum-natural}.581 582In a supervised problem, compatibility of the losses can itself583be tested. Let \(\mu_i\) be a finite distribution on local labelled584examples, let \(F_i\) be the prediction map, and let \(\lambda_i\)585be the example loss. The risk586\[587 \ell_i(\theta)=\sum_{x,y}\mu_i(x,y)588                 \lambda_i(F_i(\theta,x),y)589\]590is preserved if the transition transports the example distribution,591intertwines the predictions, and preserves the example loss.592The claim follows by substituting the transported examples in the593sum. This gives an intrinsic risk for data described in several594frames. Distinct datasets require explicitly specifying the risk595to be shared across frames.596 597\subsection{Probabilistic models under transport}598 599Markov categories give an abstract semantics for stochastic kernels,600conditional independence, and disintegration \cite{fritz-markov}.601Quasi-Borel spaces supply cartesian closed structure with probability602distributions, permitting higher-order probabilistic programs such603as random functions \cite{heunen-qbs}. Bayesian lenses describe604composition of prior-dependent inversions when the corresponding605conditionals exist \cite{smithe-bayes}. These frameworks address606probabilistic modeling and inference. Applying them to a data-domain607groupoid requires specifying what a domain change does to outcomes,608states, and probability laws.609 610In the finite case the extra condition can be checked directly.611On domains \(i,j\), take state and observation sets \(P_i,Y_i\) and612\(P_j,Y_j\), normalized priors \(q_i,q_j\), and likelihood kernels613\(K_i,K_j\). A transition consists of bijections \(u:P_i\to P_j\)614and \(v:Y_i\to Y_j\). If \(q_j(u\theta)=q_i(\theta)\) and615\(K_j(vy\mid u\theta)=K_i(y\mid\theta)\), then Bayes' rule commutes616with the transition whenever the observed evidence has positive617probability:618\[619  q_j(u\theta\mid vy)620  =\frac{K_i(y\mid\theta)q_i(\theta)}621         {\sum_{\theta'}K_i(y\mid\theta')q_i(\theta')}622  =q_i(\theta\mid y).623\]624The denominator follows by reindexing the evidence sum along \(u\).625Thus coherent transitions for priors and likelihoods induce coherent626transitions for posteriors wherever these are defined. The positivity627hypothesis matters when specifying pointwise inference. More general628disintegrations may be determined only almost surely and require629care about which version is transported.630 631Compatibility under invertible transports should also be distinguished632from consistency under marginalization. Abramsky and Brandenburger's633contextuality framework studies when families of distributions on634overlapping measurement contexts extend to a global distribution635\cite{abramsky-contextuality}. For an elementary illustration, place636an equal mixture of \((0,1)\) and \((1,0)\) on each of the pairs637\(AB,BC,CA\) of three binary variables. Their singleton marginals638agree. Any joint distribution with these marginals would impose639all three inequalities with probability one, but three binary640variables cannot be pairwise unequal. In multi-view learning this641example identifies a concrete modeling distinction: pairwise calibration642can hold even when there is no joint law realizing the local models.643A useful inference invariant here is existence of a joint law,644measured separately from agreement of the observed marginals.645\par646\section{Descent of equivariant learning data}