Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 601–700 of 1207. Source text is displayed literally.
601Quasi-Borel spaces supply cartesian closed structure with probability602distributions, permitting higher-order probabilistic programs such603as random functions \cite{heunen-qbs}. Bayesian lenses describe604composition of prior-dependent inversions when the corresponding605conditionals exist \cite{smithe-bayes}. These frameworks address606probabilistic modeling and inference. Applying them to a data-domain607groupoid requires specifying what a domain change does to outcomes,608states, and probability laws.609 610In the finite case the extra condition can be checked directly.611On domains \(i,j\), take state and observation sets \(P_i,Y_i\) and612\(P_j,Y_j\), normalized priors \(q_i,q_j\), and likelihood kernels613\(K_i,K_j\). A transition consists of bijections \(u:P_i\to P_j\)614and \(v:Y_i\to Y_j\). If \(q_j(u\theta)=q_i(\theta)\) and615\(K_j(vy\mid u\theta)=K_i(y\mid\theta)\), then Bayes' rule commutes616with the transition whenever the observed evidence has positive617probability:618\[619 q_j(u\theta\mid vy)620 =\frac{K_i(y\mid\theta)q_i(\theta)}621 {\sum_{\theta'}K_i(y\mid\theta')q_i(\theta')}622 =q_i(\theta\mid y).623\]624The denominator follows by reindexing the evidence sum along \(u\).625Thus coherent transitions for priors and likelihoods induce coherent626transitions for posteriors wherever these are defined. The positivity627hypothesis matters when specifying pointwise inference. More general628disintegrations may be determined only almost surely and require629care about which version is transported.630 631Compatibility under invertible transports should also be distinguished632from consistency under marginalization. Abramsky and Brandenburger's633contextuality framework studies when families of distributions on634overlapping measurement contexts extend to a global distribution635\cite{abramsky-contextuality}. For an elementary illustration, place636an equal mixture of \((0,1)\) and \((1,0)\) on each of the pairs637\(AB,BC,CA\) of three binary variables. Their singleton marginals638agree. Any joint distribution with these marginals would impose639all three inequalities with probability one, but three binary640variables cannot be pairwise unequal. In multi-view learning this641example identifies a concrete modeling distinction: pairwise calibration642can hold even when there is no joint law realizing the local models.643A useful inference invariant here is existence of a joint law,644measured separately from agreement of the observed marginals.645\par646\section{Descent of equivariant learning data}647\label{sec:semantic-descent}648 649An equivariant layer may be specified in local coordinates650that change with the observation context. The action topos651gives a category in which to compare these specifications.652Let \(\mathcal E\) be a Grothendieck topos, \(B\) an object,653and \(G_B\) an internal group of \(\mathcal D=\mathcal E/B\).654For example, \(G_B\) can be pulled back from a group on655\(\mathcal E\), or can itself vary over \(B\).656Use \(\mathcal C=B_{\mathcal D}G_B\), the category of657left \(G_B\)-objects in \(\mathcal D\). It is again a658Grothendieck topos \cite{topos-actions}.659Its products are computed over \(B\) with the diagonal660action.661 662A parametrized layer is a morphism663\(h:P\times X\to Y\) in \(\mathcal C\), where \(P\) is the664parameter object. In internal notation the condition is665\[666 h(gp,gx)=g h(p,x).667\]668A fixed parameter accordingly yields an equivariant669predictor for the full group when that parameter is fixed670by the full action. More generally the condition on the671specialized predictor uses the stabilizer of the parameter.672Composition of two layers uses the product of their parameter673objects, with the same diagonal action. Reusing parameters674across layers amounts to inserting an equivariant map into675that product.676 677\subsection{Predictions, losses, and updates}678 679Supervision is an object \(D\) equipped with input and target680maps \(\xi:D\to X\) and \(\tau:D\to Y\). Let \(R\) be a score681object from \(\mathcal D\) with trivial \(G_B\)-action, and682let \(c:Y\times Y\to R\) be an equivariant comparison.683The induced loss is684\[685 \ell:P\times D\to R,\qquad686 \ell(p,d)=c(h(p,\xi(d)),\tau(d)).687\]688Thus \(\ell(gp,gd)=\ell(p,d)\). In \(\mathcal D\) it factors689through the joint orbit object \((P\times D)/G_B\), using690the coequalizer of the action and projection. Passing to the691product of the two orbit objects can lose values of the loss.692For example, in sets take \(P=D=\{1,-1\}\) with the sign693action by \(C_2\), \(X=D\), and \(Y=\{1,-1\}\) with694trivial action.695The layer \(h(p,x)=px\), constant target \(1\), and comparison696\(c(a,b)=(1-ab)/2\) give two loss values on two joint orbits.697The product of the separate orbit objects is a singleton.698An objective \(P\to R\) obtained by averaging the loss699uses a data distribution and an integration operation as700additional data. For a finite data set in sets with