Independent scholarship · Open accessThe monthly digest

Indexed text attachment

topos-actions-in-machine-learning-and-ai.tex

Source for Topos Actions in Machine Learning and Artificial Intelligence

View paper · Download original file

Lines 647–746 of 1207. Source text is displayed literally.

647\label{sec:semantic-descent}648 649An equivariant layer may be specified in local coordinates650that change with the observation context. The action topos651gives a category in which to compare these specifications.652Let \(\mathcal E\) be a Grothendieck topos, \(B\) an object,653and \(G_B\) an internal group of \(\mathcal D=\mathcal E/B\).654For example, \(G_B\) can be pulled back from a group on655\(\mathcal E\), or can itself vary over \(B\).656Use \(\mathcal C=B_{\mathcal D}G_B\), the category of657left \(G_B\)-objects in \(\mathcal D\). It is again a658Grothendieck topos \cite{topos-actions}.659Its products are computed over \(B\) with the diagonal660action.661 662A parametrized layer is a morphism663\(h:P\times X\to Y\) in \(\mathcal C\), where \(P\) is the664parameter object. In internal notation the condition is665\[666 h(gp,gx)=g h(p,x).667\]668A fixed parameter accordingly yields an equivariant669predictor for the full group when that parameter is fixed670by the full action. More generally the condition on the671specialized predictor uses the stabilizer of the parameter.672Composition of two layers uses the product of their parameter673objects, with the same diagonal action. Reusing parameters674across layers amounts to inserting an equivariant map into675that product.676 677\subsection{Predictions, losses, and updates}678 679Supervision is an object \(D\) equipped with input and target680maps \(\xi:D\to X\) and \(\tau:D\to Y\). Let \(R\) be a score681object from \(\mathcal D\) with trivial \(G_B\)-action, and682let \(c:Y\times Y\to R\) be an equivariant comparison.683The induced loss is684\[685 \ell:P\times D\to R,\qquad686 \ell(p,d)=c(h(p,\xi(d)),\tau(d)).687\]688Thus \(\ell(gp,gd)=\ell(p,d)\). In \(\mathcal D\) it factors689through the joint orbit object \((P\times D)/G_B\), using690the coequalizer of the action and projection. Passing to the691product of the two orbit objects can lose values of the loss.692For example, in sets take \(P=D=\{1,-1\}\) with the sign693action by \(C_2\), \(X=D\), and \(Y=\{1,-1\}\) with694trivial action.695The layer \(h(p,x)=px\), constant target \(1\), and comparison696\(c(a,b)=(1-ab)/2\) give two loss values on two joint orbits.697The product of the separate orbit objects is a singleton.698An objective \(P\to R\) obtained by averaging the loss699uses a data distribution and an integration operation as700additional data. For a finite data set in sets with701real scores and probability weights \(\mu(d)\), the risk702is \(\sum_{d}\mu(d)\ell(p,d)\). If703\(\mu(gd)=\mu(d)\), simultaneous invariance of the704loss and reindexing the sum show that this risk is705invariant on \(P\).706 707An update is separate data \(u:P\times D\to P\) in708\(\mathcal C\). Equivariance now says709\(u(gp,gd)=g\,u(p,d)\). With the same invariant finite710sampling law, the law of the next parameter at \(gp\)711is the pushforward along \(g\) of the corresponding law712at \(p\). This follows by replacing the random sample713\(d\) by \(gd\), which has the same law.714For a parameter change \(r:P'\to P\),715the predictor reparametrizes by precomposition. An update716\(u'\) on \(P'\) is compatible if it satisfies717\[718 r\circ u'=u\circ(r\times\mathrm{id}_D).719\]720An isomorphism \(r\) transports the update uniquely by this721equation. The equation is an additional requirement for a722general \(r\), where precomposition alone leaves the result723of the old update in \(P\). For example, the update724\(p\mapsto p+1\) on \(\mathbb R\) has no compatible lift725to the single parameter value \(0\).726 727\subsection{A local-to-global criterion}728 729\begin{theorem}[Descent of layers and updates]730Let \(U\to 1\) be an epimorphism in \(\mathcal C\).731Suppose the objects \(P,X,Y,D\) are given in732\(\mathcal C/U\) with descent isomorphisms satisfying the733cocycle on \(U\times U\times U\). Give the restriction734of the trivial-action score object \(R\) its canonical735descent isomorphism. The local layer, input and target maps,736comparison, and update descend precisely when each map737commutes with the specified descent isomorphisms. The738global diagram is unique up to a unique compatible739isomorphism. The induced local loss descends through740the same comparison maps.741\end{theorem}742 743\begin{proof}744In a Grothendieck topos an epimorphism is effective745descent for objects. To recall the construction, the746transport on \(A\to U\) over \(U\times U\) identifies