Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 801–900 of 1207. Source text is displayed literally.
801\par802\section{Two uses of quotient and local symmetry}803\label{sec:applications}804 805The first use chooses a nuisance action by a monoidal806topos and computes the resulting space of feasible layers.807The second uses a symmetry that varies as an internal group808and computes the admissible coefficients of a linear network.809 810\subsection{A central action and nuisance-invariant learning}811\label{sec:nuisance}812 813Let $A$ be an abelian discrete group, let $H$ be a814discrete group, and let $\chi:A\to Z(H)$ be a homomorphism.815Write $N=\chi(A)$ and $K=H/N$. The induced action of816$\mathbf B A$ on $\mathbf B H$ has geometric quotient817$q:\mathbf B H\to\mathbf B K$. Its inverse image inflates818a $K$-action along $H\twoheadrightarrow K$.819An $H$-object descends precisely when its $N$-action820is trivial \cite{topos-actions}. This is a literal action821between topoi of the kind discussed in822Section~\ref{sec:introduction}.823 824\begin{proposition}[Orbit features from a central quotient]825\label{prop:orbit-layer}826Let $P,W$ be $H$-sets with trivial $N$-action, and $V$ an827arbitrary $H$-set. An $H$-equivariant map $F:P\times V\to W$828corresponds uniquely to a $K$-equivariant map829$\overline F:P\times(V/N)\to W$.830For topological actions and continuous layers the same831statement holds with the orbit topology on $V/N$.832\end{proposition}833 834\begin{proof}835Equivariance and the hypotheses give836$F(p,nv)=nF(p,v)=F(p,v)$. Therefore $F$ is constant837on the fibers of $(p,v)\mapsto(p,[v])$ and factors.838Normality of $N$ lets $H$ act on the orbits through $K$.839The factor map is consequently equivariant for $K$,840and the converse follows by composing with the quotient.841For continuous actions, $V\to V/N$ is an open surjection842since the saturation of an open set is the union of its843translates. Its product with $P$ is also an open surjection,844so the factor map is continuous by the quotient property.845\end{proof}846 847Suppose now that $N$ is finite and $V,W$ are real linear848representations. For $W$ with trivial $N$-action, an849$N$-invariant \emph{linear} map out of $V$ satisfies850$L=L\Pi_N$, where851\[852 \Pi_N v=\frac{1}{|N|}\sum_{n\in N} n v .853\]854The operator is a projector onto $V^N$, and for an855$H$-representation it commutes with $H$ because $N$ is856normal. The same factorization holds for an affine map857by adding its constant part. The orbit feature $V/N$858instead retains the information needed for all invariant859nonlinear maps. To see the difference, take860$H=C_4$ acting by quarter turns on $V=\mathbb R^2$ and861$N=C_2$ acting by $v\mapsto-v$. This is obtained from862$\chi:C_2\to C_4$ with image the half turn. Here863$\Pi_N=0$. A label on the input set864$\{0,\pm e_1,\pm e_2\}$ indicating whether it is nonzero865is predicted exactly by the invariant square-activation866score $\|v\|^2$. If half of the distribution is at $0$867and half is uniform on the other four points, a predictor868fed only $\Pi_N v$ is constant. Its minimum mean square869error is $1/4$, the variance of the label. The nonlinear870invariant predictor has zero error. The quotient871determines which information may be discarded, and an872implementation still has to choose an orbit encoding.873 874There is a related operation on predictors rather than875input features. If $\mu$ is the joint distribution of876input and label, assume $\mu$ is invariant under877$(v,y)\mapsto(nv,y)$. For a vector score $s$ and a loss878$\ell(y,-)$ convex in the score, define879$\overline s(v)=|N|^{-1}\sum_n s(nv)$.880For measurable scores and a nonnegative loss with finite risk,881Jensen's inequality gives882\begin{align*}883 \mathbb E_\mu[\ell(y,\overline s(v))]884 &\leq \frac1{|N|}\sum_{n\in N}885 \mathbb E_\mu[\ell(y,s(nv))]\\886 &=\mathbb E_\mu[\ell(y,s(v))].887\end{align*}888The averaged predictor descends to $V/N$ by invariance.889If $s$ had an additional $H$-equivariance with $N$ acting890trivially on the output, the factor is $K$-equivariant as891in Proposition~\ref{prop:orbit-layer}.892This is an exact-invariance example of orbit averaging893in data augmentation \cite{chen-augmentation}.894The inequality compares realized predictors. An895architecture must contain the average, for example by896parallel evaluation, before it gives a comparison897within that architecture.898 899\subsection{A filter with a twisted coefficient}900\label{sec:twisted-filter}