Independent scholarship · Open accessThe monthly digest

Indexed text attachment

topos-actions-in-machine-learning-and-ai.tex

Source for Topos Actions in Machine Learning and Artificial Intelligence

View paper · Download original file

Lines 861–960 of 1207. Source text is displayed literally.

861$N=C_2$ acting by $v\mapsto-v$. This is obtained from862$\chi:C_2\to C_4$ with image the half turn. Here863$\Pi_N=0$. A label on the input set864$\{0,\pm e_1,\pm e_2\}$ indicating whether it is nonzero865is predicted exactly by the invariant square-activation866score $\|v\|^2$. If half of the distribution is at $0$867and half is uniform on the other four points, a predictor868fed only $\Pi_N v$ is constant. Its minimum mean square869error is $1/4$, the variance of the label. The nonlinear870invariant predictor has zero error. The quotient871determines which information may be discarded, and an872implementation still has to choose an orbit encoding.873 874There is a related operation on predictors rather than875input features. If $\mu$ is the joint distribution of876input and label, assume $\mu$ is invariant under877$(v,y)\mapsto(nv,y)$. For a vector score $s$ and a loss878$\ell(y,-)$ convex in the score, define879$\overline s(v)=|N|^{-1}\sum_n s(nv)$.880For measurable scores and a nonnegative loss with finite risk,881Jensen's inequality gives882\begin{align*}883 \mathbb E_\mu[\ell(y,\overline s(v))]884 &\leq \frac1{|N|}\sum_{n\in N}885             \mathbb E_\mu[\ell(y,s(nv))]\\886 &=\mathbb E_\mu[\ell(y,s(v))].887\end{align*}888The averaged predictor descends to $V/N$ by invariance.889If $s$ had an additional $H$-equivariance with $N$ acting890trivially on the output, the factor is $K$-equivariant as891in Proposition~\ref{prop:orbit-layer}.892This is an exact-invariance example of orbit averaging893in data augmentation \cite{chen-augmentation}.894The inequality compares realized predictors. An895architecture must contain the average, for example by896parallel evaluation, before it gives a comparison897within that architecture.898 899\subsection{A filter with a twisted coefficient}900\label{sec:twisted-filter}901 902Let the deployment base be $B=S^1=\mathbb R/(2\pi\mathbb Z)$.903In $\operatorname{Sh}(B)$ let $G$ be the locally constant904group sheaf with fiber $C_3=\mathbb Z/3$ and monodromy905$j\mapsto-j$. Only $0$ is fixed by inversion, so906$\Gamma(B,G)$ is the trivial group. The local groups are907nevertheless nontrivial.908 909Use real regular-representation feature fibers910$V_t=\mathbb R[C_3]$ with basis $(e_j)_{j\in C_3}$.911The monodromy on the bundle $V$ is the orthogonal912permutation $S e_j=e_{-j}$. Let $R e_j=e_{j+1}$913be the group shift in a local lift of the angular914coordinate. Then $SRS^{-1}=R^{-1}$, which makes the915regular action compatible with the group sheaf's916transition. The sheaf of smooth sections of $V$917is an internal $G$-object. Its fiberwise $G$-linear918maps define equivariant linear layers in the relative919classifying topos.920 921\begin{proposition}[The coefficient bundle of equivariant filters]922\label{prop:twisted}923The bundle of fiberwise linear intertwiners from $V$924to itself is $\mathbb R^2\oplus L$ over $S^1$, where925$L$ is the real M\"obius line and $\mathbb R^2$ denotes926a trivial rank-two bundle. A flat filter, meaning927locally constant coefficients in the given flat928frames, has two free real coefficients. Smooth929filters can also use the third coefficient as a930section of $L$.931\end{proposition}932 933\begin{proof}934A linear map commuting with $R$ is determined by935its value $w$ on $e_0$, since its values on the other936basis vectors must be $Rw$ and $R^2w$. Every $w$937gives a map by this rule. Hence the commutant consists938precisely of $c_0 I+c_1R+c_2R^2$. Changing frame by939$S$ exchanges $c_1,c_2$. In the basis940$I,R+R^2,R-R^2$ it is the identity on the first941two directions and multiplication by $-1$ on the942third. This gives the bundle splitting and the943flat coefficient count.944\end{proof}945 946The distinction affects model capacity. Using the947global section group alone would impose no commutation948conditions and would allow all nine entries of a local949matrix. Requiring internal equivariance leaves three950local coefficients. Choosing flat global coefficients951ties the two shifted terms together, leaving two.952For smooth models the third coefficient $b(t)$ can953be any smooth function on the lifted angle satisfying954$b(t+2\pi)=-b(t)$. For example955$b(t)=\cos(t/2)$ gives a well-defined global filter956$b(t)(R-R^2)$. Every continuous such $b$ has a zero:957if $b(t)$ is nonzero, the values at $t$ and958$t+2\pi$ have opposite signs. Hence a model that959requires this coefficient to stay nonzero can work over960the orientation double cover, where the pulled-back