Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 521–620 of 1343. Source text is displayed literally.

521origin and the two punctured coordinate axes in the \((b,c)\)-plane.522Thus the quotient groupoid of parameter symmetries contains523information beyond the set of distinct output laws. Even the524infinitesimal null space can include directions that change the525output at higher order: along \((a,\varepsilon,\varepsilon)\) through526\((a,0,0)\), \(d=\varepsilon^2\) changes the prediction to second order.527Consequently equality of output laws, Fisher-null directions, and528parameter equivalence answer three different questions.529 530\subsection{A manifold decoder as a statistical model}531 532Let $Z$ be a latent manifold with fixed probability $\nu$ and533$d:P\times Z\to M$ a smooth decoder. In the absence of observation534noise the law at $p$ is $(d_p)_*\nu$. A noise channel $N:M\to Y$535then gives the observed law536\[537 \mathbb P_p(A)=\int_Z N(d_p(z),A)\,d\nu(z)538\]539for measurable $A\subseteq Y$. This is ordinary composition of540Markov kernels on these measurable spaces. The manifold541evaluation $d$ can be represented by the smooth slice542construction of Section~\ref{sec:relative}. The pushforward543additionally uses the chosen law $\nu$ and noise $N$.544If $a:Z\to Z$ preserves $\nu$, then replacing $d_p$ by545$d_p\circ a$ leaves $\mathbb P_p$ unchanged, by the546change-of-variables identity for pushforwards. A law that547observes $z$ and the output together can distinguish the two.548Thus the observation protocol determines which parameter549symmetries are invisible.550 551For an explicit case take $Z=S^1$ with Haar probability and552the ellipse $e(t)=(a\cos t,b\sin t)$ in $\mathbb R^2$ for553$a,b>0$ with $a\ne b$. A hidden phase $\theta$ in554$t\mapsto e(t+\theta)$ is entirely unidentifiable after555integrating out $t$. Adding centered isotropic Gaussian556noise with covariance $\sigma^2 I$ preserves this557invariance. The integrated Gaussian mixture density is558independent of $\theta$, so its Fisher information in559the phase direction is zero. In contrast, an output560orientation in $t\mapsto R_\theta e(t)$ rotates the561signal covariance:562\[563 \operatorname{Cov}_\theta(Y)=564 \tfrac12 R_\theta565 \begin{pmatrix}a^2&0\\0&b^2\end{pmatrix}R_\theta^\top566 +\sigma^2 I .567\]568Because $a^2\ne b^2$, equality of these covariances forces569coincidence of the principal axes and hence equality of570$\theta$ modulo $\pi$. Conversely a rotation by $\pi$ is571the latent shift $t\mapsto t+\pi$ on this ellipse, so572orientations differing by $\pi$ give the same law.573A symmetry of latent coordinates and a rotation of the574observed manifold therefore have distinct statistical effects575even within the same two-dimensional harmonic decoder.576\par577\section{Homogeneous feature bundles}578\label{geo:section}579 580Let \(G\) be a compact Lie group, \(H,K\subseteq G\) closed581subgroups, and \(P\) a topological \(G\)-space of parameters.582Write \(X=G/H\), \(Y=G/K\), and let \(\rho\) and \(\sigma\)583be finite-dimensional unitary representations of \(H\) and \(K\)584on \(V\) and \(W\). Use \(E=G\times_H V\) and \(F=G\times_K W\)585as input and output feature bundles. Our convention is586\([bh,v]=[b,\rho(h)v]\). An input field is therefore encoded by587\(f:G\to V\) with \(f(bh)=\rho(h)^{-1}f(b)\), and a group element \(g\)588acts by left translation \((g f)(b)=f(g^{-1}b)\). Haar probability589induces the invariant measures on \(X,Y\). The fixed-parameter590kernel model is the familiar homogeneous-space convolution591\cite{cohen-geiger-weiler,reprbook}; the parameter action below592allows the kernel itself to move with the symmetry.593 594\begin{proposition}[Kernels with moving parameters]595\label{geo:kernels}596Let \(L_p(y,x):E_x\to F_y\) be jointly continuous in the597parameter and the two base variables, with diagonal covariance598\[599 L_{gp}(gy,gx)=g L_p(y,x)g^{-1}.600\]601Such families correspond bijectively to continuous functions602\(B:P\times G\to\operatorname{Hom}(V,W)\) satisfying603\begin{equation}604 B(kp,kzh)=\sigma(k) B(p,z)\rho(h).605 \label{geo:bi}606\end{equation}607Here \(k\in K,h\in H\). The corresponding family of bounded608linear layers is609\begin{equation}610 (T_p f)(a)=\int_G B(a^{-1}p,a^{-1}b)f(b)\,db .611 \label{geo:conv}612\end{equation}613It satisfies \(T_{gp}(g f)=g(T_p f)\).614\end{proposition}615 616\begin{proof}617Evaluate \(L_p\) at the output basepoint \(1K\) and write618\(L_p(1K,zH)[z,v]=[1,B(p,z)v]\). Changing input representative619\(z\) to \(zh\) introduces \(\rho(h)\). Acting by \(k\in K\)620fixes the output basepoint and acts there by \(\sigma(k)\).