Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 543–642 of 1343. Source text is displayed literally.

543additionally uses the chosen law $\nu$ and noise $N$.544If $a:Z\to Z$ preserves $\nu$, then replacing $d_p$ by545$d_p\circ a$ leaves $\mathbb P_p$ unchanged, by the546change-of-variables identity for pushforwards. A law that547observes $z$ and the output together can distinguish the two.548Thus the observation protocol determines which parameter549symmetries are invisible.550 551For an explicit case take $Z=S^1$ with Haar probability and552the ellipse $e(t)=(a\cos t,b\sin t)$ in $\mathbb R^2$ for553$a,b>0$ with $a\ne b$. A hidden phase $\theta$ in554$t\mapsto e(t+\theta)$ is entirely unidentifiable after555integrating out $t$. Adding centered isotropic Gaussian556noise with covariance $\sigma^2 I$ preserves this557invariance. The integrated Gaussian mixture density is558independent of $\theta$, so its Fisher information in559the phase direction is zero. In contrast, an output560orientation in $t\mapsto R_\theta e(t)$ rotates the561signal covariance:562\[563 \operatorname{Cov}_\theta(Y)=564 \tfrac12 R_\theta565 \begin{pmatrix}a^2&0\\0&b^2\end{pmatrix}R_\theta^\top566 +\sigma^2 I .567\]568Because $a^2\ne b^2$, equality of these covariances forces569coincidence of the principal axes and hence equality of570$\theta$ modulo $\pi$. Conversely a rotation by $\pi$ is571the latent shift $t\mapsto t+\pi$ on this ellipse, so572orientations differing by $\pi$ give the same law.573A symmetry of latent coordinates and a rotation of the574observed manifold therefore have distinct statistical effects575even within the same two-dimensional harmonic decoder.576\par577\section{Homogeneous feature bundles}578\label{geo:section}579 580Let \(G\) be a compact Lie group, \(H,K\subseteq G\) closed581subgroups, and \(P\) a topological \(G\)-space of parameters.582Write \(X=G/H\), \(Y=G/K\), and let \(\rho\) and \(\sigma\)583be finite-dimensional unitary representations of \(H\) and \(K\)584on \(V\) and \(W\). Use \(E=G\times_H V\) and \(F=G\times_K W\)585as input and output feature bundles. Our convention is586\([bh,v]=[b,\rho(h)v]\). An input field is therefore encoded by587\(f:G\to V\) with \(f(bh)=\rho(h)^{-1}f(b)\), and a group element \(g\)588acts by left translation \((g f)(b)=f(g^{-1}b)\). Haar probability589induces the invariant measures on \(X,Y\). The fixed-parameter590kernel model is the familiar homogeneous-space convolution591\cite{cohen-geiger-weiler,reprbook}; the parameter action below592allows the kernel itself to move with the symmetry.593 594\begin{proposition}[Kernels with moving parameters]595\label{geo:kernels}596Let \(L_p(y,x):E_x\to F_y\) be jointly continuous in the597parameter and the two base variables, with diagonal covariance598\[599 L_{gp}(gy,gx)=g L_p(y,x)g^{-1}.600\]601Such families correspond bijectively to continuous functions602\(B:P\times G\to\operatorname{Hom}(V,W)\) satisfying603\begin{equation}604 B(kp,kzh)=\sigma(k) B(p,z)\rho(h).605 \label{geo:bi}606\end{equation}607Here \(k\in K,h\in H\). The corresponding family of bounded608linear layers is609\begin{equation}610 (T_p f)(a)=\int_G B(a^{-1}p,a^{-1}b)f(b)\,db .611 \label{geo:conv}612\end{equation}613It satisfies \(T_{gp}(g f)=g(T_p f)\).614\end{proposition}615 616\begin{proof}617Evaluate \(L_p\) at the output basepoint \(1K\) and write618\(L_p(1K,zH)[z,v]=[1,B(p,z)v]\). Changing input representative619\(z\) to \(zh\) introduces \(\rho(h)\). Acting by \(k\in K\)620fixes the output basepoint and acts there by \(\sigma(k)\).621Together these give \eqref{geo:bi}. Conversely, define622\[623 L_p(aK,bH)[b,v]624   =[a,B(a^{-1}p,a^{-1}b)v].625\]626If \(a,b\) are replaced by \(ak,bh\), then the arguments of \(B\)627are replaced by \(k^{-1}a^{-1}p,k^{-1}a^{-1}bh\), and \(v\) by628\(\rho(h)^{-1}v\). The two associated-bundle relations and629\eqref{geo:bi} cancel these changes. Continuity follows in local630sections of \(G\to X,Y\), and diagonal covariance follows since631\((ga)^{-1}gp=a^{-1}p\). The two formulas are inverse.632 633Integrating \(L_p\) over \(X\) gives \eqref{geo:conv}. The integrand634is independent of the input representative, and the output has635the required right \(K\)-transformation. For fixed \(p\) the636kernel is continuous on \(Y\times X\), hence square-integrable637in the Hilbert--Schmidt norm on the finite-dimensional fibers.638Cauchy--Schwarz gives639\(\|T_p\|\leq\|L_p\|_{L^2(Y\times X)}\).640Joint continuity and compactness of \(Y\times X\) give641uniform convergence of \(L_{p'}\) to \(L_p\) as \(p'\to p\),642so \(p\mapsto T_p\) is operator-norm continuous.