Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 681–780 of 1343. Source text is displayed literally.

681this infinite-dimensional space fails that condition.682 683\begin{example}[A learned axis]684\label{geo:axis}685Take \(P=X=Y=S^2\) with the rotation action and scalar686channels. For \(p\in P\) define \(q_p(x)=p\cdot x\) and687choose the kernel \(L_p(y,x)=3q_p(y)q_p(x)\).688Rotational invariance of Haar probability on the sphere gives689\(\int_{S^2} x_i x_j\,dx=\delta_{ij}/3\), so \(3\|q_p\|_2^2=1\).690The layer \(T_p\) is the orthogonal projection onto691\(\mathbb C q_p\), a line of degree-one spherical harmonics.692Every \(g\) carries \(q_p\) to \(q_{gp}\), hence693\(T_{gp}=gT_pg^{-1}\). A fixed-\(p\) layer commutes with694rotations preserving the axis \(\mathbb{R}p\).695In particular \(T_{-p}=T_p\), and its symmetry group in \(G\)696is strictly larger than the stabilizer of \(p\).697The parameter evaluation identifies opposite poles and realizes698the family of rank-one projections parameterized by699\(\mathbb{RP}^{2}\). This is a model symmetry that arises from700the map that realizes the parameters.701\end{example}702 703\paragraph{Spin channels.}704For \(S^2=\mathrm{SO}(3)/\mathrm{SO}(2)\), let \(L_m\) correspond705to \(\rho_m(e^{it})=e^{imt}\). The \(\ell\)-th spherical type706\(U_\ell\) restricts to the stabilizer with weights707\(-\ell,\ldots,\ell\), each of multiplicity one.708Thus \(L^2(L_m)\) contains one copy of \(U_\ell\) for709\(\ell\geq |m|\). The section representations at weights \(1\)710and \(-1\) are therefore unitarily equivalent. The oriented711complex tangent line is \(L_1\), and \(L_m\) is its712\(m\)-th tensor power, using duals for \(m<0\).713The tangent Euler number \(2\) gives714\[715 c_1(L_m)=2m\,u,\qquad716 \langle u,[S^2]\rangle=1717\]718\cite[Ch.~3]{hatcher-vbkt}. The two opposite channels therefore719carry inequivalent complex bundles. Fiberwise nonlinear layers720detect the characters directly. A continuous fiber map induced721by \(\phi:\mathbb C\to\mathbb C\) from \(L_2\) to \(L_3\) would722obey \(\phi(e^{2it}v)=e^{3it}\phi(v)\). Taking \(t=\pi\) forces723\(\phi=0\). A nonzero global intertwiner from \(L_2\) to \(L_3\)724is obtained by retaining the common \(\ell=3\) block.725The equivariance constraints on the two operations arise at726different levels of the representation.727 728\paragraph{Manifolds with local frames.}729The kernel description also makes sense without a transitive730global action. On a smooth manifold \(M\), let \(Q\to M\) be731a principal \(H\)-bundle and use \(\rho,\sigma\) as its two732feature representations. Local frames of \(Q\) identify a733kernel \(E_x\to F_y\) with a matrix \(A(y,x)\). Replacing734the frames independently by right translates by735\(h_x,h_y\in H\) transforms the matrix by736\[737 A'(y,x)=\sigma(h_y)^{-1} A(y,x)\rho(h_x).738\]739Indeed, new input coordinates must first be converted to the740old ones by \(\rho(h_x)\), and old output coordinates to new741ones by \(\sigma(h_y)^{-1}\). This relation is exactly the742gluing condition for a section of the bundle with fibers743\(\operatorname{Hom}(E_x,F_y)\) on \(M\times M\). Integration744against a chosen smooth positive density defines a global745layer whenever the kernel satisfies the needed integrability746conditions. Setting \(x=y\) yields the corresponding frame747change for a fiber map. A fiberwise nonlinearity can also be748glued if it obeys the same change of coordinates in the sense749of intertwining the two actions.750 751\paragraph{Quotients and descent.}752For a smooth action on a manifold \(P\), the quotient stacks give753\begin{equation}754 [\,(P\times G/H)/G\,]\ \simeq\ [P/H].755 \label{geo:stack}756\end{equation}757An object over a test manifold consists of a right \(G\)-torsor758with a \(G\)-equivariant map to \(P\times G/H\), with inverse759action on the target in the right-torsor convention. The inverse760image of \(1H\) under the second component is an \(H\)-reduction761of the torsor. The first component restricts to an equivariant762map from that reduction to \(P\). Conversely, extending the763\(H\)-torsor along \(H\hookrightarrow G\) produces the original764object and both maps. The constructions commute with pullback765and identify torsor isomorphisms \cite[Chs.~20, 28]{reprbook}.766For \(P=*\) this gives the smooth classifying stack \(BH\). A representation \(H\to767\mathrm{U}(V)\) then gives a morphism of smooth classifying768stacks and a descended feature bundle. More generally a769homotopy-coherent coefficient system on an action groupoid770retains the diagrams of identifications over all composable771arrows. Formulas \eqref{geo:bi} and \eqref{geo:conv} illustrate772the ordinary continuous case with a chosen invariant measure.773The descent data determine how to transport fibers, while the774measure enters the integration operation separately.775\par776\section{Harmonic decomposition with moving weights}777\label{sec:harmonics}778 779A fixed-parameter equivariant layer can mix multiplicities of an780irreducible representation but preserves its isotypic component.