Independent scholarship · Open accessThe monthly digest

Indexed text attachment

representation-theory-of-parametrized-neural-models-on-spaces.tex

Source for Representation Theory of Parametrized Neural Models on Spaces

View paper · Download original file

Lines 661–760 of 1343. Source text is displayed literally.

661\paragraph{Linear layers.}662For a fixed parameter, Peter--Weyl and compact Frobenius663reciprocity decompose \(L^2(E)\) as the Hilbert direct sum of664\(U_\lambda\otimes M_\lambda^E\), where \(U_\lambda\) runs through665irreducibles of \(G\) and666\(M_\lambda^E=\operatorname{Hom}_H(U_\lambda|_H,V)\).667For \(F\) use \(M_\lambda^F=\operatorname{Hom}_K(U_\lambda|_K,W)\).668Give the multiplicity spaces the inner products that make the669isotypic decompositions unitary. Schur's lemma identifies bounded670intertwiners with the blocks \(1_{U_\lambda}\otimes A_\lambda\),671where \(A_\lambda:M_\lambda^E\to M_\lambda^F\) and672\(\sup_\lambda\|A_\lambda\|<\infty\). Finite spectral support673gives smooth kernels by finite sums of matrix coefficients.674A general bounded intertwiner has a distributional kernel by675applying the Schwartz kernel theorem to676\(\Gamma^\infty(E)\to L^2(F)\to\Gamma^{-\infty}(F)\)677\cite[Thm.~1.16]{kottke-lam}. If \(E=F\) and \(X\) is678positive-dimensional with \(E\) of positive rank, the identity679has a diagonal delta kernel. Continuous kernels on the compact680product give Hilbert--Schmidt operators, whereas the identity on681this infinite-dimensional space fails that condition.682 683\begin{example}[A learned axis]684\label{geo:axis}685Take \(P=X=Y=S^2\) with the rotation action and scalar686channels. For \(p\in P\) define \(q_p(x)=p\cdot x\) and687choose the kernel \(L_p(y,x)=3q_p(y)q_p(x)\).688Rotational invariance of Haar probability on the sphere gives689\(\int_{S^2} x_i x_j\,dx=\delta_{ij}/3\), so \(3\|q_p\|_2^2=1\).690The layer \(T_p\) is the orthogonal projection onto691\(\mathbb C q_p\), a line of degree-one spherical harmonics.692Every \(g\) carries \(q_p\) to \(q_{gp}\), hence693\(T_{gp}=gT_pg^{-1}\). A fixed-\(p\) layer commutes with694rotations preserving the axis \(\mathbb{R}p\).695In particular \(T_{-p}=T_p\), and its symmetry group in \(G\)696is strictly larger than the stabilizer of \(p\).697The parameter evaluation identifies opposite poles and realizes698the family of rank-one projections parameterized by699\(\mathbb{RP}^{2}\). This is a model symmetry that arises from700the map that realizes the parameters.701\end{example}702 703\paragraph{Spin channels.}704For \(S^2=\mathrm{SO}(3)/\mathrm{SO}(2)\), let \(L_m\) correspond705to \(\rho_m(e^{it})=e^{imt}\). The \(\ell\)-th spherical type706\(U_\ell\) restricts to the stabilizer with weights707\(-\ell,\ldots,\ell\), each of multiplicity one.708Thus \(L^2(L_m)\) contains one copy of \(U_\ell\) for709\(\ell\geq |m|\). The section representations at weights \(1\)710and \(-1\) are therefore unitarily equivalent. The oriented711complex tangent line is \(L_1\), and \(L_m\) is its712\(m\)-th tensor power, using duals for \(m<0\).713The tangent Euler number \(2\) gives714\[715 c_1(L_m)=2m\,u,\qquad716 \langle u,[S^2]\rangle=1717\]718\cite[Ch.~3]{hatcher-vbkt}. The two opposite channels therefore719carry inequivalent complex bundles. Fiberwise nonlinear layers720detect the characters directly. A continuous fiber map induced721by \(\phi:\mathbb C\to\mathbb C\) from \(L_2\) to \(L_3\) would722obey \(\phi(e^{2it}v)=e^{3it}\phi(v)\). Taking \(t=\pi\) forces723\(\phi=0\). A nonzero global intertwiner from \(L_2\) to \(L_3\)724is obtained by retaining the common \(\ell=3\) block.725The equivariance constraints on the two operations arise at726different levels of the representation.727 728\paragraph{Manifolds with local frames.}729The kernel description also makes sense without a transitive730global action. On a smooth manifold \(M\), let \(Q\to M\) be731a principal \(H\)-bundle and use \(\rho,\sigma\) as its two732feature representations. Local frames of \(Q\) identify a733kernel \(E_x\to F_y\) with a matrix \(A(y,x)\). Replacing734the frames independently by right translates by735\(h_x,h_y\in H\) transforms the matrix by736\[737 A'(y,x)=\sigma(h_y)^{-1} A(y,x)\rho(h_x).738\]739Indeed, new input coordinates must first be converted to the740old ones by \(\rho(h_x)\), and old output coordinates to new741ones by \(\sigma(h_y)^{-1}\). This relation is exactly the742gluing condition for a section of the bundle with fibers743\(\operatorname{Hom}(E_x,F_y)\) on \(M\times M\). Integration744against a chosen smooth positive density defines a global745layer whenever the kernel satisfies the needed integrability746conditions. Setting \(x=y\) yields the corresponding frame747change for a fiber map. A fiberwise nonlinearity can also be748glued if it obeys the same change of coordinates in the sense749of intertwining the two actions.750 751\paragraph{Quotients and descent.}752For a smooth action on a manifold \(P\), the quotient stacks give753\begin{equation}754 [\,(P\times G/H)/G\,]\ \simeq\ [P/H].755 \label{geo:stack}756\end{equation}757An object over a test manifold consists of a right \(G\)-torsor758with a \(G\)-equivariant map to \(P\times G/H\), with inverse759action on the target in the right-torsor convention. The inverse760image of \(1H\) under the second component is an \(H\)-reduction