Indexed text attachment
representation-theory-of-parametrized-neural-models-on-spaces.tex
Source for Representation Theory of Parametrized Neural Models on Spaces
View paper · Download original file
Lines 643–742 of 1343. Source text is displayed literally.
643Changing variables \(b\mapsto gb\) proves the stated operator644covariance. If two continuous kernels give the same operators,645testing in local frames against smooth sections supported in646coordinate charts identifies their distributional pairings.647The resulting equality almost everywhere extends everywhere648because the measures have full support.649\end{proof}650 651When \(G\) fixes \(p\), each \(T_p\) is an intertwiner.652Equation~\eqref{geo:bi} then constrains the permissible value at653\(z\) to intertwiners for \(K\cap zHz^{-1}\), where the input654representation is transported by \(k\mapsto z^{-1}kz\).655This follows by setting \(h=z^{-1}k^{-1}z\) for a \(k\) in the656intersection. When a parameter moves, the analogous constraint657uses \(K_p\cap zHz^{-1}\), where \(K_p\) fixes \(p\). Thus a658parameter with a larger stabilizer can impose more equations on659the kernel at an orbit representative.660 661\paragraph{Linear layers.}662For a fixed parameter, Peter--Weyl and compact Frobenius663reciprocity decompose \(L^2(E)\) as the Hilbert direct sum of664\(U_\lambda\otimes M_\lambda^E\), where \(U_\lambda\) runs through665irreducibles of \(G\) and666\(M_\lambda^E=\operatorname{Hom}_H(U_\lambda|_H,V)\).667For \(F\) use \(M_\lambda^F=\operatorname{Hom}_K(U_\lambda|_K,W)\).668Give the multiplicity spaces the inner products that make the669isotypic decompositions unitary. Schur's lemma identifies bounded670intertwiners with the blocks \(1_{U_\lambda}\otimes A_\lambda\),671where \(A_\lambda:M_\lambda^E\to M_\lambda^F\) and672\(\sup_\lambda\|A_\lambda\|<\infty\). Finite spectral support673gives smooth kernels by finite sums of matrix coefficients.674A general bounded intertwiner has a distributional kernel by675applying the Schwartz kernel theorem to676\(\Gamma^\infty(E)\to L^2(F)\to\Gamma^{-\infty}(F)\)677\cite[Thm.~1.16]{kottke-lam}. If \(E=F\) and \(X\) is678positive-dimensional with \(E\) of positive rank, the identity679has a diagonal delta kernel. Continuous kernels on the compact680product give Hilbert--Schmidt operators, whereas the identity on681this infinite-dimensional space fails that condition.682 683\begin{example}[A learned axis]684\label{geo:axis}685Take \(P=X=Y=S^2\) with the rotation action and scalar686channels. For \(p\in P\) define \(q_p(x)=p\cdot x\) and687choose the kernel \(L_p(y,x)=3q_p(y)q_p(x)\).688Rotational invariance of Haar probability on the sphere gives689\(\int_{S^2} x_i x_j\,dx=\delta_{ij}/3\), so \(3\|q_p\|_2^2=1\).690The layer \(T_p\) is the orthogonal projection onto691\(\mathbb C q_p\), a line of degree-one spherical harmonics.692Every \(g\) carries \(q_p\) to \(q_{gp}\), hence693\(T_{gp}=gT_pg^{-1}\). A fixed-\(p\) layer commutes with694rotations preserving the axis \(\mathbb{R}p\).695In particular \(T_{-p}=T_p\), and its symmetry group in \(G\)696is strictly larger than the stabilizer of \(p\).697The parameter evaluation identifies opposite poles and realizes698the family of rank-one projections parameterized by699\(\mathbb{RP}^{2}\). This is a model symmetry that arises from700the map that realizes the parameters.701\end{example}702 703\paragraph{Spin channels.}704For \(S^2=\mathrm{SO}(3)/\mathrm{SO}(2)\), let \(L_m\) correspond705to \(\rho_m(e^{it})=e^{imt}\). The \(\ell\)-th spherical type706\(U_\ell\) restricts to the stabilizer with weights707\(-\ell,\ldots,\ell\), each of multiplicity one.708Thus \(L^2(L_m)\) contains one copy of \(U_\ell\) for709\(\ell\geq |m|\). The section representations at weights \(1\)710and \(-1\) are therefore unitarily equivalent. The oriented711complex tangent line is \(L_1\), and \(L_m\) is its712\(m\)-th tensor power, using duals for \(m<0\).713The tangent Euler number \(2\) gives714\[715 c_1(L_m)=2m\,u,\qquad716 \langle u,[S^2]\rangle=1717\]718\cite[Ch.~3]{hatcher-vbkt}. The two opposite channels therefore719carry inequivalent complex bundles. Fiberwise nonlinear layers720detect the characters directly. A continuous fiber map induced721by \(\phi:\mathbb C\to\mathbb C\) from \(L_2\) to \(L_3\) would722obey \(\phi(e^{2it}v)=e^{3it}\phi(v)\). Taking \(t=\pi\) forces723\(\phi=0\). A nonzero global intertwiner from \(L_2\) to \(L_3\)724is obtained by retaining the common \(\ell=3\) block.725The equivariance constraints on the two operations arise at726different levels of the representation.727 728\paragraph{Manifolds with local frames.}729The kernel description also makes sense without a transitive730global action. On a smooth manifold \(M\), let \(Q\to M\) be731a principal \(H\)-bundle and use \(\rho,\sigma\) as its two732feature representations. Local frames of \(Q\) identify a733kernel \(E_x\to F_y\) with a matrix \(A(y,x)\). Replacing734the frames independently by right translates by735\(h_x,h_y\in H\) transforms the matrix by736\[737 A'(y,x)=\sigma(h_y)^{-1} A(y,x)\rho(h_x).738\]739Indeed, new input coordinates must first be converted to the740old ones by \(\rho(h_x)\), and old output coordinates to new741ones by \(\sigma(h_y)^{-1}\). This relation is exactly the742gluing condition for a section of the bundle with fibers