Indexed text attachment
representation-theory-of-parametrized-neural-models-on-spaces.tex
Source for Representation Theory of Parametrized Neural Models on Spaces
View paper · Download original file
Lines 583–682 of 1343. Source text is displayed literally.
583be finite-dimensional unitary representations of \(H\) and \(K\)584on \(V\) and \(W\). Use \(E=G\times_H V\) and \(F=G\times_K W\)585as input and output feature bundles. Our convention is586\([bh,v]=[b,\rho(h)v]\). An input field is therefore encoded by587\(f:G\to V\) with \(f(bh)=\rho(h)^{-1}f(b)\), and a group element \(g\)588acts by left translation \((g f)(b)=f(g^{-1}b)\). Haar probability589induces the invariant measures on \(X,Y\). The fixed-parameter590kernel model is the familiar homogeneous-space convolution591\cite{cohen-geiger-weiler,reprbook}; the parameter action below592allows the kernel itself to move with the symmetry.593 594\begin{proposition}[Kernels with moving parameters]595\label{geo:kernels}596Let \(L_p(y,x):E_x\to F_y\) be jointly continuous in the597parameter and the two base variables, with diagonal covariance598\[599 L_{gp}(gy,gx)=g L_p(y,x)g^{-1}.600\]601Such families correspond bijectively to continuous functions602\(B:P\times G\to\operatorname{Hom}(V,W)\) satisfying603\begin{equation}604 B(kp,kzh)=\sigma(k) B(p,z)\rho(h).605 \label{geo:bi}606\end{equation}607Here \(k\in K,h\in H\). The corresponding family of bounded608linear layers is609\begin{equation}610 (T_p f)(a)=\int_G B(a^{-1}p,a^{-1}b)f(b)\,db .611 \label{geo:conv}612\end{equation}613It satisfies \(T_{gp}(g f)=g(T_p f)\).614\end{proposition}615 616\begin{proof}617Evaluate \(L_p\) at the output basepoint \(1K\) and write618\(L_p(1K,zH)[z,v]=[1,B(p,z)v]\). Changing input representative619\(z\) to \(zh\) introduces \(\rho(h)\). Acting by \(k\in K\)620fixes the output basepoint and acts there by \(\sigma(k)\).621Together these give \eqref{geo:bi}. Conversely, define622\[623 L_p(aK,bH)[b,v]624 =[a,B(a^{-1}p,a^{-1}b)v].625\]626If \(a,b\) are replaced by \(ak,bh\), then the arguments of \(B\)627are replaced by \(k^{-1}a^{-1}p,k^{-1}a^{-1}bh\), and \(v\) by628\(\rho(h)^{-1}v\). The two associated-bundle relations and629\eqref{geo:bi} cancel these changes. Continuity follows in local630sections of \(G\to X,Y\), and diagonal covariance follows since631\((ga)^{-1}gp=a^{-1}p\). The two formulas are inverse.632 633Integrating \(L_p\) over \(X\) gives \eqref{geo:conv}. The integrand634is independent of the input representative, and the output has635the required right \(K\)-transformation. For fixed \(p\) the636kernel is continuous on \(Y\times X\), hence square-integrable637in the Hilbert--Schmidt norm on the finite-dimensional fibers.638Cauchy--Schwarz gives639\(\|T_p\|\leq\|L_p\|_{L^2(Y\times X)}\).640Joint continuity and compactness of \(Y\times X\) give641uniform convergence of \(L_{p'}\) to \(L_p\) as \(p'\to p\),642so \(p\mapsto T_p\) is operator-norm continuous.643Changing variables \(b\mapsto gb\) proves the stated operator644covariance. If two continuous kernels give the same operators,645testing in local frames against smooth sections supported in646coordinate charts identifies their distributional pairings.647The resulting equality almost everywhere extends everywhere648because the measures have full support.649\end{proof}650 651When \(G\) fixes \(p\), each \(T_p\) is an intertwiner.652Equation~\eqref{geo:bi} then constrains the permissible value at653\(z\) to intertwiners for \(K\cap zHz^{-1}\), where the input654representation is transported by \(k\mapsto z^{-1}kz\).655This follows by setting \(h=z^{-1}k^{-1}z\) for a \(k\) in the656intersection. When a parameter moves, the analogous constraint657uses \(K_p\cap zHz^{-1}\), where \(K_p\) fixes \(p\). Thus a658parameter with a larger stabilizer can impose more equations on659the kernel at an orbit representative.660 661\paragraph{Linear layers.}662For a fixed parameter, Peter--Weyl and compact Frobenius663reciprocity decompose \(L^2(E)\) as the Hilbert direct sum of664\(U_\lambda\otimes M_\lambda^E\), where \(U_\lambda\) runs through665irreducibles of \(G\) and666\(M_\lambda^E=\operatorname{Hom}_H(U_\lambda|_H,V)\).667For \(F\) use \(M_\lambda^F=\operatorname{Hom}_K(U_\lambda|_K,W)\).668Give the multiplicity spaces the inner products that make the669isotypic decompositions unitary. Schur's lemma identifies bounded670intertwiners with the blocks \(1_{U_\lambda}\otimes A_\lambda\),671where \(A_\lambda:M_\lambda^E\to M_\lambda^F\) and672\(\sup_\lambda\|A_\lambda\|<\infty\). Finite spectral support673gives smooth kernels by finite sums of matrix coefficients.674A general bounded intertwiner has a distributional kernel by675applying the Schwartz kernel theorem to676\(\Gamma^\infty(E)\to L^2(F)\to\Gamma^{-\infty}(F)\)677\cite[Thm.~1.16]{kottke-lam}. If \(E=F\) and \(X\) is678positive-dimensional with \(E\) of positive rank, the identity679has a diagonal delta kernel. Continuous kernels on the compact680product give Hilbert--Schmidt operators, whereas the identity on681this infinite-dimensional space fails that condition.682