Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 441–540 of 1074. Source text is displayed literally.

441\begin{proof}442For $h\in H$, differentiate $F_p(hx')=hF_p(x')$ at $x$.443The result is $D_x F_p\circ D_xh=h\circ D_xF_p$.444The derivative vanishes on $T_x(Kx)$ since $F_p$ is zero445along the orbit. Surjectivity is preserved when the derivative446factors through this quotient. Because $H$ is compact, an447averaged inner product gives an $H$-invariant complement of448the kernel of a surjection. Therefore a copy of the target449sits in the source representation, with the asserted450inequality on each isotypic part.451 452For the converse, identify $T_xX/T_x(Kx)$ with the normal453representation $N_x$ for an invariant metric. The compact454slice theorem gives an invariant neighborhood of $Kx$455modeled on $K\times_H S$, where $S$ is an invariant456neighborhood of $0$ in $N_x$457\cite[Ch.~25]{higher-hirsch}. For an $H$-linear surjection458$L:N_x\to V$, the assignment $[k,s]\mapsto kL(s)$ is459well defined by equivariance of $L$. Its normal derivative460at $[1,0]$ is $L$, and translation by $K$ transports this461surjectivity along the orbit.462\end{proof}463 464The converse concerns arbitrary equivariant smooth germs.465An architecture restricts which such germs can occur.466In the total space the parameter derivative contributes an467additional $H$-map $T_pP\to V$. Surjectivity at $(p,x)$468tests the sum of its image and the image of the feature469derivative. The two regularity tests correspond to different470domains of the same evaluation.471 472\subsection{Two sign readouts on a torus}473 474Take $G=C_2$ and $X=(\mathbb R/2\pi\mathbb Z)^2$, with475the reflection $r(\theta,\phi)=(-\theta,\phi)$. Write476$\mathbb R_+$ and $\mathbb R_-$ for the trivial and sign477representations. Use two real sign readouts, so that478$V=\mathbb R_-^2$. The fixed locus consists of the two479circles $\theta=0,\pi$. A $G$-equivariant readout vanishes480there because $V^G=0$. On each fixed circle the source481tangent representation is $\mathbb R_-\oplus\mathbb R_+$:482the normal angle changes sign and the angle along the circle483is fixed. Its derivative can have rank at most one by484Proposition~\ref{prop:eq-readout}. Every smooth equivariant485map of this feature and output type therefore has a critical486zero at every point of the fixed locus. Increasing the487number of trainable coefficients within this equivariance488constraint leaves the representation bound in force. A489single sign readout can be regular along the circles, as490is demonstrated by the first output below by itself.491The obstruction concerns the simultaneous pair of492constraints. Increasing output multiplicity makes their493joint zero set a different regularity problem.494 495Consider the specific realization496\[497 f(\theta,\phi)=(\sin\theta,\ \sin\theta\cos\phi).498\]499The coordinate features $u=\sin\theta$ and $v=\cos\phi$500are respectively sign and trivial features. A square501activation realizes their product using502$uv=((u+v)^2-(u-v)^2)/4$. Under reflection the two inputs503$u+v,u-v$ are swapped and simultaneously negated, so their504squares are swapped and the difference changes sign.505Both readouts therefore have the required transformation506law. The zero set is exactly the pair of fixed circles.507 508\subsection{Functions and infinitesimal equations}509 510Near a fixed circle, $t=\sin\theta$ is a transverse511coordinate, since its derivative in the angular chart is512$1$ at $\theta=0$ and $-1$ at $\theta=\pi$. Write513$c=\cos\phi$ and $A$ for the smooth germ ring on the514ambient chart at a point of the circle. The equations515are $t=0$ and $tc=0$. The ordinary ideal is $(t)$,516so the ordinary fiber has a smooth one-dimensional517function ring on this chart.518 519For the derived fiber, use the smooth Koszul model520\cite[Chs.~21--22]{higher-hirsch}.521Its cochain algebra is $A\otimes\bigwedge(\eta_1,\eta_2)$522with $|\eta_i|=-1$, $d\eta_1=t$, and $d\eta_2=tc$.523The invertible change $\eta_2'=\eta_2-c\eta_1$ gives524$d\eta_2'=0$. Multiplication by $t$ on smooth germs is525injective because $t\ne0$ on a dense open subset of the526chart. The first generator thus resolves $A/(t)$ and527the second gives cohomology528\[529 H^0=A/(t),\qquad530 H^{-1}=(A/(t))\eta_2'.531\]532The remaining negative cohomology vanishes. The generator533$\eta_2'$ transforms by sign. The tangent calculation gives534the same excess on the dual side:535\[536 T_x^{\rm der}537 =\bigl[\,538   \mathbb R_-\oplus\mathbb R_+539   \xrightarrow{\ (a,b)\mapsto(a,ca)\ }540   \mathbb R_-^2