Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 401–500 of 1074. Source text is displayed literally.

401 402More generally, for a compact $M$ and a scalar403evaluation $F:P\times M\to\mathbb R$ with $0$ a regular404value, the projection of the zero locus is a proper405submersion off its discriminant. It is a smooth406bundle over each regular chamber in its image407\cite[Ch.~20]{higher-hirsch}. Over the other chambers the408fibers are empty. Local triviality concerns a whole chamber. It leaves409open the possibility of monodromy around a loop in that410chamber, even if the point count is fixed. Section~\ref{sec:boundary-monodromy}411computes that effect in a different network.412\par413\section{A stabilizer obstruction to regular readouts}414\label{sec:eq-readout}415 416A model with moving parameters can be equivariant even when417most of its fixed-parameter maps have few symmetries. Let a418compact Lie group $G$ act smoothly on manifolds $P,X$, and419linearly on a finite-dimensional real space $V$. Suppose a420smooth readout $F:P\times X\to V$ satisfies421\[422 F(gp,gx)=gF(p,x).423\]424A single realization $F_p(x)=F(p,x)$ is equivariant for425$K=G_p$. At a zero $x$ of $F_p$, the subgroup governing its426derivative is $H=K_x=G_p\cap G_x$. The general relation between427stabilizers and normal derivatives is developed in428\cite[Chs.~25--26]{higher-hirsch}.429 430\begin{proposition}[Regularity at a parameter stabilizer]431\label{prop:eq-readout}432If $F_p$ is transverse to zero at $x$, then433$T_xX/T_x(Kx)$ admits a surjective $H$-linear map to $V$.434In particular its $\sigma$-isotypic summand has dimension435at least that of $V_\sigma$ for each real irreducible436$\sigma$ of $H$. Conversely, a surjection of these437representations is realizable as the normal derivative of438a smooth $K$-equivariant map defined near the orbit $Kx$.439\end{proposition}440 441\begin{proof}442For $h\in H$, differentiate $F_p(hx')=hF_p(x')$ at $x$.443The result is $D_x F_p\circ D_xh=h\circ D_xF_p$.444The derivative vanishes on $T_x(Kx)$ since $F_p$ is zero445along the orbit. Surjectivity is preserved when the derivative446factors through this quotient. Because $H$ is compact, an447averaged inner product gives an $H$-invariant complement of448the kernel of a surjection. Therefore a copy of the target449sits in the source representation, with the asserted450inequality on each isotypic part.451 452For the converse, identify $T_xX/T_x(Kx)$ with the normal453representation $N_x$ for an invariant metric. The compact454slice theorem gives an invariant neighborhood of $Kx$455modeled on $K\times_H S$, where $S$ is an invariant456neighborhood of $0$ in $N_x$457\cite[Ch.~25]{higher-hirsch}. For an $H$-linear surjection458$L:N_x\to V$, the assignment $[k,s]\mapsto kL(s)$ is459well defined by equivariance of $L$. Its normal derivative460at $[1,0]$ is $L$, and translation by $K$ transports this461surjectivity along the orbit.462\end{proof}463 464The converse concerns arbitrary equivariant smooth germs.465An architecture restricts which such germs can occur.466In the total space the parameter derivative contributes an467additional $H$-map $T_pP\to V$. Surjectivity at $(p,x)$468tests the sum of its image and the image of the feature469derivative. The two regularity tests correspond to different470domains of the same evaluation.471 472\subsection{Two sign readouts on a torus}473 474Take $G=C_2$ and $X=(\mathbb R/2\pi\mathbb Z)^2$, with475the reflection $r(\theta,\phi)=(-\theta,\phi)$. Write476$\mathbb R_+$ and $\mathbb R_-$ for the trivial and sign477representations. Use two real sign readouts, so that478$V=\mathbb R_-^2$. The fixed locus consists of the two479circles $\theta=0,\pi$. A $G$-equivariant readout vanishes480there because $V^G=0$. On each fixed circle the source481tangent representation is $\mathbb R_-\oplus\mathbb R_+$:482the normal angle changes sign and the angle along the circle483is fixed. Its derivative can have rank at most one by484Proposition~\ref{prop:eq-readout}. Every smooth equivariant485map of this feature and output type therefore has a critical486zero at every point of the fixed locus. Increasing the487number of trainable coefficients within this equivariance488constraint leaves the representation bound in force. A489single sign readout can be regular along the circles, as490is demonstrated by the first output below by itself.491The obstruction concerns the simultaneous pair of492constraints. Increasing output multiplicity makes their493joint zero set a different regularity problem.494 495Consider the specific realization496\[497 f(\theta,\phi)=(\sin\theta,\ \sin\theta\cos\phi).498\]499The coordinate features $u=\sin\theta$ and $v=\cos\phi$500are respectively sign and trivial features. A square