Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 554–653 of 1074. Source text is displayed literally.

554\[555 \ell(t,\phi)=\tfrac12\|f(t,\phi)\|^2556       =\tfrac12 t^2(1+\cos^2\phi).557\]558At $t=0$ the Hessian has normal eigenvalue559$1+\cos^2\phi$ and tangential eigenvalue zero in the560chosen coordinate basis. Thus each fixed circle is a561smooth critical manifold with positive-definite Hessian562on its normal line. This loss detects movement away from563the circle. The excess target line survives even though564the loss controls every normal displacement. At a zero,565the general identity $D^2\ell=(Df)^*Df$ identifies the566null directions of the Hessian with $\ker Df$, whereas567the derived tangent complex encodes the cokernel in a568separate degree.569 570\subsection{A moving bias and the symmetric fiber}571 572Take a bias $b\in P=V$ transforming by the same sign action,573and set $F(b,x)=f(x)-b$. The total zero equation is regular574because $D_bF=-I_V$. Its ordinary total incidence space575$Z=\{(b,x):b=f(x)\}$ is the smooth graph of $f$.576Projection $Z\to P$ identifies with $f:X\to V$ and is577singular along its fiber over zero. The parameter $b=0$578has stabilizer $G$, and the readout on this slice is579exactly the original rank-deficient equivariant map.580In contrast, if $0<|b_1|<1$ and $|b_2/b_1|<1$,581the fiber has four points. The equations prescribe582$\sin\theta=b_1$ and $\cos\phi=b_2/b_1$, with two choices583for each angle. The derivative determinant is584$-\cos\theta\sin\theta\sin\phi$, which is nonzero at585all four points. At such a bias the parameter stabilizer586is trivial.587 588The quotient also retains the type of the parameter.589For this finite group, the smooth action stacks give590a map $[Z/G]\to[P/G]$ as in591\cite[Ch.~17]{higher-hirsch}. The zero-parameter orbit592in the base is $BG=[\{0\}/G]$. In the classical smooth593setting the pullback over this orbit is $[C/G]$,594where $C$ is the pair of fixed circles. It has the595product description $C\times BG$. In derived smooth596stacks the homotopy pullback over that orbit is597$[Z_0^{\rm der}/G]$, with $Z_0^{\rm der}$ the derived598zero fiber computed above. Indeed, pulling back along599the atlas of $BG$ trivializes the torsor and gives600the derived incidence square on the manifolds. Gluing601this calculation under the torsor's changes of frame602recovers its $G$-action. Thus the total evaluation can603be made regular by the bias, while its symmetric604fiber keeps both the stabilizer and the excess605equation of the fixed-parameter model.606\par607\section{Decision boundaries with parameter monodromy}608\label{sec:boundary-monodromy}609 610The regular fibers of a score can carry a permutation611representation of the parameter space's fundamental group.612An invariant computed separately at each parameter can miss613this information. We apply the signed-fiber, relative mapping,614discriminant, and persistence constructions of615\cite[Chs.~9, 18, 20, and~28]{higher-hirsch} to one exact616network whose decision boundary has this property.617 618\subsection{An exact rotation-covariant score}619 620Fix an integer $k\geq2$. Let the input and parameter spaces621both be the unit circle in $\mathbb C$, and consider622\begin{equation}623 s(p,z)=\operatorname{Im}(\overline p\,z^k).624 \label{eq:circle-network-score}625\end{equation}626The real and imaginary coordinates of $z^k$ can be computed627by a finite network with affine maps and the activation628$x\mapsto x^2$. Indeed,629\[630 ab=\tfrac14\bigl((a+b)^2-(a-b)^2\bigr)631\]632implements a real multiplication, hence the real and imaginary633parts of a complex multiplication. Iterating this operation634computes the indicated power. If a scalar needs to pass through635an activation, the identity is636$x=((x+1)^2-(x-1)^2)/4$. With $p=p_1+ip_2$ the final affine637layer reads out $p_1\operatorname{Im}(z^k)-p_2\operatorname{Re}(z^k)$.638Thus $p_1,p_2$ may be treated as the only trainable weights,639constrained to the parameter circle. The classifier thresholds640the score at zero.641 642For $g\in S^1$, act on inputs by $z\mapsto gz$ and on643parameters by $p\mapsto g^kp$. The feature $z^k$ transforms644by the character $g\mapsto g^k$, with its real two-dimensional645form in the network. The final expression pairs that feature646with a parameter of the same character, giving647$s(g^kp,gz)=s(p,z)$. At a fixed $p$ the rotations stabilizing648the parameter are the $k$th roots of unity, denoted $\mu_k$.649They act on the entire zero fiber of the classifier.650 651\subsection{Oriented sheets and their transport}652 653Orient the input circle by increasing angle. In the