Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 601–700 of 1074. Source text is displayed literally.

601this calculation under the torsor's changes of frame602recovers its $G$-action. Thus the total evaluation can603be made regular by the bias, while its symmetric604fiber keeps both the stabilizer and the excess605equation of the fixed-parameter model.606\par607\section{Decision boundaries with parameter monodromy}608\label{sec:boundary-monodromy}609 610The regular fibers of a score can carry a permutation611representation of the parameter space's fundamental group.612An invariant computed separately at each parameter can miss613this information. We apply the signed-fiber, relative mapping,614discriminant, and persistence constructions of615\cite[Chs.~9, 18, 20, and~28]{higher-hirsch} to one exact616network whose decision boundary has this property.617 618\subsection{An exact rotation-covariant score}619 620Fix an integer $k\geq2$. Let the input and parameter spaces621both be the unit circle in $\mathbb C$, and consider622\begin{equation}623 s(p,z)=\operatorname{Im}(\overline p\,z^k).624 \label{eq:circle-network-score}625\end{equation}626The real and imaginary coordinates of $z^k$ can be computed627by a finite network with affine maps and the activation628$x\mapsto x^2$. Indeed,629\[630 ab=\tfrac14\bigl((a+b)^2-(a-b)^2\bigr)631\]632implements a real multiplication, hence the real and imaginary633parts of a complex multiplication. Iterating this operation634computes the indicated power. If a scalar needs to pass through635an activation, the identity is636$x=((x+1)^2-(x-1)^2)/4$. With $p=p_1+ip_2$ the final affine637layer reads out $p_1\operatorname{Im}(z^k)-p_2\operatorname{Re}(z^k)$.638Thus $p_1,p_2$ may be treated as the only trainable weights,639constrained to the parameter circle. The classifier thresholds640the score at zero.641 642For $g\in S^1$, act on inputs by $z\mapsto gz$ and on643parameters by $p\mapsto g^kp$. The feature $z^k$ transforms644by the character $g\mapsto g^k$, with its real two-dimensional645form in the network. The final expression pairs that feature646with a parameter of the same character, giving647$s(g^kp,gz)=s(p,z)$. At a fixed $p$ the rotations stabilizing648the parameter are the $k$th roots of unity, denoted $\mu_k$.649They act on the entire zero fiber of the classifier.650 651\subsection{Oriented sheets and their transport}652 653Orient the input circle by increasing angle. In the654coordinates $p=e^{i\theta}$ and $z=e^{i\varphi}$, the score655is $\sin(k\varphi-\theta)$, with input derivative656$k\cos(k\varphi-\theta)$. A boundary point is therefore657regular and has positive or negative derivative according658as $z^k=p$ or $z^k=-p$. Write659\[660 B^\pm=\{(p,z):z^k=\pm p\},\qquad661 B=B^+\amalg B^-,662\]663with projection $\pi:B\to P=S^1$.664 665\begin{proposition}[Permutation of boundary points]666\label{prop:circle-boundary-transport}667Each of $B^+$ and $B^-$ is a connected $k$-sheeted668covering of $P$. The positive generator of $\pi_1(P)$669acts on each signed fiber by a $k$-cycle. Neither covering670admits a continuous section.671\end{proposition}672 673\begin{proof}674Projection onto $z$ identifies each component with a circle.675The projection onto $p$ is then $z\mapsto z^k$ on the positive676component and $z\mapsto -z^k$ on the negative one. Over an677angular interval the roots can be written as678\[679 z_j^\pm(\theta)680   =\exp\!\left(i\,\frac{\theta+c_\pm+2\pi j}{k}\right),681 \qquad c_+=0,\quad c_-=\pi,682\]683for $j$ reduced modulo $k$. Increasing $\theta$ by $2\pi$684sends label $j$ to $j+1$. A continuous section would select685a point fixed by this permutation. Equivalently, its map686into the input circle would have a degree which, when687multiplied by $k$, gives $1$, contradicting integral degree.688\end{proof}689 690Pullback along the parameter cover $r(\alpha)=\alpha^k$691trivializes the positive component.692\[693\begin{tikzcd}[column sep=2.1em,row sep=1.7em]694 S^1\times\mu_k \arrow[r,"h"] \arrow[d,"\operatorname{pr}_1"'] &695 B^+ \arrow[d,"\pi"]\\696 S^1 \arrow[r,"r"'] & P697\end{tikzcd}698\]699Here $h(\alpha,\zeta)=(\alpha^k,\alpha\zeta)$. Choosing a700fixed $k$th root of $-1$ gives the corresponding