Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 781–880 of 1074. Source text is displayed literally.

781is zero. The sublevel diagram over $P$ additionally records782which components are identified by going around a loop.783For example, the constant family of scores784$s_0(p,z)=\operatorname{Im}(z^k)$ has the same barcode at785every parameter. Its component local system is trivial,786whereas the generator of the parameter loop acts on the787components for $s$ by the nonidentity permutation.788The two filtered diagrams therefore cannot be equivalent789over $P$. For two general scores on the same parameter790and input spaces, a uniform bound on their difference791also gives the shifted inclusions of the \emph{total}792sublevels over $P$. This makes the sublevel diagram over793the parameter space a stable comparison object while794still retaining the representation.795\par796\section{Reconstruction through a Euclidean latent space}797\label{sec:reconstruction}798 799Let \(X\) be a nonempty compact smooth manifold without boundary,800and fix an embedding \(i:X\hookrightarrow\mathbb R^D\).801An encoder \(e:X\to\mathbb R^m\) and ambient decoder802\(d:\mathbb R^m\to\mathbb R^D\) realize the reconstruction803\(x\mapsto d(e(x))\). Here the latent width \(m\) is the dimension804of the intermediate Euclidean space. The encoder and decoder may be805evaluation maps of neural architectures at selected parameters.806We first allow arbitrary smooth maps to isolate the topological807constraints on any such architecture. This applies the tube and808factorization constructions of \cite[Chs.~8, 24, 27]{higher-hirsch}.809For related effects of data topology on autoencoding, see810\cite{batson-autoencoders}.811 812\subsection{Exact recovery and smooth families}813 814\begin{theorem}[Smooth ambient reconstruction]815\label{thm:ambient-reconstruction}816A smooth encoder \(e:X\to\mathbb R^m\) admits a smooth817ambient decoder on all of \(\mathbb R^m\) satisfying818\(d\circ e=i\) if and only if \(e\) is a smooth embedding.819Moreover \(e\) extends to a smooth encoder on all of820\(\mathbb R^D\) agreeing with it along \(i(X)\).821\end{theorem}822 823\begin{proof}824If \(d\circ e=i\), injectivity of \(i\) forces injectivity of \(e\).825Differentiating gives \(D_x i=D_{e(x)}d\,D_x e\), so \(D_xe\)826is injective. The compact-to-Hausdorff criterion makes \(e\) a827homeomorphism onto its image. That and immersion give an embedding.828 829Conversely let \(U\subset\mathbb R^m\) be a tubular neighborhood830of \(e(X)\) with smooth retraction \(\pi:U\to e(X)\). On \(U\) the831map \(d_0=i\circ e^{-1}\circ\pi\) satisfies \(d_0\circ e=i\).832Choose a smooth bump \(\chi\) equal to one near \(e(X)\) and with833compact support in \(U\). Set \(d=\chi d_0\) on \(U\) and \(d=0\)834off \(U\). The support condition makes this extension smooth,835and the equality holds along the image. To extend \(e\) from836\(i(X)\) to \(\mathbb R^D\), apply the same construction to837\(e\circ i^{-1}\) using a tube about \(i(X)\).838\end{proof}839 840Consequently the least possible exact latent width in unrestricted841smooth ambient reconstruction is the smooth Euclidean embedding842dimension of \(X\). Derivative rank alone does not determine that843width. For example, the double traversal \(z\mapsto z^2\) from844\(S^1\) to the unit circle in \(\mathbb R^2\) is a smooth immersion,845but a decoder cannot separate the two points at any encoded value.846The tube construction supplies a smooth map in the ambient model847class. Exact realization by a prescribed finite neural architecture848requires a separate representability argument.849 850\begin{corollary}[A decoder varying smoothly with parameters]851\label{cor:family-decoder}852Let \(P\) be a smooth manifold and853\(e:P\times X\to\mathbb R^m\) a smooth family. There exists a854jointly smooth \(d:P\times\mathbb R^m\to\mathbb R^D\) with855\(d(p,e(p,x))=i(x)\) for all \(p,x\) if and only if each \(e_p\)856is a smooth embedding.857\end{corollary}858 859\begin{proof}860Necessity follows by restricting to any \(p\). For sufficiency,861the map \(j(p,x)=(p,e(p,x))\) is an injective immersion, since a862vector in its derivative kernel has zero parameter component and863then zero \(X\)-component. The preimage of a compact set \(K\)864is closed in the compact set \(\operatorname{pr}_P(K)\times X\).865Thus \(j\) is proper and is a closed embedding. Take a smooth866tubular retraction \(\pi\) to its image in \(P\times\mathbb R^m\).867On this tube use \(i\circ\operatorname{pr}_X\circ j^{-1}\circ\pi\).868A smooth cutoff equal to one near the closed image and supported869in the tube lets us extend this map by zero. The cutoff can be870chosen by a partition of unity subordinate to the tube and the871complement of the image, with locally finite supports when \(P\)872is noncompact. Its value on \(j(p,x)\) is \(i(x)\), as required.873\end{proof}874 875\subsection{Uniform approximation and a sharp spherical bound}876 877The existence of a uniform error gap extends beyond exact recovery.878Suppose \(\dim X=n>0\). Choose a tubular neighborhood879\(U\subset\mathbb R^D\) of \(i(X)\) and a number \(\delta>0\) such880that all points at distance less than \(\delta\) from \(i(X)\)