Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 854–953 of 1074. Source text is displayed literally.

854jointly smooth \(d:P\times\mathbb R^m\to\mathbb R^D\) with855\(d(p,e(p,x))=i(x)\) for all \(p,x\) if and only if each \(e_p\)856is a smooth embedding.857\end{corollary}858 859\begin{proof}860Necessity follows by restricting to any \(p\). For sufficiency,861the map \(j(p,x)=(p,e(p,x))\) is an injective immersion, since a862vector in its derivative kernel has zero parameter component and863then zero \(X\)-component. The preimage of a compact set \(K\)864is closed in the compact set \(\operatorname{pr}_P(K)\times X\).865Thus \(j\) is proper and is a closed embedding. Take a smooth866tubular retraction \(\pi\) to its image in \(P\times\mathbb R^m\).867On this tube use \(i\circ\operatorname{pr}_X\circ j^{-1}\circ\pi\).868A smooth cutoff equal to one near the closed image and supported869in the tube lets us extend this map by zero. The cutoff can be870chosen by a partition of unity subordinate to the tube and the871complement of the image, with locally finite supports when \(P\)872is noncompact. Its value on \(j(p,x)\) is \(i(x)\), as required.873\end{proof}874 875\subsection{Uniform approximation and a sharp spherical bound}876 877The existence of a uniform error gap extends beyond exact recovery.878Suppose \(\dim X=n>0\). Choose a tubular neighborhood879\(U\subset\mathbb R^D\) of \(i(X)\) and a number \(\delta>0\) such880that all points at distance less than \(\delta\) from \(i(X)\)881belong to \(U\). Such a number exists by compactness.882 883\begin{proposition}[A closed-manifold error gap]884\label{prop:closed-gap}885If \(m\le n\), every continuous encoder and ambient decoder satisfy886\(\sup_{x\in X}\|d(e(x))-i(x)\|\ge\delta\).887\end{proposition}888 889\begin{proof}890If the supremum were less than \(\delta\), the straight segment891from \(i(x)\) to \(d(e(x))\) would lie in \(U\) for every \(x\).892Put \(V=d^{-1}(U)\), an open subset of \(\mathbb R^m\) containing893\(e(X)\), and let \(r:U\to X\) be the tubular retraction with894\(r\circ i=\mathrm{id}_X\). Applying \(r\) to those segments gives895a homotopy to the identity for the composite896\[897X\xrightarrow{e}V\xrightarrow{\,d|_V\,}U\xrightarrow{r}X.898\]899With coefficients in \(\mathbb F_2\), the identity on \(H_n(X)\)900would then factor through \(H_n(V)\). For \(m>0\), every901component of \(V\) is a noncompact \(m\)-manifold, whose ordinary902homology vanishes in degrees at least \(m\)903\cite[Prop.~3.29]{hatcher-at}. For \(m=0\) its positive-degree904homology also vanishes. The mod-two fundamental class of any905component of \(X\) is nonzero, yielding a contradiction.906\end{proof}907 908The use of the open latent subset \(V\) is essential because the909decoder may leave the tubular neighborhood elsewhere in latent910space. The argument applies to continuous layers, including911piecewise smooth realizations. It bounds uniform error for an912entire data manifold and therefore also bounds each eligible913member of a parameter family.914 915\begin{proposition}[Precise bounds on the unit sphere]916\label{prop:sphere-gap}917Take \(X=S^n\) and let \(i:S^n\hookrightarrow\mathbb R^{n+1}\)918be the standard unit sphere, with \(n\ge1\).919For \(m\le n\), the optimal uniform error among continuous920encoders and ambient decoders is \(1\).921For \(m\ge n+1\), exact ambient recovery is possible.922If the decoder is constrained to take values in \(S^n\)923on all of \(\mathbb R^m\), the uniform error is \(2\)924for every width \(m\).925\end{proposition}926 927\begin{proof}928For \(m\le n\), apply Borsuk--Ulam to the composite of the929encoder and the coordinate inclusion of \(\mathbb R^m\) in930\(\mathbb R^n\). It gives an \(x\) with \(e(x)=e(-x)\)931\cite{matousek-borsuk}.932If \(u\) is their common decoded value,933\(2\le\|x-u\|+\|-x-u\|\). The zero decoder attains error \(1\).934For \(m\ge n+1\), a coordinate inclusion and projection give935exact recovery. Finally a sphere-valued reconstruction through936all of \(\mathbb R^m\) is nullhomotopic. If it avoided937antipodal coincidences, its straight segments to the identity,938normalized to unit length, would give a homotopy to the identity.939This would contradict its zero degree. Hence it reaches error940\(2\), the largest distance between points of the sphere.941\end{proof}942 943The optimal errors for the unit sphere are summarized below.944\begin{center}945\begin{tabular}{@{}lcc@{}}946\hline947Decoder range & \(m\le n\) & \(m\ge n+1\)\\948\hline949Ambient \(\mathbb R^{n+1}\) & \(1\) & \(0\)\\950Sphere \(S^n\) & \(2\) & \(2\)\\951\hline952\end{tabular}953\end{center}