Independent scholarship · Open accessThe monthly digest

Indexed text attachment

differential-topology-of-parametrized-neural-networks.tex

Source for Differential Topology of Parametrized Neural Networks

View paper · Download original file

Lines 874–973 of 1074. Source text is displayed literally.

874 875\subsection{Uniform approximation and a sharp spherical bound}876 877The existence of a uniform error gap extends beyond exact recovery.878Suppose \(\dim X=n>0\). Choose a tubular neighborhood879\(U\subset\mathbb R^D\) of \(i(X)\) and a number \(\delta>0\) such880that all points at distance less than \(\delta\) from \(i(X)\)881belong to \(U\). Such a number exists by compactness.882 883\begin{proposition}[A closed-manifold error gap]884\label{prop:closed-gap}885If \(m\le n\), every continuous encoder and ambient decoder satisfy886\(\sup_{x\in X}\|d(e(x))-i(x)\|\ge\delta\).887\end{proposition}888 889\begin{proof}890If the supremum were less than \(\delta\), the straight segment891from \(i(x)\) to \(d(e(x))\) would lie in \(U\) for every \(x\).892Put \(V=d^{-1}(U)\), an open subset of \(\mathbb R^m\) containing893\(e(X)\), and let \(r:U\to X\) be the tubular retraction with894\(r\circ i=\mathrm{id}_X\). Applying \(r\) to those segments gives895a homotopy to the identity for the composite896\[897X\xrightarrow{e}V\xrightarrow{\,d|_V\,}U\xrightarrow{r}X.898\]899With coefficients in \(\mathbb F_2\), the identity on \(H_n(X)\)900would then factor through \(H_n(V)\). For \(m>0\), every901component of \(V\) is a noncompact \(m\)-manifold, whose ordinary902homology vanishes in degrees at least \(m\)903\cite[Prop.~3.29]{hatcher-at}. For \(m=0\) its positive-degree904homology also vanishes. The mod-two fundamental class of any905component of \(X\) is nonzero, yielding a contradiction.906\end{proof}907 908The use of the open latent subset \(V\) is essential because the909decoder may leave the tubular neighborhood elsewhere in latent910space. The argument applies to continuous layers, including911piecewise smooth realizations. It bounds uniform error for an912entire data manifold and therefore also bounds each eligible913member of a parameter family.914 915\begin{proposition}[Precise bounds on the unit sphere]916\label{prop:sphere-gap}917Take \(X=S^n\) and let \(i:S^n\hookrightarrow\mathbb R^{n+1}\)918be the standard unit sphere, with \(n\ge1\).919For \(m\le n\), the optimal uniform error among continuous920encoders and ambient decoders is \(1\).921For \(m\ge n+1\), exact ambient recovery is possible.922If the decoder is constrained to take values in \(S^n\)923on all of \(\mathbb R^m\), the uniform error is \(2\)924for every width \(m\).925\end{proposition}926 927\begin{proof}928For \(m\le n\), apply Borsuk--Ulam to the composite of the929encoder and the coordinate inclusion of \(\mathbb R^m\) in930\(\mathbb R^n\). It gives an \(x\) with \(e(x)=e(-x)\)931\cite{matousek-borsuk}.932If \(u\) is their common decoded value,933\(2\le\|x-u\|+\|-x-u\|\). The zero decoder attains error \(1\).934For \(m\ge n+1\), a coordinate inclusion and projection give935exact recovery. Finally a sphere-valued reconstruction through936all of \(\mathbb R^m\) is nullhomotopic. If it avoided937antipodal coincidences, its straight segments to the identity,938normalized to unit length, would give a homotopy to the identity.939This would contradict its zero degree. Hence it reaches error940\(2\), the largest distance between points of the sphere.941\end{proof}942 943The optimal errors for the unit sphere are summarized below.944\begin{center}945\begin{tabular}{@{}lcc@{}}946\hline947Decoder range & \(m\le n\) & \(m\ge n+1\)\\948\hline949Ambient \(\mathbb R^{n+1}\) & \(1\) & \(0\)\\950Sphere \(S^n\) & \(2\) & \(2\)\\951\hline952\end{tabular}953\end{center}954For a sphere of radius \(R>0\), rescaling replaces the positive955entries by \(R\) and \(2R\), respectively. The bound holds for every956parameter of an appropriate family. Its proof chooses an957antipodal pair separately at each parameter and needs no958continuous choice of witnesses.959 960Uniform and average losses have different implications. For example,961at width \(n\) choose stereographic coordinates \(\sigma\) from the962complement of the north pole of \(S^n\) to \(\mathbb R^n\).963Multiply \(\sigma\) by a smooth cutoff equal to one outside a small964polar cap \(C\) and equal to zero near the pole, extending the965encoder by zero there. The smooth decoder \(d=\sigma^{-1}\) gives966exact recovery off \(C\). For normalized spherical measure \(\mu\),967the squared error averaged over the sphere is at most \(4\mu(C)\),968which can be made arbitrarily small while the uniform error is \(2\).969Thus the uniform obstruction does not force a positive lower bound970on this average across all smooth maps of width \(n\).971 972A uniform Lipschitz bound prevents this concentration. If the973reconstruction has Lipschitz constant at most \(L\) with respect to