Indexed text attachment
parametrized-neural-networks-and-their-representations-in-differential-topology.tex
Source for Parametrized Neural Networks and Their Representations in Differential Topology
View paper · Download original file
Lines 2001–2100 of 2141. Source text is displayed literally.
2001remove an $S^0$). Between critical times the points move continuously. If $X$ is a circle, $f_t$ changes2002sign at each point of $B_t$ and returns to its original sign after one circuit, so $\#B_t$ is even for every2003regular $t$.2004 2005\sol{5.3} Two. By \cref{prop:chi} with $d=3$, the Euler characteristic of the region changes by2006$\pm(-1)^k$ at each surgery; the solid torus has $\chi=0$ and needs at least one $0$-handle to be non-empty,2007so at least one more handle of odd index is needed. A $0$-handle (a ball appears, $k=0$) followed by a2008$1$-handle attached to it (the ball becomes a solid torus, $k=1$) suffices. The boundary goes from empty to2009a sphere to a torus.2010 2011\sol{5.4} Write $\Phi=\Phi_\gamma$ and let $t(x)$ be defined by $\Phi(t(x),x)=0$ near $x_i$, with2012$t(x_i)=t_i$. Differentiating once, $\Phi_t\,\partial_jt+\partial_j\Phi=0$, so $\partial_jt(x_i)=0$ because2013$d_x\Phi=0$ there. Differentiating again and evaluating at $x_i$, where the terms containing first2014derivatives of $t$ vanish, gives $\Phi_t\,\partial_j\partial_kt+\partial_j\partial_k\Phi=0$. Hence2015$\nabla^2t(x_i)=-\nabla^2_x\Phi(t_i,x_i)/\Phi_t(t_i,x_i)$.2016%% ---- end sol/05.tex ----2017%% ---- begin sol/06.tex ----2018\section*{Chapter 6}2019 2020\sol{6.1} $\ev(\theta)=\Phi\theta$ with $\Phi$ of rank $n$. Every $y$ is a regular value;2021$M_y=\theta^*+\ker\Phi$ is an affine subspace of dimension $P-n$, where $\theta^*$ is any solution, for2022instance the minimum-norm one $\Phi^{\mathsf T}(\Phi\Phi^{\mathsf T})^{-1}y$. The Hessian is the constant2023matrix $\Phi^{\mathsf T}\Phi$, of rank $n$, with kernel $\ker\Phi$.2024 2025\sol{6.2} $\nabla\loss=(ab-1)(b,a)$, which vanishes when $ab=1$ or $(a,b)=0$. The Hessian is2026$\bigl(\begin{smallmatrix}b^2&2ab-1\\2ab-1&a^2\end{smallmatrix}\bigr)$. On $ab=1$ it is2027$\bigl(\begin{smallmatrix}b^2&1\\1&a^2\end{smallmatrix}\bigr)$, of determinant $a^2b^2-1=0$ and trace2028$a^2+b^2>0$: rank $1$, with kernel spanned by $(a,-b)$, the tangent to the hyperbola. At the origin it is2029$\bigl(\begin{smallmatrix}0&-1\\-1&0\end{smallmatrix}\bigr)$, with eigenvalues $\pm1$: a saddle of index2030$1$ with value $\frac12$. The sublevel set is $\{1-\sqrt{2c}\le ab\le1+\sqrt{2c}\}$. For $c<\frac12$ the2031lower bound is positive, and the set lies in the two quadrants $ab>0$, one component in each. For2032$c\ge\frac12$ it contains the axes and is connected. Any path from $(1,1)$ to $(-1,-1)$ must cross the set2033$ab=0$, where $\loss=\frac12$.2034 2035\sol{6.3} The flow line $\gamma$ has length at most $\ell=\sqrt{2\loss(\theta_0)/\mu}$ while it stays in $U$.2036If the closed ball of radius $\ell$ about $\theta_0$ lies in $U$, the flow line cannot leave $U$: at the2037first exit time it would be at distance at most its length, which is at most $\ell$, from $\theta_0$,2038hence still inside. So the flow line is defined for all time, has finite length, and therefore converges; the2039limit $\theta_\infty$ satisfies $|\theta_\infty-\theta_0|\le\ell$ and $\loss(\theta_\infty)=\lim2040\loss(\gamma(t))=0$.2041 2042\sol{6.4} $\loss=\frac12(\prod a_i-1)^2$ and $M_y=\{\prod a_i=1\}$, the preimage of the regular value $1$ of2043the product map (its gradient at a point with all $a_i\ne0$ is non-zero); so $\dim M_y=k-1$. On $M_y$ every2044$a_i\ne0$ and the number of negative $a_i$ is even; each admissible sign pattern gives one component (a2045graph over the positive orthant in the first $k-1$ coordinates), and there are $2^{k-1}$ sign patterns with2046an even number of minus signs. The gradient is $(\prod a_i-1)\bigl(\prod_{j\ne i}a_j\bigr)_i$, which vanishes2047at the origin for $k\ge2$; the Hessian at the origin has entries2048$-\partial_i\partial_j\prod a_l$, which are zero for $k\ge3$. So the origin is a critical point with zero2049Hessian.2050%% ---- end sol/06.tex ----2051%% ---- begin sol/07.tex ----2052\section*{Chapter 7}2053 2054\sol{7.1} Let $\varphi^t$ be the flow. If $x<y$ then $\varphi^t(x)\ne\varphi^t(y)$ for all $t$ by uniqueness2055of solutions, so $\varphi^t(y)-\varphi^t(x)$ is continuous in $t$, never zero, and positive at $t=0$; hence2056positive at $t=1$.2057 2058\sol{7.2} Let $\rho\colon[0,\infty)\to[0,1]$ be smooth with $\rho=1$ on $[0,2]$ and $\rho=0$ on $[3,\infty)$,2059and let $v(x)=\pi\rho(|x|)\,(-x_2,x_1)$. The flow preserves $|x|$, since $v$ is tangent to circles, and on2060$\{|x|\le2\}$ it is rotation with angular speed $\pi$. At time one the points $(\pm1,0)$ have been rotated2061by the angle $\pi$ and are exchanged.2062 2063\sol{7.3} Let $\varphi_t$ be the diffeotopy and $K_t=\varphi_t(K)$, $L_t=\varphi_t(L)$; they are disjoint for2064every $t$ because $\varphi_t$ is injective. The Gauss maps2065$G_t\colon K\times L\to S^2$, $G_t(x,y)=(\varphi_t(y)-\varphi_t(x))/|\varphi_t(y)-\varphi_t(x)|$, form a2066homotopy, and the degree is a homotopy invariant by \DT{Thm.~7.2}.2067 2068\sol{7.4} The first projection is $t\mapsto(\cos t,\sin t)$, the standard embedding of the circle. The second2069is $t\mapsto(\cos2t,\sin2t)$, which takes the same value at $t$ and $t+\pi$; it is an immersion but not2070injective, a double cover of the circle.2071%% ---- end sol/07.tex ----2072%% ---- begin sol/08.tex ----2073\section*{Chapter 8}2074 2075\sol{8.1} For fixed $y$ the map $T(x)=y-g(x)$ satisfies $|T(x)-T(x')|\le L|x-x'|$, so it has a unique fixed2076point $x^*$, which is the unique solution of $x+g(x)=y$, and $|x_j-x^*|=|T^j(x_0)-T^j(x^*)|\le L^j|x_0-x^*|$.2077So $f$ is a bijection. And $|f(x)-f(x')|\ge|x-x'|-|g(x)-g(x')|\ge(1-L)|x-x'|$, which says that $f^{-1}$ is2078Lipschitz with constant $(1-L)^{-1}$.2079 2080\sol{8.2} The singular values of $1+Dg$ are at least $1-L$, since $|(1+Dg)v|\ge|v|-|Dg\,v|\ge(1-L)|v|$. The2081absolute value of the determinant is the product of the singular values, so it is at least $(1-L)^d$; and2082the determinant is positive by \cref{prop:iresnet}(b).2083 2084\sol{8.3} $z=0$ is always a fixed point, with $1-g'(0)=1-a$. For $a\le1$ it is the only one: $\tanh(az)$ is2085strictly concave for $z>0$ with slope $a\le1$ at $0$, so $\tanh(az)<z$ for $z>0$, and symmetrically for2086$z<0$. For $a>1$, $\tanh(az)>z$ for small $z>0$ and $\tanh(az)<1<z$ for $z>1$, so there is a fixed point2087$z^*\in(0,1)$, unique by concavity, and $-z^*$ by symmetry. At $z^*$ the graph of $\tanh(az)$ crosses the2088diagonal from above, so $g'(z^*)<1$ and the sign is $+$; at $0$ the sign of $1-a$ is $-$. The signed count is2089$1-1+1=1$. For $a<1$ it is the single $+1$. At $a=1$ the fixed point $0$ is degenerate ($1-g'(0)=0$): this2090is the exceptional parameter at which the pair $\pm z^*$ is born, a pitchfork, which is not a generic2091birth--death because of the symmetry $z\mapsto-z$ present when $u=0$.2092 2093\sol{8.4} A point $y$ lies in $\operatorname{supp}(h_*\mu)$ if and only if every open neighbourhood $V$ of $y$2094has $\mu(h^{-1}(V))>0$. As $h$ is a homeomorphism, the sets $h^{-1}(V)$ are exactly the open neighbourhoods of2095$h^{-1}(y)$; so the condition is $h^{-1}(y)\in\operatorname{supp}\mu$. If $h_*\mu=\nu$ then2096$h$ restricts to a homeomorphism from $\operatorname{supp}\mu$ to $\operatorname{supp}\nu$.2097%% ---- end sol/08.tex ----2098\backmatter2099%% ---- begin bib.tex ----2100\chapter{References}