Independent scholarship · Open accessThe monthly digest

Indexed text attachment

parametrized-neural-networks-and-their-representations-in-differential-topology.tex

Source for Parametrized Neural Networks and Their Representations in Differential Topology

View paper · Download original file

Lines 1981–2080 of 2141. Source text is displayed literally.

1981 1982\sol{4.4} At a triple point $p$, $f_\theta$ is transverse to $A_{\{1,2,3\}}$, the line $y_1=y_2=y_3$ in $\R^3$,1983of codimension $2$. So the map $x\mapsto(f_1-f_2,f_2-f_3)$ from the plane to $\R^2$ is a local1984diffeomorphism at $p$, and in the coordinates $(u,v)=(f_1-f_2,f_2-f_3)$ the decision regions are1985$\{u>0,\ u+v>0\}$, $\{u<0,\ v>0\}$ and $\{v<0,\ u+v<0\}$: three sectors bounded by the rays1986$\{u=0,v>0\}$, $\{v=0,u>0\}$ and $\{u+v=0,u<0\}$.1987%% ---- end sol/04.tex ----1988%% ---- begin sol/05.tex ----1989\section*{Chapter 5}1990 1991\sol{5.1} $g=(x_1^2-1)^2+x_2^2$ has $\nabla g=(4x_1(x_1^2-1),2x_2)$, vanishing at $(0,0)$ and $(\pm1,0)$. The1992Hessian is $\operatorname{diag}(12x_1^2-4,\,2)$: at $(\pm1,0)$ it is $\operatorname{diag}(8,2)$, index $0$,1993value $0$; at the origin $\operatorname{diag}(-4,2)$, index $1$, value $1$. The sublevel set $\{g\le t\}$ is1994empty for $t<0$, two discs around $(\pm1,0)$ for $0<t<1$ (on the line $x_2=0$ it is1995$\{|x_1^2-1|\le\sqrt t\}$, two intervals when $t<1$), and one disc for $t>1$. Its Euler characteristic is1996$0,2,1$, consistent with attaching two $0$-cells and then one $1$-cell.1997 1998\sol{5.2} Here $B_t$ is a finite set and $W_\gamma$ is a compact $1$-manifold with boundary $B_0\sqcup B_1$.1999A critical point of $\tau$ on a $1$-manifold is a local minimum or maximum of $t$ along a curve: two points2000of $B_t$ appear together or disappear together (surgery with $\lambda=0$ or $1$ on a $0$-manifold: add or2001remove an $S^0$). Between critical times the points move continuously. If $X$ is a circle, $f_t$ changes2002sign at each point of $B_t$ and returns to its original sign after one circuit, so $\#B_t$ is even for every2003regular $t$.2004 2005\sol{5.3} Two. By \cref{prop:chi} with $d=3$, the Euler characteristic of the region changes by2006$\pm(-1)^k$ at each surgery; the solid torus has $\chi=0$ and needs at least one $0$-handle to be non-empty,2007so at least one more handle of odd index is needed. A $0$-handle (a ball appears, $k=0$) followed by a2008$1$-handle attached to it (the ball becomes a solid torus, $k=1$) suffices. The boundary goes from empty to2009a sphere to a torus.2010 2011\sol{5.4} Write $\Phi=\Phi_\gamma$ and let $t(x)$ be defined by $\Phi(t(x),x)=0$ near $x_i$, with2012$t(x_i)=t_i$. Differentiating once, $\Phi_t\,\partial_jt+\partial_j\Phi=0$, so $\partial_jt(x_i)=0$ because2013$d_x\Phi=0$ there. Differentiating again and evaluating at $x_i$, where the terms containing first2014derivatives of $t$ vanish, gives $\Phi_t\,\partial_j\partial_kt+\partial_j\partial_k\Phi=0$. Hence2015$\nabla^2t(x_i)=-\nabla^2_x\Phi(t_i,x_i)/\Phi_t(t_i,x_i)$.2016%% ---- end sol/05.tex ----2017%% ---- begin sol/06.tex ----2018\section*{Chapter 6}2019 2020\sol{6.1} $\ev(\theta)=\Phi\theta$ with $\Phi$ of rank $n$. Every $y$ is a regular value;2021$M_y=\theta^*+\ker\Phi$ is an affine subspace of dimension $P-n$, where $\theta^*$ is any solution, for2022instance the minimum-norm one $\Phi^{\mathsf T}(\Phi\Phi^{\mathsf T})^{-1}y$. The Hessian is the constant2023matrix $\Phi^{\mathsf T}\Phi$, of rank $n$, with kernel $\ker\Phi$.2024 2025\sol{6.2} $\nabla\loss=(ab-1)(b,a)$, which vanishes when $ab=1$ or $(a,b)=0$. The Hessian is2026$\bigl(\begin{smallmatrix}b^2&2ab-1\\2ab-1&a^2\end{smallmatrix}\bigr)$. On $ab=1$ it is2027$\bigl(\begin{smallmatrix}b^2&1\\1&a^2\end{smallmatrix}\bigr)$, of determinant $a^2b^2-1=0$ and trace2028$a^2+b^2>0$: rank $1$, with kernel spanned by $(a,-b)$, the tangent to the hyperbola. At the origin it is2029$\bigl(\begin{smallmatrix}0&-1\\-1&0\end{smallmatrix}\bigr)$, with eigenvalues $\pm1$: a saddle of index2030$1$ with value $\frac12$. The sublevel set is $\{1-\sqrt{2c}\le ab\le1+\sqrt{2c}\}$. For $c<\frac12$ the2031lower bound is positive, and the set lies in the two quadrants $ab>0$, one component in each. For2032$c\ge\frac12$ it contains the axes and is connected. Any path from $(1,1)$ to $(-1,-1)$ must cross the set2033$ab=0$, where $\loss=\frac12$.2034 2035\sol{6.3} The flow line $\gamma$ has length at most $\ell=\sqrt{2\loss(\theta_0)/\mu}$ while it stays in $U$.2036If the closed ball of radius $\ell$ about $\theta_0$ lies in $U$, the flow line cannot leave $U$: at the2037first exit time it would be at distance at most its length, which is at most $\ell$, from $\theta_0$,2038hence still inside. So the flow line is defined for all time, has finite length, and therefore converges; the2039limit $\theta_\infty$ satisfies $|\theta_\infty-\theta_0|\le\ell$ and $\loss(\theta_\infty)=\lim2040\loss(\gamma(t))=0$.2041 2042\sol{6.4} $\loss=\frac12(\prod a_i-1)^2$ and $M_y=\{\prod a_i=1\}$, the preimage of the regular value $1$ of2043the product map (its gradient at a point with all $a_i\ne0$ is non-zero); so $\dim M_y=k-1$. On $M_y$ every2044$a_i\ne0$ and the number of negative $a_i$ is even; each admissible sign pattern gives one component (a2045graph over the positive orthant in the first $k-1$ coordinates), and there are $2^{k-1}$ sign patterns with2046an even number of minus signs. The gradient is $(\prod a_i-1)\bigl(\prod_{j\ne i}a_j\bigr)_i$, which vanishes2047at the origin for $k\ge2$; the Hessian at the origin has entries2048$-\partial_i\partial_j\prod a_l$, which are zero for $k\ge3$. So the origin is a critical point with zero2049Hessian.2050%% ---- end sol/06.tex ----2051%% ---- begin sol/07.tex ----2052\section*{Chapter 7}2053 2054\sol{7.1} Let $\varphi^t$ be the flow. If $x<y$ then $\varphi^t(x)\ne\varphi^t(y)$ for all $t$ by uniqueness2055of solutions, so $\varphi^t(y)-\varphi^t(x)$ is continuous in $t$, never zero, and positive at $t=0$; hence2056positive at $t=1$.2057 2058\sol{7.2} Let $\rho\colon[0,\infty)\to[0,1]$ be smooth with $\rho=1$ on $[0,2]$ and $\rho=0$ on $[3,\infty)$,2059and let $v(x)=\pi\rho(|x|)\,(-x_2,x_1)$. The flow preserves $|x|$, since $v$ is tangent to circles, and on2060$\{|x|\le2\}$ it is rotation with angular speed $\pi$. At time one the points $(\pm1,0)$ have been rotated2061by the angle $\pi$ and are exchanged.2062 2063\sol{7.3} Let $\varphi_t$ be the diffeotopy and $K_t=\varphi_t(K)$, $L_t=\varphi_t(L)$; they are disjoint for2064every $t$ because $\varphi_t$ is injective. The Gauss maps2065$G_t\colon K\times L\to S^2$, $G_t(x,y)=(\varphi_t(y)-\varphi_t(x))/|\varphi_t(y)-\varphi_t(x)|$, form a2066homotopy, and the degree is a homotopy invariant by \DT{Thm.~7.2}.2067 2068\sol{7.4} The first projection is $t\mapsto(\cos t,\sin t)$, the standard embedding of the circle. The second2069is $t\mapsto(\cos2t,\sin2t)$, which takes the same value at $t$ and $t+\pi$; it is an immersion but not2070injective, a double cover of the circle.2071%% ---- end sol/07.tex ----2072%% ---- begin sol/08.tex ----2073\section*{Chapter 8}2074 2075\sol{8.1} For fixed $y$ the map $T(x)=y-g(x)$ satisfies $|T(x)-T(x')|\le L|x-x'|$, so it has a unique fixed2076point $x^*$, which is the unique solution of $x+g(x)=y$, and $|x_j-x^*|=|T^j(x_0)-T^j(x^*)|\le L^j|x_0-x^*|$.2077So $f$ is a bijection. And $|f(x)-f(x')|\ge|x-x'|-|g(x)-g(x')|\ge(1-L)|x-x'|$, which says that $f^{-1}$ is2078Lipschitz with constant $(1-L)^{-1}$.2079 2080\sol{8.2} The singular values of $1+Dg$ are at least $1-L$, since $|(1+Dg)v|\ge|v|-|Dg\,v|\ge(1-L)|v|$. The