Independent scholarship · Open accessThe monthly digest

Indexed text attachment

parametrized-neural-networks-and-their-representations-in-differential-topology.tex

Source for Parametrized Neural Networks and Their Representations in Differential Topology

View paper · Download original file

Lines 1961–2060 of 2141. Source text is displayed literally.

1961%% ---- end sol/03.tex ----1962%% ---- begin sol/04.tex ----1963\section*{Chapter 4}1964 1965\sol{4.1} By \cref{thm:boundary}(c), for almost every $\theta$ the set $B_\theta$ is a compact neat1966$0$-dimensional submanifold of $[-R,R]$: a finite set of interior points at which $f_\theta'\ne0$, with1967$f_\theta(\pm R)\ne0$. At each such point $f_\theta$ changes sign. So the number of zeros is even if1968$f_\theta(-R)$ and $f_\theta(R)$ have the same sign and odd otherwise.1969 1970\sol{4.2} A compact neat $1$-dimensional submanifold of the disc: a finite disjoint union of circles in the1971interior and arcs with both endpoints on the boundary circle, meeting it transversally. Any such1972configuration, up to isotopy, is determined by the nesting of the circles and the pairing of the endpoints1973by non-crossing arcs.1974 1975\sol{4.3} $d_xF=(2x_1,-2x_2)$ vanishes only at $x=0$, where $F=c$. So $\Sigma=\{(0,0)\}$ and $\Delta=\{0\}$.1976There are two chambers, $c>0$ and $c<0$. In each the boundary is a hyperbola; for $c<0$ its branches open1977along the $x_1$-axis ($x_1^2=x_2^2-c\ge-c$) and for $c>0$ along the $x_2$-axis. The region $\{f<0\}$ is connected for1978$c<0$ (it contains the origin) and has two components for $c>0$. (The plane is not compact; the statement about chambers holds here1979because the family is a product near infinity up to scaling. Restricted to a disc the same description1980holds.)1981 1982\sol{4.4} At a triple point $p$, $f_\theta$ is transverse to $A_{\{1,2,3\}}$, the line $y_1=y_2=y_3$ in $\R^3$,1983of codimension $2$. So the map $x\mapsto(f_1-f_2,f_2-f_3)$ from the plane to $\R^2$ is a local1984diffeomorphism at $p$, and in the coordinates $(u,v)=(f_1-f_2,f_2-f_3)$ the decision regions are1985$\{u>0,\ u+v>0\}$, $\{u<0,\ v>0\}$ and $\{v<0,\ u+v<0\}$: three sectors bounded by the rays1986$\{u=0,v>0\}$, $\{v=0,u>0\}$ and $\{u+v=0,u<0\}$.1987%% ---- end sol/04.tex ----1988%% ---- begin sol/05.tex ----1989\section*{Chapter 5}1990 1991\sol{5.1} $g=(x_1^2-1)^2+x_2^2$ has $\nabla g=(4x_1(x_1^2-1),2x_2)$, vanishing at $(0,0)$ and $(\pm1,0)$. The1992Hessian is $\operatorname{diag}(12x_1^2-4,\,2)$: at $(\pm1,0)$ it is $\operatorname{diag}(8,2)$, index $0$,1993value $0$; at the origin $\operatorname{diag}(-4,2)$, index $1$, value $1$. The sublevel set $\{g\le t\}$ is1994empty for $t<0$, two discs around $(\pm1,0)$ for $0<t<1$ (on the line $x_2=0$ it is1995$\{|x_1^2-1|\le\sqrt t\}$, two intervals when $t<1$), and one disc for $t>1$. Its Euler characteristic is1996$0,2,1$, consistent with attaching two $0$-cells and then one $1$-cell.1997 1998\sol{5.2} Here $B_t$ is a finite set and $W_\gamma$ is a compact $1$-manifold with boundary $B_0\sqcup B_1$.1999A critical point of $\tau$ on a $1$-manifold is a local minimum or maximum of $t$ along a curve: two points2000of $B_t$ appear together or disappear together (surgery with $\lambda=0$ or $1$ on a $0$-manifold: add or2001remove an $S^0$). Between critical times the points move continuously. If $X$ is a circle, $f_t$ changes2002sign at each point of $B_t$ and returns to its original sign after one circuit, so $\#B_t$ is even for every2003regular $t$.2004 2005\sol{5.3} Two. By \cref{prop:chi} with $d=3$, the Euler characteristic of the region changes by2006$\pm(-1)^k$ at each surgery; the solid torus has $\chi=0$ and needs at least one $0$-handle to be non-empty,2007so at least one more handle of odd index is needed. A $0$-handle (a ball appears, $k=0$) followed by a2008$1$-handle attached to it (the ball becomes a solid torus, $k=1$) suffices. The boundary goes from empty to2009a sphere to a torus.2010 2011\sol{5.4} Write $\Phi=\Phi_\gamma$ and let $t(x)$ be defined by $\Phi(t(x),x)=0$ near $x_i$, with2012$t(x_i)=t_i$. Differentiating once, $\Phi_t\,\partial_jt+\partial_j\Phi=0$, so $\partial_jt(x_i)=0$ because2013$d_x\Phi=0$ there. Differentiating again and evaluating at $x_i$, where the terms containing first2014derivatives of $t$ vanish, gives $\Phi_t\,\partial_j\partial_kt+\partial_j\partial_k\Phi=0$. Hence2015$\nabla^2t(x_i)=-\nabla^2_x\Phi(t_i,x_i)/\Phi_t(t_i,x_i)$.2016%% ---- end sol/05.tex ----2017%% ---- begin sol/06.tex ----2018\section*{Chapter 6}2019 2020\sol{6.1} $\ev(\theta)=\Phi\theta$ with $\Phi$ of rank $n$. Every $y$ is a regular value;2021$M_y=\theta^*+\ker\Phi$ is an affine subspace of dimension $P-n$, where $\theta^*$ is any solution, for2022instance the minimum-norm one $\Phi^{\mathsf T}(\Phi\Phi^{\mathsf T})^{-1}y$. The Hessian is the constant2023matrix $\Phi^{\mathsf T}\Phi$, of rank $n$, with kernel $\ker\Phi$.2024 2025\sol{6.2} $\nabla\loss=(ab-1)(b,a)$, which vanishes when $ab=1$ or $(a,b)=0$. The Hessian is2026$\bigl(\begin{smallmatrix}b^2&2ab-1\\2ab-1&a^2\end{smallmatrix}\bigr)$. On $ab=1$ it is2027$\bigl(\begin{smallmatrix}b^2&1\\1&a^2\end{smallmatrix}\bigr)$, of determinant $a^2b^2-1=0$ and trace2028$a^2+b^2>0$: rank $1$, with kernel spanned by $(a,-b)$, the tangent to the hyperbola. At the origin it is2029$\bigl(\begin{smallmatrix}0&-1\\-1&0\end{smallmatrix}\bigr)$, with eigenvalues $\pm1$: a saddle of index2030$1$ with value $\frac12$. The sublevel set is $\{1-\sqrt{2c}\le ab\le1+\sqrt{2c}\}$. For $c<\frac12$ the2031lower bound is positive, and the set lies in the two quadrants $ab>0$, one component in each. For2032$c\ge\frac12$ it contains the axes and is connected. Any path from $(1,1)$ to $(-1,-1)$ must cross the set2033$ab=0$, where $\loss=\frac12$.2034 2035\sol{6.3} The flow line $\gamma$ has length at most $\ell=\sqrt{2\loss(\theta_0)/\mu}$ while it stays in $U$.2036If the closed ball of radius $\ell$ about $\theta_0$ lies in $U$, the flow line cannot leave $U$: at the2037first exit time it would be at distance at most its length, which is at most $\ell$, from $\theta_0$,2038hence still inside. So the flow line is defined for all time, has finite length, and therefore converges; the2039limit $\theta_\infty$ satisfies $|\theta_\infty-\theta_0|\le\ell$ and $\loss(\theta_\infty)=\lim2040\loss(\gamma(t))=0$.2041 2042\sol{6.4} $\loss=\frac12(\prod a_i-1)^2$ and $M_y=\{\prod a_i=1\}$, the preimage of the regular value $1$ of2043the product map (its gradient at a point with all $a_i\ne0$ is non-zero); so $\dim M_y=k-1$. On $M_y$ every2044$a_i\ne0$ and the number of negative $a_i$ is even; each admissible sign pattern gives one component (a2045graph over the positive orthant in the first $k-1$ coordinates), and there are $2^{k-1}$ sign patterns with2046an even number of minus signs. The gradient is $(\prod a_i-1)\bigl(\prod_{j\ne i}a_j\bigr)_i$, which vanishes2047at the origin for $k\ge2$; the Hessian at the origin has entries2048$-\partial_i\partial_j\prod a_l$, which are zero for $k\ge3$. So the origin is a critical point with zero2049Hessian.2050%% ---- end sol/06.tex ----2051%% ---- begin sol/07.tex ----2052\section*{Chapter 7}2053 2054\sol{7.1} Let $\varphi^t$ be the flow. If $x<y$ then $\varphi^t(x)\ne\varphi^t(y)$ for all $t$ by uniqueness2055of solutions, so $\varphi^t(y)-\varphi^t(x)$ is continuous in $t$, never zero, and positive at $t=0$; hence2056positive at $t=1$.2057 2058\sol{7.2} Let $\rho\colon[0,\infty)\to[0,1]$ be smooth with $\rho=1$ on $[0,2]$ and $\rho=0$ on $[3,\infty)$,2059and let $v(x)=\pi\rho(|x|)\,(-x_2,x_1)$. The flow preserves $|x|$, since $v$ is tangent to circles, and on2060$\{|x|\le2\}$ it is rotation with angular speed $\pi$. At time one the points $(\pm1,0)$ have been rotated