Indexed text attachment
partial-differential-equations-in-machine-learning-and-ai.tex
Source for Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates
View paper · Download original file
Lines 461–560 of 1085. Source text is displayed literally.
461\draw[thick,bwd] plot coordinates {(1.26,3.149) (2.52,2.434) (3.78,1.89) (5.04,1.321) (6.3,0.933)};462\fill[fwd] (0,2.619) circle (1.6pt);463\fill[fwd] (1.26,2.126) circle (1.6pt);464\fill[fwd] (2.52,1.636) circle (1.6pt);465\fill[fwd] (3.78,1.146) circle (1.6pt);466\fill[fwd] (5.04,0.656) circle (1.6pt);467\fill[fwd] (6.3,0.166) circle (1.6pt);468\fill[bwd] (1.26,3.149) circle (1.6pt);469\fill[bwd] (2.52,2.434) circle (1.6pt);470\fill[bwd] (3.78,1.89) circle (1.6pt);471\fill[bwd] (5.04,1.321) circle (1.6pt);472\fill[bwd] (6.3,0.933) circle (1.6pt);473\node[fwd] at (1.666,1.092) {\scriptsize mean of $\relu(\sin x)$};474\node[bwd] at (4.923,2.261) {\scriptsize random input, band $4$};475\node[ink2] at (2.926,2.751) {\scriptsize slope $-2$};476\end{scope}477\end{tikzpicture}478\caption{Aliasing error of a rectifier against the number of grid points, on logarithmic scales: the closed-form case of \cref{thm:alias}(d) and the largest in-band coefficient error for a random input of band $4$. Both follow the slope $-2$.}\label{fig:alias}479\end{figure}480 481\noindent For a random input of band $5$, the multiplier $(1+k^2)^{-1}$ computed on $16$ and on $64$ points agrees482with the exact result to $10^{-14}$; the stencil is in relative error by $0.221$ and $0.0153$, its symbol at $k=5$483being $0.7173$ and $0.9801$ of the true one. \chk{A1} For band $4$ the in-band coefficients of the square are484exact on $13$ points and wrong by $1.4$ on $12$: the ``two-thirds rule'' of spectral methods \cite{Ors71}.485\chk{A2} For the rectifier one case is closed-form: the mean of $\relu(\sin x)$ sampled on $N$ points is486$\frac1N\cot\frac\pi N$ instead of $\frac1\pi$, an error of $-\pi/3N^2$ to leading order; $N^2$ times the error is487$-1.058$, $-1.050$, $-1.048$, $-1.047$ for $N=8,\dots,64$. \chk{A3} For a random input of band $4$ the largest488in-band error of the rectified signal is $7.4\times10^{-2}$, $9.8\times10^{-3}$, $2.1\times10^{-3}$,489$4.2\times10^{-4}$, $1.4\times10^{-4}$ on $16,\dots,256$ points (\cref{fig:alias}). \chk{A4} This is the quantitative content of the490practice of applying nonlinearities on an oversampled grid and filtering \cite{KALHHLA21}: oversampling by a491factor $M$ buys $M^{2}$, and no finite factor buys exactness. An architecture built from (a) alone is492resolution-independent \cite{LKALBSA21}; one with stencils is so to the order of the stencil; one with493rectifiers is so only approximately, at every resolution.494 495\section{Diffusion models}\label{sec:diffusion}496 497Let data $X_0\sim p_0$ on the line be noised by $X_t=X_0+W_t$, so that the density solves the heat equation498$\pd_tp=\frac12\pd_x^2p$, and write $s=\pd_x\log p_t$ for the \term{score}. Our example is a mixture of two499Gaussians, for which everything is closed-form. \chk{S1}500 501\begin{proposition}\label{prop:tweedie}502\Scl[\cite{Efr11,Hyv05,Vin11}] \emph{(a)} $\E[X_0\mid X_t=x]=x+t\,s(x,t)$. \emph{(b)} For every candidate $\sigma$,503\begin{multline*}504\E\bigl|\sigma(X_t)+\tfrac{X_t-X_0}t\bigr|^2\\=\E|\sigma(X_t)-s(X_t,t)|^2+\tfrac1t-J(p_t),505\end{multline*}506where $J$ is the Fisher information: denoising and explicit score matching differ by a constant.507\end{proposition}508 509\noindent Both hold to rounding in the example; for two-point data at $t=\frac12$ the constant is $0.924073$ for510each of two test functions. \chk{S2, S8} Since $\frac12\pd_x^2p=-\pd_x\bigl(p\cdot(-\frac12s)\bigr)$, the heat equation511is also a continuity equation, and its characteristics are the \term{probability flow}512$\dot x=-\frac12s(x,t)$ \cite{SSKKEP21}.513 514\begin{theorem}[Where the ill-posedness goes]\label{thm:illposed}515\Srf[\cite{PDEHC}, Ex.~2.5; \cite{And82,SSKKEP21}]516\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]517\item The backward heat equation multiplies the $k$-th Fourier mode of $p_T$ by $e^{k^2T/2}$: recovering $p_0$518from $p_T$ is ill-posed in every Sobolev norm.519\item Given the score on $[t_0,T]$ with $t_0>0$, the probability flow is an ordinary differential equation with520a Lipschitz field; it transports $p_T$ to $p_{t_0}$, and an error $\epsilon$ in the score, in the supremum norm,521moves each sample by at most $C(t_0)\epsilon$.522\item For data on the two points $\pm a$, $\pd_xs(0,t)=-1/t+a^2/t^2$. The Lipschitz constant of the field is not523integrable at $t=0$, and $C(t_0)\to\infty$ as $t_0\to0$.524\end{enumerate}525\end{theorem}526\begin{proof}527(a) is Fourier analysis. (b) is the uniqueness theorem for characteristics together with Gr\"onwall's528inequality. (c) Here $s=-x/t+(a/t)\tanh(ax/t)$.529\end{proof}530 531\begin{figure}[t]532\centering533\begin{tikzpicture}534\begin{scope}[shift={(0,0)}]535\draw[ink2] (0,3.4)--(0,0)--(6,0);536\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $-2$};537\draw[ink2] (1.5,0)--(1.5,-0.07) node[below,inner sep=1.5pt]{\scriptsize $-1$};538\draw[ink2] (3,0)--(3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0$};539\draw[ink2] (4.5,0)--(4.5,-0.07) node[below,inner sep=1.5pt]{\scriptsize $1$};540\draw[ink2] (6,0)--(6,-0.07) node[below,inner sep=1.5pt]{\scriptsize $2$};541\draw[ink2] (0,0.283)--(-0.07,0.283) node[left,inner sep=1.5pt]{\scriptsize $-10$};542\draw[ink2] (0,1.7)--(-0.07,1.7) node[left,inner sep=1.5pt]{\scriptsize $0$};543\draw[ink2] (0,3.117)--(-0.07,3.117) node[left,inner sep=1.5pt]{\scriptsize $10$};544\node[below] at (3,-0.38) {\scriptsize $x$};545\draw[ink2,very thin] (0,1.7)--(6,1.7);546\draw[thick,ink2] plot coordinates {(0,1.847) (0.015,1.845) (0.03,1.844) (0.045,1.843) (0.06,1.842) (0.075,1.84) (0.09,1.839) (0.105,1.838) (0.12,1.836) (0.135,1.835) (0.15,1.834) (0.165,1.832) (0.18,1.831) (0.195,1.83) (0.21,1.829) (0.225,1.827) (0.24,1.826) (0.255,1.825) (0.27,1.823) (0.285,1.822) (0.3,1.821) (0.315,1.82) (0.33,1.818) (0.345,1.817) (0.36,1.816) (0.375,1.815) (0.39,1.813) (0.405,1.812) (0.42,1.811) (0.435,1.81) (0.45,1.808) (0.465,1.807) (0.48,1.806) (0.495,1.805) (0.51,1.803) (0.525,1.802) (0.54,1.801) (0.555,1.8) (0.57,1.799) (0.585,1.797) (0.6,1.796) (0.615,1.795) (0.63,1.794) (0.645,1.793) (0.66,1.791) (0.675,1.79) (0.69,1.789) (0.705,1.788) (0.72,1.787) (0.735,1.785) (0.75,1.784) (0.765,1.783) (0.78,1.782) (0.795,1.781) (0.81,1.78) (0.825,1.779) (0.84,1.777) (0.855,1.776) (0.87,1.775) (0.885,1.774) (0.9,1.773) (0.915,1.772) (0.93,1.771) (0.945,1.77) (0.96,1.769) (0.975,1.767) (0.99,1.766) (1.005,1.765) (1.02,1.764) (1.035,1.763) (1.05,1.762) (1.065,1.761) (1.08,1.76) (1.095,1.759) (1.11,1.758) (1.125,1.757) (1.14,1.756) (1.155,1.755) (1.17,1.754) (1.185,1.753) (1.2,1.752) (1.215,1.751) (1.23,1.75) (1.245,1.749) (1.26,1.748) (1.275,1.747) (1.29,1.746) (1.305,1.745) (1.32,1.744) (1.335,1.743) (1.35,1.742) (1.365,1.742) (1.38,1.741) (1.395,1.74) (1.41,1.739) (1.425,1.738) (1.44,1.737) (1.455,1.736) (1.47,1.735) (1.485,1.735) (1.5,1.734) (1.515,1.733) (1.53,1.732) (1.545,1.731) (1.56,1.731) (1.575,1.73) (1.59,1.729) (1.605,1.728) (1.62,1.727) (1.635,1.727) (1.65,1.726) (1.665,1.725) (1.68,1.725) (1.695,1.724) (1.71,1.723) (1.725,1.723) (1.74,1.722) (1.755,1.721) (1.77,1.721) (1.785,1.72) (1.8,1.719) (1.815,1.719) (1.83,1.718) (1.845,1.717) (1.86,1.717) (1.875,1.716) (1.89,1.716) (1.905,1.715) (1.92,1.715) (1.935,1.714) (1.95,1.714) (1.965,1.713) (1.98,1.713) (1.995,1.712) (2.01,1.712) (2.025,1.711) (2.04,1.711) (2.055,1.71) (2.07,1.71) (2.085,1.709) (2.1,1.709) (2.115,1.709) (2.13,1.708) (2.145,1.708) (2.16,1.707) (2.175,1.707) (2.19,1.707) (2.205,1.706) (2.22,1.706) (2.235,1.706) (2.25,1.705) (2.265,1.705) (2.28,1.705) (2.295,1.705) (2.31,1.704) (2.325,1.704) (2.34,1.704) (2.355,1.703) (2.37,1.703) (2.385,1.703) (2.4,1.703) (2.415,1.703) (2.43,1.702) (2.445,1.702) (2.46,1.702) (2.475,1.702) (2.49,1.702) (2.505,1.702) (2.52,1.701) (2.535,1.701) (2.55,1.701) (2.565,1.701) (2.58,1.701) (2.595,1.701) (2.61,1.701) (2.625,1.701) (2.64,1.701) (2.655,1.701) (2.67,1.7) (2.685,1.7) (2.7,1.7) (2.715,1.7) (2.73,1.7) (2.745,1.7) (2.76,1.7) (2.775,1.7) (2.79,1.7) (2.805,1.7) (2.82,1.7) (2.835,1.7) (2.85,1.7) (2.865,1.7) (2.88,1.7) (2.895,1.7) (2.91,1.7) (2.925,1.7) (2.94,1.7) (2.955,1.7) (2.97,1.7) (2.985,1.7) (3,1.7) (3.015,1.7) (3.03,1.7) (3.045,1.7) (3.06,1.7) (3.075,1.7) (3.09,1.7) (3.105,1.7) (3.12,1.7) (3.135,1.7) (3.15,1.7) (3.165,1.7) (3.18,1.7) (3.195,1.7) (3.21,1.7) (3.225,1.7) (3.24,1.7) (3.255,1.7) (3.27,1.7) (3.285,1.7) (3.3,1.7) (3.315,1.7) (3.33,1.7) (3.345,1.699) (3.36,1.699) (3.375,1.699) (3.39,1.699) (3.405,1.699) (3.42,1.699) (3.435,1.699) (3.45,1.699) (3.465,1.699) (3.48,1.699) (3.495,1.698) (3.51,1.698) (3.525,1.698) (3.54,1.698) (3.555,1.698) (3.57,1.698) (3.585,1.697) (3.6,1.697) (3.615,1.697) (3.63,1.697) (3.645,1.697) (3.66,1.696) (3.675,1.696) (3.69,1.696) (3.705,1.695) (3.72,1.695) (3.735,1.695) (3.75,1.695) (3.765,1.694) (3.78,1.694) (3.795,1.694) (3.81,1.693) (3.825,1.693) (3.84,1.693) (3.855,1.692) (3.87,1.692) (3.885,1.691) (3.9,1.691) (3.915,1.691) (3.93,1.69) (3.945,1.69) (3.96,1.689) (3.975,1.689) (3.99,1.688) (4.005,1.688) (4.02,1.687) (4.035,1.687) (4.05,1.686) (4.065,1.686) (4.08,1.685) (4.095,1.685) (4.11,1.684) (4.125,1.684) (4.14,1.683) (4.155,1.683) (4.17,1.682) (4.185,1.681) (4.2,1.681) (4.215,1.68) (4.23,1.679) (4.245,1.679) (4.26,1.678) (4.275,1.677) (4.29,1.677) (4.305,1.676) (4.32,1.675) (4.335,1.675) (4.35,1.674) (4.365,1.673) (4.38,1.673) (4.395,1.672) (4.41,1.671) (4.425,1.67) (4.44,1.669) (4.455,1.669) (4.47,1.668) (4.485,1.667) (4.5,1.666) (4.515,1.665) (4.53,1.665) (4.545,1.664) (4.56,1.663) (4.575,1.662) (4.59,1.661) (4.605,1.66) (4.62,1.659) (4.635,1.658) (4.65,1.658) (4.665,1.657) (4.68,1.656) (4.695,1.655) (4.71,1.654) (4.725,1.653) (4.74,1.652) (4.755,1.651) (4.77,1.65) (4.785,1.649) (4.8,1.648) (4.815,1.647) (4.83,1.646) (4.845,1.645) (4.86,1.644) (4.875,1.643) (4.89,1.642) (4.905,1.641) (4.92,1.64) (4.935,1.639) (4.95,1.638) (4.965,1.637) (4.98,1.636) (4.995,1.635) (5.01,1.634) (5.025,1.633) (5.04,1.631) (5.055,1.63) (5.07,1.629) (5.085,1.628) (5.1,1.627) (5.115,1.626) (5.13,1.625) (5.145,1.624) (5.16,1.623) (5.175,1.621) (5.19,1.62) (5.205,1.619) (5.22,1.618) (5.235,1.617) (5.25,1.616) (5.265,1.615) (5.28,1.613) (5.295,1.612) (5.31,1.611) (5.325,1.61) (5.34,1.609) (5.355,1.607) (5.37,1.606) (5.385,1.605) (5.4,1.604) (5.415,1.603) (5.43,1.601) (5.445,1.6) (5.46,1.599) (5.475,1.598) (5.49,1.597) (5.505,1.595) (5.52,1.594) (5.535,1.593) (5.55,1.592) (5.565,1.59) (5.58,1.589) (5.595,1.588) (5.61,1.587) (5.625,1.585) (5.64,1.584) (5.655,1.583) (5.67,1.582) (5.685,1.58) (5.7,1.579) (5.715,1.578) (5.73,1.577) (5.745,1.575) (5.76,1.574) (5.775,1.573) (5.79,1.571) (5.805,1.57) (5.82,1.569) (5.835,1.568) (5.85,1.566) (5.865,1.565) (5.88,1.564) (5.895,1.562) (5.91,1.561) (5.925,1.56) (5.94,1.558) (5.955,1.557) (5.97,1.556) (5.985,1.555) (6,1.553)};547\draw[thick,fwd] plot coordinates {(0,2.172) (0.015,2.168) (0.03,2.163) (0.045,2.158) (0.06,2.153) (0.075,2.149) (0.09,2.144) (0.105,2.139) (0.12,2.134) (0.135,2.13) (0.15,2.125) (0.165,2.12) (0.18,2.116) (0.195,2.111) (0.21,2.106) (0.225,2.101) (0.24,2.097) (0.255,2.092) (0.27,2.087) (0.285,2.083) (0.3,2.078) (0.315,2.073) (0.33,2.068) (0.345,2.064) (0.36,2.059) (0.375,2.054) (0.39,2.049) (0.405,2.045) (0.42,2.04) (0.435,2.035) (0.45,2.031) (0.465,2.026) (0.48,2.021) (0.495,2.016) (0.51,2.012) (0.525,2.007) (0.54,2.002) (0.555,1.998) (0.57,1.993) (0.585,1.988) (0.6,1.983) (0.615,1.979) (0.63,1.974) (0.645,1.969) (0.66,1.964) (0.675,1.96) (0.69,1.955) (0.705,1.95) (0.72,1.946) (0.735,1.941) (0.75,1.936) (0.765,1.931) (0.78,1.927) (0.795,1.922) (0.81,1.917) (0.825,1.913) (0.84,1.908) (0.855,1.903) (0.87,1.898) (0.885,1.894) (0.9,1.889) (0.915,1.884) (0.93,1.88) (0.945,1.875) (0.96,1.87) (0.975,1.865) (0.99,1.861) (1.005,1.856) (1.02,1.851) (1.035,1.847) (1.05,1.842) (1.065,1.837) (1.08,1.832) (1.095,1.828) (1.11,1.823) (1.125,1.818) (1.14,1.814) (1.155,1.809) (1.17,1.804) (1.185,1.799) (1.2,1.795) (1.215,1.79) (1.23,1.785) (1.245,1.781) (1.26,1.776) (1.275,1.771) (1.29,1.767) (1.305,1.762) (1.32,1.757) (1.335,1.753) (1.35,1.748) (1.365,1.743) (1.38,1.738) (1.395,1.734) (1.41,1.729) (1.425,1.724) (1.44,1.72) (1.455,1.715) (1.47,1.71) (1.485,1.706) (1.5,1.701) (1.515,1.697) (1.53,1.692) (1.545,1.687) (1.56,1.683) (1.575,1.678) (1.59,1.673) (1.605,1.669) (1.62,1.664) (1.635,1.66) (1.65,1.655) (1.665,1.651) (1.68,1.646) (1.695,1.641) (1.71,1.637) (1.725,1.632) (1.74,1.628) (1.755,1.623) (1.77,1.619) (1.785,1.615) (1.8,1.61) (1.815,1.606) (1.83,1.601) (1.845,1.597) (1.86,1.593) (1.875,1.588) (1.89,1.584) (1.905,1.58) (1.92,1.575) (1.935,1.571) (1.95,1.567) (1.965,1.563) (1.98,1.559) (1.995,1.555) (2.01,1.551) (2.025,1.547) (2.04,1.543) (2.055,1.539) (2.07,1.535) (2.085,1.532) (2.1,1.528) (2.115,1.525) (2.13,1.521) (2.145,1.518) (2.16,1.514) (2.175,1.511) (2.19,1.508) (2.205,1.505) (2.22,1.502) (2.235,1.499) (2.25,1.496) (2.265,1.494) (2.28,1.491) (2.295,1.489) (2.31,1.487) (2.325,1.485) (2.34,1.483) (2.355,1.482) (2.37,1.48) (2.385,1.479) (2.4,1.478) (2.415,1.477) (2.43,1.477) (2.445,1.476) (2.46,1.476) (2.475,1.477) (2.49,1.477) (2.505,1.478) (2.52,1.479) (2.535,1.48) (2.55,1.482) (2.565,1.484) (2.58,1.486) (2.595,1.489) (2.61,1.492) (2.625,1.496) (2.64,1.5) (2.655,1.504) (2.67,1.509) (2.685,1.514) (2.7,1.519) (2.715,1.525) (2.73,1.531) (2.745,1.538) (2.76,1.545) (2.775,1.553) (2.79,1.56) (2.805,1.569) (2.82,1.577) (2.835,1.586) (2.85,1.595) (2.865,1.605) (2.88,1.615) (2.895,1.625) (2.91,1.635) (2.925,1.646) (2.94,1.656) (2.955,1.667) (2.97,1.678) (2.985,1.689) (3,1.7) (3.015,1.711) (3.03,1.722) (3.045,1.733) (3.06,1.744) (3.075,1.754) (3.09,1.765) (3.105,1.775) (3.12,1.785) (3.135,1.795) (3.15,1.805) (3.165,1.814) (3.18,1.823) (3.195,1.831) (3.21,1.84) (3.225,1.847) (3.24,1.855) (3.255,1.862) (3.27,1.869) (3.285,1.875) (3.3,1.881) (3.315,1.886) (3.33,1.891) (3.345,1.896) (3.36,1.9) (3.375,1.904) (3.39,1.908) (3.405,1.911) (3.42,1.914) (3.435,1.916) (3.45,1.918) (3.465,1.92) (3.48,1.921) (3.495,1.922) (3.51,1.923) (3.525,1.923) (3.54,1.924) (3.555,1.924) (3.57,1.923) (3.585,1.923) (3.6,1.922) (3.615,1.921) (3.63,1.92) (3.645,1.918) (3.66,1.917) (3.675,1.915) (3.69,1.913) (3.705,1.911) (3.72,1.909) (3.735,1.906) (3.75,1.904) (3.765,1.901) (3.78,1.898) (3.795,1.895) (3.81,1.892) (3.825,1.889) (3.84,1.886) (3.855,1.882) (3.87,1.879) (3.885,1.875) (3.9,1.872) (3.915,1.868) (3.93,1.865) (3.945,1.861) (3.96,1.857) (3.975,1.853) (3.99,1.849) (4.005,1.845) (4.02,1.841) (4.035,1.837) (4.05,1.833) (4.065,1.829) (4.08,1.825) (4.095,1.82) (4.11,1.816) (4.125,1.812) (4.14,1.807) (4.155,1.803) (4.17,1.799) (4.185,1.794) (4.2,1.79) (4.215,1.785) (4.23,1.781) (4.245,1.777) (4.26,1.772) (4.275,1.768) (4.29,1.763) (4.305,1.759) (4.32,1.754) (4.335,1.749) (4.35,1.745) (4.365,1.74) (4.38,1.736) (4.395,1.731) (4.41,1.727) (4.425,1.722) (4.44,1.717) (4.455,1.713) (4.47,1.708) (4.485,1.703) (4.5,1.699) (4.515,1.694) (4.53,1.69) (4.545,1.685) (4.56,1.68) (4.575,1.676) (4.59,1.671) (4.605,1.666) (4.62,1.662) (4.635,1.657) (4.65,1.652) (4.665,1.647) (4.68,1.643) (4.695,1.638) (4.71,1.633) (4.725,1.629) (4.74,1.624) (4.755,1.619) (4.77,1.615) (4.785,1.61) (4.8,1.605) (4.815,1.601) (4.83,1.596) (4.845,1.591) (4.86,1.586) (4.875,1.582) (4.89,1.577) (4.905,1.572) (4.92,1.568) (4.935,1.563) (4.95,1.558) (4.965,1.553) (4.98,1.549) (4.995,1.544) (5.01,1.539) (5.025,1.535) (5.04,1.53) (5.055,1.525) (5.07,1.52) (5.085,1.516) (5.1,1.511) (5.115,1.506) (5.13,1.502) (5.145,1.497) (5.16,1.492) (5.175,1.487) (5.19,1.483) (5.205,1.478) (5.22,1.473) (5.235,1.469) (5.25,1.464) (5.265,1.459) (5.28,1.454) (5.295,1.45) (5.31,1.445) (5.325,1.44) (5.34,1.436) (5.355,1.431) (5.37,1.426) (5.385,1.421) (5.4,1.417) (5.415,1.412) (5.43,1.407) (5.445,1.402) (5.46,1.398) (5.475,1.393) (5.49,1.388) (5.505,1.384) (5.52,1.379) (5.535,1.374) (5.55,1.369) (5.565,1.365) (5.58,1.36) (5.595,1.355) (5.61,1.351) (5.625,1.346) (5.64,1.341) (5.655,1.336) (5.67,1.332) (5.685,1.327) (5.7,1.322) (5.715,1.317) (5.73,1.313) (5.745,1.308) (5.76,1.303) (5.775,1.299) (5.79,1.294) (5.805,1.289) (5.82,1.284) (5.835,1.28) (5.85,1.275) (5.865,1.27) (5.88,1.266) (5.895,1.261) (5.91,1.256) (5.925,1.251) (5.94,1.247) (5.955,1.242) (5.97,1.237) (5.985,1.232) (6,1.228)};548\draw[thick,bwd] plot coordinates {(0,3.117) (0.015,3.102) (0.03,3.088) (0.045,3.074) (0.06,3.06) (0.075,3.046) (0.09,3.032) (0.105,3.017) (0.12,3.003) (0.135,2.989) (0.15,2.975) (0.165,2.961) (0.18,2.947) (0.195,2.933) (0.21,2.918) (0.225,2.904) (0.24,2.89) (0.255,2.876) (0.27,2.862) (0.285,2.848) (0.3,2.833) (0.315,2.819) (0.33,2.805) (0.345,2.791) (0.36,2.777) (0.375,2.763) (0.39,2.748) (0.405,2.734) (0.42,2.72) (0.435,2.706) (0.45,2.692) (0.465,2.678) (0.48,2.663) (0.495,2.649) (0.51,2.635) (0.525,2.621) (0.54,2.607) (0.555,2.593) (0.57,2.578) (0.585,2.564) (0.6,2.55) (0.615,2.536) (0.63,2.522) (0.645,2.508) (0.66,2.493) (0.675,2.479) (0.69,2.465) (0.705,2.451) (0.72,2.437) (0.735,2.423) (0.75,2.408) (0.765,2.394) (0.78,2.38) (0.795,2.366) (0.81,2.352) (0.825,2.338) (0.84,2.323) (0.855,2.309) (0.87,2.295) (0.885,2.281) (0.9,2.267) (0.915,2.253) (0.93,2.238) (0.945,2.224) (0.96,2.21) (0.975,2.196) (0.99,2.182) (1.005,2.168) (1.02,2.153) (1.035,2.139) (1.05,2.125) (1.065,2.111) (1.08,2.097) (1.095,2.083) (1.11,2.068) (1.125,2.054) (1.14,2.04) (1.155,2.026) (1.17,2.012) (1.185,1.998) (1.2,1.983) (1.215,1.969) (1.23,1.955) (1.245,1.941) (1.26,1.927) (1.275,1.913) (1.29,1.898) (1.305,1.884) (1.32,1.87) (1.335,1.856) (1.35,1.842) (1.365,1.828) (1.38,1.813) (1.395,1.799) (1.41,1.785) (1.425,1.771) (1.44,1.757) (1.455,1.743) (1.47,1.728) (1.485,1.714) (1.5,1.7) (1.515,1.686) (1.53,1.672) (1.545,1.658) (1.56,1.643) (1.575,1.629) (1.59,1.615) (1.605,1.601) (1.62,1.587) (1.635,1.573) (1.65,1.558) (1.665,1.544) (1.68,1.53) (1.695,1.516) (1.71,1.502) (1.725,1.488) (1.74,1.473) (1.755,1.459) (1.77,1.445) (1.785,1.431) (1.8,1.417) (1.815,1.403) (1.83,1.388) (1.845,1.374) (1.86,1.36) (1.875,1.346) (1.89,1.332) (1.905,1.318) (1.92,1.303) (1.935,1.289) (1.95,1.275) (1.965,1.261) (1.98,1.247) (1.995,1.233) (2.01,1.218) (2.025,1.204) (2.04,1.19) (2.055,1.176) (2.07,1.162) (2.085,1.148) (2.1,1.133) (2.115,1.119) (2.13,1.105) (2.145,1.091) (2.16,1.077) (2.175,1.063) (2.19,1.048) (2.205,1.034) (2.22,1.02) (2.235,1.006) (2.25,0.992) (2.265,0.978) (2.28,0.964) (2.295,0.949) (2.31,0.935) (2.325,0.921) (2.34,0.907) (2.355,0.893) (2.37,0.879) (2.385,0.865) (2.4,0.851) (2.415,0.837) (2.43,0.823) (2.445,0.809) (2.46,0.795) (2.475,0.782) (2.49,0.768) (2.505,0.755) (2.52,0.741) (2.535,0.728) (2.55,0.715) (2.565,0.703) (2.58,0.69) (2.595,0.679) (2.61,0.667) (2.625,0.656) (2.64,0.646) (2.655,0.637) (2.67,0.629) (2.685,0.623) (2.7,0.618) (2.715,0.614) (2.73,0.614) (2.745,0.616) (2.76,0.621) (2.775,0.63) (2.79,0.644) (2.805,0.663) (2.82,0.689) (2.835,0.722) (2.85,0.763) (2.865,0.813) (2.88,0.873) (2.895,0.943) (2.91,1.024) (2.925,1.116) (2.94,1.218) (2.955,1.33) (2.97,1.449) (2.985,1.573) (3,1.7) (3.015,1.827) (3.03,1.951) (3.045,2.07) (3.06,2.182) (3.075,2.284) (3.09,2.376) (3.105,2.457) (3.12,2.527) (3.135,2.587) (3.15,2.637) (3.165,2.678) (3.18,2.711) (3.195,2.737) (3.21,2.756) (3.225,2.77) (3.24,2.779) (3.255,2.784) (3.27,2.786) (3.285,2.786) (3.3,2.782) (3.315,2.777) (3.33,2.771) (3.345,2.763) (3.36,2.754) (3.375,2.744) (3.39,2.733) (3.405,2.721) (3.42,2.71) (3.435,2.697) (3.45,2.685) (3.465,2.672) (3.48,2.659) (3.495,2.645) (3.51,2.632) (3.525,2.618) (3.54,2.605) (3.555,2.591) (3.57,2.577) (3.585,2.563) (3.6,2.549) (3.615,2.535) (3.63,2.521) (3.645,2.507) (3.66,2.493) (3.675,2.479) (3.69,2.465) (3.705,2.451) (3.72,2.436) (3.735,2.422) (3.75,2.408) (3.765,2.394) (3.78,2.38) (3.795,2.366) (3.81,2.352) (3.825,2.337) (3.84,2.323) (3.855,2.309) (3.87,2.295) (3.885,2.281) (3.9,2.267) (3.915,2.252) (3.93,2.238) (3.945,2.224) (3.96,2.21) (3.975,2.196) (3.99,2.182) (4.005,2.167) (4.02,2.153) (4.035,2.139) (4.05,2.125) (4.065,2.111) (4.08,2.097) (4.095,2.082) (4.11,2.068) (4.125,2.054) (4.14,2.04) (4.155,2.026) (4.17,2.012) (4.185,1.997) (4.2,1.983) (4.215,1.969) (4.23,1.955) (4.245,1.941) (4.26,1.927) (4.275,1.912) (4.29,1.898) (4.305,1.884) (4.32,1.87) (4.335,1.856) (4.35,1.842) (4.365,1.827) (4.38,1.813) (4.395,1.799) (4.41,1.785) (4.425,1.771) (4.44,1.757) (4.455,1.742) (4.47,1.728) (4.485,1.714) (4.5,1.7) (4.515,1.686) (4.53,1.672) (4.545,1.657) (4.56,1.643) (4.575,1.629) (4.59,1.615) (4.605,1.601) (4.62,1.587) (4.635,1.572) (4.65,1.558) (4.665,1.544) (4.68,1.53) (4.695,1.516) (4.71,1.502) (4.725,1.487) (4.74,1.473) (4.755,1.459) (4.77,1.445) (4.785,1.431) (4.8,1.417) (4.815,1.402) (4.83,1.388) (4.845,1.374) (4.86,1.36) (4.875,1.346) (4.89,1.332) (4.905,1.317) (4.92,1.303) (4.935,1.289) (4.95,1.275) (4.965,1.261) (4.98,1.247) (4.995,1.232) (5.01,1.218) (5.025,1.204) (5.04,1.19) (5.055,1.176) (5.07,1.162) (5.085,1.147) (5.1,1.133) (5.115,1.119) (5.13,1.105) (5.145,1.091) (5.16,1.077) (5.175,1.062) (5.19,1.048) (5.205,1.034) (5.22,1.02) (5.235,1.006) (5.25,0.992) (5.265,0.977) (5.28,0.963) (5.295,0.949) (5.31,0.935) (5.325,0.921) (5.34,0.907) (5.355,0.892) (5.37,0.878) (5.385,0.864) (5.4,0.85) (5.415,0.836) (5.43,0.822) (5.445,0.807) (5.46,0.793) (5.475,0.779) (5.49,0.765) (5.505,0.751) (5.52,0.737) (5.535,0.722) (5.55,0.708) (5.565,0.694) (5.58,0.68) (5.595,0.666) (5.61,0.652) (5.625,0.637) (5.64,0.623) (5.655,0.609) (5.67,0.595) (5.685,0.581) (5.7,0.567) (5.715,0.552) (5.73,0.538) (5.745,0.524) (5.76,0.51) (5.775,0.496) (5.79,0.482) (5.805,0.467) (5.82,0.453) (5.835,0.439) (5.85,0.425) (5.865,0.411) (5.88,0.397) (5.895,0.382) (5.91,0.368) (5.925,0.354) (5.94,0.34) (5.955,0.326) (5.97,0.312) (5.985,0.297) (6,0.283)};549\draw[accent,very thin,densely dashed] (1.5,0)--(1.5,3.4);550\draw[accent,very thin,densely dashed] (4.5,0)--(4.5,3.4);551\node[bwd,right] at (6.075,0.283) {\scriptsize $t=0.1$};552\node[fwd,right] at (6.075,1.133) {\scriptsize $0.3$};553\node[ink2,right] at (6.075,1.615) {\scriptsize $1$};554\end{scope}555\end{tikzpicture}556\caption{The score $s(x,t)=-x/t+\tanh(x/t)/t$ of data on the two points $\pm1$ (dashed), at three noise levels. As $t\to0$ it becomes a restoring force towards each point, joined by a transition of slope $1/t^2$ at the midpoint.}\label{fig:score}557\end{figure}558 559\noindent For $T=1$ and $k=10$ the factor in (a) is $5.2\times10^{21}$. In the example the flow from $T=1$ to $0$560preserves the value of the distribution function along five trajectories to $10^{-14}$, and a perturbation