Complete flattened LaTeX source
\documentclass[10pt,twocolumn]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[sc]{mathpazo}
\usepackage[scaled=0.9]{helvet}
\usepackage[letterpaper,margin=0.78in,columnsep=0.3in]{geometry}
\usepackage{amsmath,amssymb,amsthm,mathtools}
\usepackage{microtype}
\usepackage{xcolor}
\definecolor{accent}{HTML}{7A1F2B}
\definecolor{ink2}{HTML}{55595F}
\definecolor{fwd}{HTML}{1F4E79}
\definecolor{bwd}{HTML}{5B3A7A}
\usepackage{tikz,tikz-cd}
\usetikzlibrary{arrows.meta,positioning}
\usepackage{booktabs,array}
\usepackage{enumitem}
\usepackage[font=small,labelfont={bf,sf}]{caption}
\usepackage[colorlinks=true,linkcolor=accent,citecolor=accent,urlcolor=accent,
pdftitle={Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates},
pdfauthor={Anonymous},
pdfsubject={Partial differential equations as a framework for convolution, generative diffusion, graph networks and learning dynamics},
pdfkeywords={partial differential equations, machine learning, boundary conditions, generative diffusion, graph networks, numerical stability, spectral rates}]{hyperref}
\usepackage[nameinlink,capitalise,noabbrev]{cleveref}
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
\newcommand{\stag}[2]{{\footnotesize\sffamily\upshape\color{#1}[#2]}\enspace\ignorespaces}
\newcommand{\Scl}[1][]{\stag{ink2}{classical\if\relax\detokenize{#1}\relax\else; #1\fi}}
\newcommand{\Sli}[1][]{\stag{fwd}{literature\if\relax\detokenize{#1}\relax\else; #1\fi}}
\newcommand{\Srf}[1][]{\stag{accent}{reformulation\if\relax\detokenize{#1}\relax\else; cf.\ #1\fi}}
\newcommand{\Sob}{\stag{accent}{observation}}
\newcommand{\Spg}{\stag{bwd}{programme}}
\newcommand{\chk}[1]{{\footnotesize\sffamily\upshape\color{accent}[#1]}}
\newcommand{\cat}[1]{\mathsf{#1}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\one}{\mathbf{1}}
\newcommand{\T}{\mathsf{T}}
\newcommand{\Lap}{\Delta}
\newcommand{\pd}{\partial}
\newcommand{\KL}[2]{D(#1\,\Vert\,#2)}
\newcommand{\Var}{\operatorname{Var}}
\newcommand{\relu}{\operatorname{relu}}
\newcommand{\sinc}{\operatorname{sinc}}
\newcommand{\term}[1]{\textbf{\itshape #1}}
\setlength{\parskip}{0pt}
\frenchspacing
\raggedbottom
\title{\textbf{Partial Differential Equations in Machine Learning and AI:\\ Naturality, Well-Posedness, and Spectral Rates}}
\author{Anonymous}
\date{Higher Categorical Semantics \quad$\cdot$\quad Version 1.0, 18 November 2026}
\begin{document}
\twocolumn[
\maketitle
\begin{center}\begin{minipage}{0.86\textwidth}\small
\textbf{Abstract.} Convolutional layers are difference operators, depth is a time variable, generative diffusion is
the heat equation, message passing is a graph Laplacian, and a kernel is a Green's function. We organize these
identifications by three questions that the theory of partial differential equations asks of any construction.
\emph{Is it natural}, that is, does it commute with the maps of domains, resolutions and symmetries along which
one wants to move it? \emph{Is it well-posed}, that is, does the output depend continuously on the input?
\emph{At what rate} does it act on each mode? Each qualitative answer is paired with a computed one. Naturality:
half-sample reflection models the Neumann condition. Circular layers preserve the reflected subspace exactly
when the periodized kernels are symmetric.
zero padding is the Dirichlet condition only for three-point stencils; Fourier multipliers commute with
resampling, stencils do so to second order, squaring needs more than $3K$ points for band $K$, and a rectifier
is aliased at order $N^{-2}$ on every grid. Well-posedness: reversing diffusion amplifies mode $10$ by
$5\times10^{21}$, while the reverse flow driven by the score moves samples by $0.26$ per unit of score error, the
difficulty reappearing as a score of slope $1/t^2$. Forward Euler amplifies nonzero oscillatory modes. In the
stable range $0<h\omega<2$, the leapfrog condition number is at most $(1+h\omega/2)/(1-h\omega/2)$. A strong-form loss squares the
condition number of the energy loss, $414$ against $171{,}682$ on $31$ unknowns. Rates: oversmoothing proceeds
at $1-\tau\lambda_2$ per layer, with depth scales $415$, $5$ and $1$ on a path, a hypercube and a complete graph
of $32$ nodes; sheaf diffusion converges to global sections, which exist for $16$ of the $128$ sign sheaves on a
small graph; kernel regression learns mode $k$ at rate $1/(1+k^2)$ plus aliases; and the spectral gap of
Langevin dynamics in a double well approaches the Kramers rate at low noise. All numerical statements are recomputed by a companion
script with $42$ checks. No new theorem is claimed; the contribution is the organization, the exact constants,
and the three questions asked uniformly.
\end{minipage}\end{center}
\vspace{1.2em}
]
\section{Introduction}
The companion volume \cite{PDEHC} retells an introductory course on partial differential equations
\cite{Str08} around one question: which constructions are natural for maps of domains, of equations and of
function spaces. This paper asks the same question of machine learning, where the same equations have been
rediscovered piecemeal. A convolutional layer with a three-point kernel is a difference operator, and its
padding rule is a boundary condition \cite{Str99}. A residual network is a difference scheme in a time variable
called depth \cite{E17,HaRu17,CRBD18}. A diffusion model noises data with the heat equation and samples by
reversing it \cite{SWMG15,HJA20,SSKKEP21}. Message passing on a graph is a discrete diffusion
\cite{CRGBB21}, or a discrete wave equation \cite{RCRMB22}, or a diffusion of sections of a sheaf
\cite{HaGh19,BDCLB22}. Kernel regression inverts an elliptic operator, physics-informed networks minimize the
residual of an equation \cite{RPK19}, and operator learning approximates its solution map
\cite{LKALBSA21,KLLABSA23,LJPZK21}. A recent survey organizes the last two families \cite{DCWJXKY26}.
Three questions organize the theory of the equations themselves, and we ask each of them of each construction.
\emph{Naturality.} A domain can be reflected, translated, refined or glued, and a construction is natural if it
commutes with the corresponding maps. Equivariance \cite{CoWe16,BBCV21} is naturality for symmetries;
discretization invariance and the absence of aliasing \cite{KALHHLA21,BBRMMA23} are naturality for changes of
resolution.
\emph{Well-posedness.} Hadamard's criterion asks that a solution exist, be unique, and depend continuously on
the data. Learning problems fail the third condition in characteristic ways: by running a parabolic equation
backwards, by discretizing a conservative system with a dissipative scheme or the reverse, or by squaring an
operator in the loss.
\emph{Rates.} For linear constructions the answer to ``how fast'' is a spectrum. Oversmoothing, spectral bias,
mixing of stochastic gradient descent and the rank of a learned operator are all statements about eigenvalues
of a Laplacian.
\paragraph{What is and is not claimed.} Every statement below carries a status tag: classical, literature,
reformulation, observation or programme. Almost all are classical or known; where the form or the constant
seems not to be standard we say ``observation''. The checks, tagged like \chk{N3}, are performed by the companion
script \texttt{verify-pde-learning.py}, which uses only the Python standard library and a fixed seed, and are
listed in \cref{app:checks}. They recompute instances; they are evidence and not proofs. Most models are linear
and one-dimensional, so that every number is exact or accurate to many digits; \cref{sec:limits} says what this
leaves out. The references were assembled from memory, except for \cite{DCWJXKY26,BBRMMA23,BDCLB22}, which were
checked online on 17 November 2026.
\section{Setting}\label{sec:setting}
Let $\cat{Dom}$ be a category of domains: intervals, circles, grids, graphs, with the maps of interest between
them (inclusions, coverings, symmetries, refinements). A \term{signal space} is a functor
$F\colon\cat{Dom}^{\mathrm{op}}\to\cat{Vect}$, such as functions on the domain with restriction or pullback. A
\term{layer} is a family of maps $T_D\colon F(D)\to F(D)$, not necessarily linear.
\begin{definition}\label{def:natural}
The layer $T$ is \term{natural} for a map $f\colon D\to D'$ if the square
\[\begin{tikzcd}[column sep=large]
F(D')\arrow[r,"T_{D'}"]\arrow[d,"f^*"'] & F(D')\arrow[d,"f^*"]\\
F(D)\arrow[r,"T_D"'] & F(D)
\end{tikzcd}\]
commutes. Its \term{defect} on an input $x$ is $\lVert f^*T_{D'}x-T_Df^*x\rVert$, relative to the norm of either
term.
\end{definition}
\noindent For $f$ a symmetry of $D$ this is equivariance. For $f$ the inclusion of a coarse grid in a fine one,
with $F$ restricted to band-limited signals, it is discretization invariance. For $f$ a covering map and $F$
replaced by its invariant part, it is the method of images, with which we begin. A \term{depth semigroup} is a
family $T^t$ with $T^{t+s}=T^tT^s$, and a network of $N$ layers of step $h$ is a discretization of $T^{Nh}$;
naturality in $t$ is consistency of a scheme, and well-posedness is its stability \cite[Ch.~12]{PDEHC}.
\section{Padding is a boundary condition}\label{sec:padding}
Let $x\in\R^n$ and let $w=(w_{-r},\dots,w_r)$ be a kernel. The layer is $(C_wx)_i=\sum_jw_jx_{i+j}$, and a
\term{padding} rule supplies the values $x_i$ for $i$ outside $0,\dots,n-1$: \emph{zero} ($x_i=0$),
\emph{circular} ($x_{i+n}=x_i$) or \emph{reflect} ($x_{-1-i}=x_i$ and $x_{n+i}=x_{n-1-i}$).
\begin{proposition}\label{prop:laplacians}
\Scl[\cite{Str99}] With the kernel $(-1,2,-1)$ the three rules give the Dirichlet, periodic and Neumann
difference Laplacians. They are diagonalized by the sine transform $\sin\frac{\pi k(j+1)}{n+1}$, the discrete
Fourier transform, and the cosine transform $\cos\frac{\pi k(j+1/2)}n$, with eigenvalues
$4\sin^2\frac{\pi k}{2(n+1)}$, $4\sin^2\frac{\pi k}n$ and $4\sin^2\frac{\pi k}{2n}$. \chk{N1, N2}
\end{proposition}
\begin{figure}[t]
\centering
\begin{tikzpicture}
\fill[fwd!8] (-0.2,-0.72) rectangle (2.2,0.78);
\draw[ink2] (-3.04,0)--(5.04,0);
\draw[accent,densely dashed] (-0.2,-0.72)--(-0.2,0.78);
\draw[accent,densely dashed] (2.2,-0.72)--(2.2,0.78);
\draw[thick,bwd] (-2.8,0)--(-2.8,0.66);
\fill[bwd] (-2.8,0.66) circle (1.3pt);
\draw[thick,bwd] (-2.4,0)--(-2.4,0.66);
\fill[bwd] (-2.4,0.66) circle (1.3pt);
\draw[thick,bwd] (-2,0)--(-2,0.363);
\fill[bwd] (-2,0.363) circle (1.3pt);
\draw[thick,bwd] (-1.6,0)--(-1.6,0.198);
\fill[bwd] (-1.6,0.198) circle (1.3pt);
\draw[thick,bwd] (-1.2,0)--(-1.2,0.462);
\fill[bwd] (-1.2,0.462) circle (1.3pt);
\draw[thick,bwd] (-0.8,0)--(-0.8,0.594);
\fill[bwd] (-0.8,0.594) circle (1.3pt);
\draw[thick,bwd] (-0.4,0)--(-0.4,0.33);
\fill[bwd] (-0.4,0.33) circle (1.3pt);
\draw[thick,fwd] (0,0)--(0,0.33);
\fill[fwd] (0,0.33) circle (1.3pt);
\draw[thick,fwd] (0.4,0)--(0.4,0.594);
\fill[fwd] (0.4,0.594) circle (1.3pt);
\draw[thick,fwd] (0.8,0)--(0.8,0.462);
\fill[fwd] (0.8,0.462) circle (1.3pt);
\draw[thick,fwd] (1.2,0)--(1.2,0.198);
\fill[fwd] (1.2,0.198) circle (1.3pt);
\draw[thick,fwd] (1.6,0)--(1.6,0.363);
\fill[fwd] (1.6,0.363) circle (1.3pt);
\draw[thick,fwd] (2,0)--(2,0.66);
\fill[fwd] (2,0.66) circle (1.3pt);
\draw[thick,bwd] (2.4,0)--(2.4,0.66);
\fill[bwd] (2.4,0.66) circle (1.3pt);
\draw[thick,bwd] (2.8,0)--(2.8,0.363);
\fill[bwd] (2.8,0.363) circle (1.3pt);
\draw[thick,bwd] (3.2,0)--(3.2,0.198);
\fill[bwd] (3.2,0.198) circle (1.3pt);
\draw[thick,bwd] (3.6,0)--(3.6,0.462);
\fill[bwd] (3.6,0.462) circle (1.3pt);
\draw[thick,bwd] (4,0)--(4,0.594);
\fill[bwd] (4,0.594) circle (1.3pt);
\draw[thick,bwd] (4.4,0)--(4.4,0.33);
\fill[bwd] (4.4,0.33) circle (1.3pt);
\draw[thick,bwd] (4.8,0)--(4.8,0.33);
\fill[bwd] (4.8,0.33) circle (1.3pt);
\node[anchor=west] at (-3.04,1.02) {\scriptsize reflect: even extension};
\fill[fwd!8] (-0.2,-2.72) rectangle (2.2,-1.22);
\draw[ink2] (-3.04,-2)--(5.04,-2);
\draw[accent,densely dashed] (-0.4,-2.72)--(-0.4,-1.22);
\draw[accent,densely dashed] (2.4,-2.72)--(2.4,-1.22);
\draw[thick,bwd] (-2.8,-2)--(-2.8,-2.66);
\fill[bwd] (-2.8,-2.66) circle (1.3pt);
\draw[thick,bwd] (-2.4,-2)--(-2.4,-2.363);
\fill[bwd] (-2.4,-2.363) circle (1.3pt);
\draw[thick,bwd] (-2,-2)--(-2,-2.198);
\fill[bwd] (-2,-2.198) circle (1.3pt);
\draw[thick,bwd] (-1.6,-2)--(-1.6,-2.462);
\fill[bwd] (-1.6,-2.462) circle (1.3pt);
\draw[thick,bwd] (-1.2,-2)--(-1.2,-2.594);
\fill[bwd] (-1.2,-2.594) circle (1.3pt);
\draw[thick,bwd] (-0.8,-2)--(-0.8,-2.33);
\fill[bwd] (-0.8,-2.33) circle (1.3pt);
\draw[thick,bwd] (-0.4,-2)--(-0.4,-2);
\fill[bwd] (-0.4,-2) circle (1.3pt);
\draw[thick,fwd] (0,-2)--(0,-1.67);
\fill[fwd] (0,-1.67) circle (1.3pt);
\draw[thick,fwd] (0.4,-2)--(0.4,-1.406);
\fill[fwd] (0.4,-1.406) circle (1.3pt);
\draw[thick,fwd] (0.8,-2)--(0.8,-1.538);
\fill[fwd] (0.8,-1.538) circle (1.3pt);
\draw[thick,fwd] (1.2,-2)--(1.2,-1.802);
\fill[fwd] (1.2,-1.802) circle (1.3pt);
\draw[thick,fwd] (1.6,-2)--(1.6,-1.637);
\fill[fwd] (1.6,-1.637) circle (1.3pt);
\draw[thick,fwd] (2,-2)--(2,-1.34);
\fill[fwd] (2,-1.34) circle (1.3pt);
\draw[thick,bwd] (2.4,-2)--(2.4,-2);
\fill[bwd] (2.4,-2) circle (1.3pt);
\draw[thick,bwd] (2.8,-2)--(2.8,-2.66);
\fill[bwd] (2.8,-2.66) circle (1.3pt);
\draw[thick,bwd] (3.2,-2)--(3.2,-2.363);
\fill[bwd] (3.2,-2.363) circle (1.3pt);
\draw[thick,bwd] (3.6,-2)--(3.6,-2.198);
\fill[bwd] (3.6,-2.198) circle (1.3pt);
\draw[thick,bwd] (4,-2)--(4,-2.462);
\fill[bwd] (4,-2.462) circle (1.3pt);
\draw[thick,bwd] (4.4,-2)--(4.4,-2.594);
\fill[bwd] (4.4,-2.594) circle (1.3pt);
\draw[thick,bwd] (4.8,-2)--(4.8,-2.33);
\fill[bwd] (4.8,-2.33) circle (1.3pt);
\node[anchor=west] at (-3.04,-0.98) {\scriptsize odd extension (Dirichlet images)};
\fill[fwd!8] (-0.2,-4.72) rectangle (2.2,-3.22);
\draw[ink2] (-3.04,-4)--(5.04,-4);
\draw[thick,bwd] (-2.8,-4)--(-2.8,-4);
\fill[bwd] (-2.8,-4) circle (1.3pt);
\draw[thick,bwd] (-2.4,-4)--(-2.4,-4);
\fill[bwd] (-2.4,-4) circle (1.3pt);
\draw[thick,bwd] (-2,-4)--(-2,-4);
\fill[bwd] (-2,-4) circle (1.3pt);
\draw[thick,bwd] (-1.6,-4)--(-1.6,-4);
\fill[bwd] (-1.6,-4) circle (1.3pt);
\draw[thick,bwd] (-1.2,-4)--(-1.2,-4);
\fill[bwd] (-1.2,-4) circle (1.3pt);
\draw[thick,bwd] (-0.8,-4)--(-0.8,-4);
\fill[bwd] (-0.8,-4) circle (1.3pt);
\draw[thick,bwd] (-0.4,-4)--(-0.4,-4);
\fill[bwd] (-0.4,-4) circle (1.3pt);
\draw[thick,fwd] (0,-4)--(0,-3.67);
\fill[fwd] (0,-3.67) circle (1.3pt);
\draw[thick,fwd] (0.4,-4)--(0.4,-3.406);
\fill[fwd] (0.4,-3.406) circle (1.3pt);
\draw[thick,fwd] (0.8,-4)--(0.8,-3.538);
\fill[fwd] (0.8,-3.538) circle (1.3pt);
\draw[thick,fwd] (1.2,-4)--(1.2,-3.802);
\fill[fwd] (1.2,-3.802) circle (1.3pt);
\draw[thick,fwd] (1.6,-4)--(1.6,-3.637);
\fill[fwd] (1.6,-3.637) circle (1.3pt);
\draw[thick,fwd] (2,-4)--(2,-3.34);
\fill[fwd] (2,-3.34) circle (1.3pt);
\draw[thick,bwd] (2.4,-4)--(2.4,-4);
\fill[bwd] (2.4,-4) circle (1.3pt);
\draw[thick,bwd] (2.8,-4)--(2.8,-4);
\fill[bwd] (2.8,-4) circle (1.3pt);
\draw[thick,bwd] (3.2,-4)--(3.2,-4);
\fill[bwd] (3.2,-4) circle (1.3pt);
\draw[thick,bwd] (3.6,-4)--(3.6,-4);
\fill[bwd] (3.6,-4) circle (1.3pt);
\draw[thick,bwd] (4,-4)--(4,-4);
\fill[bwd] (4,-4) circle (1.3pt);
\draw[thick,bwd] (4.4,-4)--(4.4,-4);
\fill[bwd] (4.4,-4) circle (1.3pt);
\draw[thick,bwd] (4.8,-4)--(4.8,-4);
\fill[bwd] (4.8,-4) circle (1.3pt);
\node[anchor=west] at (-3.04,-2.98) {\scriptsize zero};
\fill[fwd!8] (-0.2,-6.72) rectangle (2.2,-5.22);
\draw[ink2] (-3.04,-6)--(5.04,-6);
\draw[accent,densely dashed] (-0.2,-6.72)--(-0.2,-5.22);
\draw[accent,densely dashed] (2.2,-6.72)--(2.2,-5.22);
\draw[thick,bwd] (-2.8,-6)--(-2.8,-5.34);
\fill[bwd] (-2.8,-5.34) circle (1.3pt);
\draw[thick,bwd] (-2.4,-6)--(-2.4,-5.67);
\fill[bwd] (-2.4,-5.67) circle (1.3pt);
\draw[thick,bwd] (-2,-6)--(-2,-5.406);
\fill[bwd] (-2,-5.406) circle (1.3pt);
\draw[thick,bwd] (-1.6,-6)--(-1.6,-5.538);
\fill[bwd] (-1.6,-5.538) circle (1.3pt);
\draw[thick,bwd] (-1.2,-6)--(-1.2,-5.802);
\fill[bwd] (-1.2,-5.802) circle (1.3pt);
\draw[thick,bwd] (-0.8,-6)--(-0.8,-5.637);
\fill[bwd] (-0.8,-5.637) circle (1.3pt);
\draw[thick,bwd] (-0.4,-6)--(-0.4,-5.34);
\fill[bwd] (-0.4,-5.34) circle (1.3pt);
\draw[thick,fwd] (0,-6)--(0,-5.67);
\fill[fwd] (0,-5.67) circle (1.3pt);
\draw[thick,fwd] (0.4,-6)--(0.4,-5.406);
\fill[fwd] (0.4,-5.406) circle (1.3pt);
\draw[thick,fwd] (0.8,-6)--(0.8,-5.538);
\fill[fwd] (0.8,-5.538) circle (1.3pt);
\draw[thick,fwd] (1.2,-6)--(1.2,-5.802);
\fill[fwd] (1.2,-5.802) circle (1.3pt);
\draw[thick,fwd] (1.6,-6)--(1.6,-5.637);
\fill[fwd] (1.6,-5.637) circle (1.3pt);
\draw[thick,fwd] (2,-6)--(2,-5.34);
\fill[fwd] (2,-5.34) circle (1.3pt);
\draw[thick,bwd] (2.4,-6)--(2.4,-5.67);
\fill[bwd] (2.4,-5.67) circle (1.3pt);
\draw[thick,bwd] (2.8,-6)--(2.8,-5.406);
\fill[bwd] (2.8,-5.406) circle (1.3pt);
\draw[thick,bwd] (3.2,-6)--(3.2,-5.538);
\fill[bwd] (3.2,-5.538) circle (1.3pt);
\draw[thick,bwd] (3.6,-6)--(3.6,-5.802);
\fill[bwd] (3.6,-5.802) circle (1.3pt);
\draw[thick,bwd] (4,-6)--(4,-5.637);
\fill[bwd] (4,-5.637) circle (1.3pt);
\draw[thick,bwd] (4.4,-6)--(4.4,-5.34);
\fill[bwd] (4.4,-5.34) circle (1.3pt);
\draw[thick,bwd] (4.8,-6)--(4.8,-5.67);
\fill[bwd] (4.8,-5.67) circle (1.3pt);
\node[anchor=west] at (-3.04,-4.98) {\scriptsize circular};
\end{tikzpicture}
\caption{Four ways of continuing a signal of six samples (shaded) beyond its ends. Reflect padding is the even extension about the half-sample points (dashed), which is the Neumann condition. The odd extension about the neighbouring grid points is the Dirichlet condition; zero padding agrees with it only on the first sample outside. Circular padding is the periodic extension.}\label{fig:pad}
\end{figure}
The method of images \cite[Thm.~3.5]{PDEHC} explains this construction. In \cref{fig:pad}, the padding reflects
about half-sample points, repeating the end samples. Let $E\colon\R^n\to\R^{2n}$ be the even extension
$Ex=(x_0,\dots,x_{n-1},x_{n-1},\dots,x_0)$, regarded as a signal on the cycle $\Z/2n$, and let $R$ be the
restriction to the first $n$ entries. The image of $E$ is the space of signals invariant under the reflection
$\rho\colon i\mapsto2n-1-i$.
\begin{theorem}[Reflect padding is the Neumann condition]\label{thm:reflect}
\Srf[\cite{Str99}; \cite{PDEHC}, Thm.~3.5] Write $C^{\mathrm{refl}}_w$ and $C^{\mathrm{circ}}_w$ for the layer with
reflect padding on $\R^n$ and circular padding on $\R^{2n}$.
\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]
\item $C^{\mathrm{refl}}_w=R\,C^{\mathrm{circ}}_wE$ for every kernel.
\item $\rho\,C^{\mathrm{circ}}_w\rho=C^{\mathrm{circ}}_{\check w}$ with $\check w_j=w_{-j}$. Hence if $w$ is symmetric,
$C^{\mathrm{circ}}_wE=E\,C^{\mathrm{refl}}_w$, and a network of symmetric kernels, reflect padding and pointwise
nonlinearities is the restriction to invariant signals of a circular network on the doubled cycle.
\item The circular layer preserves the reflected subspace exactly when the kernel, reduced modulo $2n$ by
summing coefficients with the same residue, is symmetric. An asymmetric periodized kernel sends some
even input outside that subspace.
\end{enumerate}
\end{theorem}
\begin{proof}
(a) The reflect rule is the definition of $E$ read at indices outside $0,\dots,n-1$. (b) Substituting
$i\mapsto2n-1-i$ in the sum reverses the kernel. Pointwise maps commute with $E$ and $R$.
For (c), work with the periodized kernel and set $a=w-\check w$. If the circular layer preserves the
reflected subspace, then $C^{\mathrm{circ}}_a$ annihilates the even input $\delta_0+\delta_{-1}$,
by (b). Its output at $i$ is $a_i+a_{i+1}=0$. Since $a_0=0$, all $a_i$ vanish.
\end{proof}
\noindent For a random input and the kernels $(0.2,-0.5,1,-0.5,0.2)$ and $(0.2,-0.5,1,0.7,-0.3)$: identity (a)
holds to rounding for both; the output on the double is invariant to rounding for the first and has an
anti-invariant part of relative size $1.18$ for the second; and two layers on the double differ from two
reflect-padded layers by $0$ when the symmetric kernel comes first and by $0.100$ when the other does.
\chk{N3} The circular model with symmetric kernels is equivariant under the dihedral group of the doubled
cycle. The half-sample construction restricts it to signals fixed by the chosen reflection. The Dirichlet
construction uses the odd extension about the neighbouring grid points.
\begin{proposition}[Zero padding is Dirichlet only for short kernels]\label{prop:zero}
\Sob Let $O\colon\R^n\to\R^{2n+2}$ be the odd extension $(0,x,0,-x^{\mathrm{rev}})$. For a symmetric kernel of radius
one, zero padding equals circular convolution on $Ox$ restricted to the original positions. For radius two or
more it does not: the image construction supplies $-x_0$ beyond the boundary where zero padding supplies $0$.
Pointwise maps commute with $O$ only if they are odd.
\end{proposition}
\noindent The relative difference is $0$ for $(0.3,1,0.3)$ and $0.160$ for the five-point symmetric kernel above.
\chk{N4} The Dirichlet image construction can differ from zero padding for wider kernels. A rectifier can
also destroy the odd symmetry required by the construction.
\begin{proposition}[What padding breaks and keeps]\label{prop:mass}
\Scl \emph{(a)} Circular convolution commutes with translation. With zero padding the commutator with a shift is
supported within the kernel radius of the boundary. \emph{(b)} A layer with a non-negative symmetric kernel of
sum one and reflect padding is doubly stochastic: it conserves the mean and obeys the maximum principle. With
zero padding it is substochastic, and the mass remaining after $m$ layers is the probability that a random
walk with steps distributed by the kernel has not left the interval.
\end{proposition}
\noindent For a five-point kernel on $20$ positions the commutator is supported on positions $0$, $18$ and $19$.
\chk{N5} Position information therefore enters a zero-padded network at the boundary and spreads inwards by one
kernel radius per layer, which is the mechanism behind the observation that such networks learn absolute
position \cite{ILB20}. For the kernel $(\frac14,\frac12,\frac14)$ on $32$ positions, $200$ layers with reflect
padding change the total mass by $2\times10^{-15}$, and with zero padding leave $0.5320$ of it, in agreement
with the sum over the sine basis. \chk{N6}
\section{Resolution: what commutes with resampling}\label{sec:resolution}
Let $B_K$ be the trigonometric polynomials of degree at most $K$ on the circle and
$S_N\colon B_K\to\R^N$ the sampling map on $N$ equally spaced points, injective for $N>2K$. An operator $T$ on
functions with discretizations $T_N$ is natural for refinement (\cref{def:natural}) when $S_NT=T_NS_N$ on $B_K$;
this is the requirement of representation equivalence in \cite{BBRMMA23}.
\begin{theorem}[Three kinds of layer]\label{thm:alias}
\Srf[\cite{BBRMMA23,KALHHLA21}] On $B_K$ and for $N>2K$:
\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]
\item A Fourier multiplier $m(D)$, discretized by the discrete Fourier transform, is natural exactly.
\item The three-point stencil for $-\pd_x^2$ has symbol $k^2\sinc^2(kh/2)$, $h=2\pi/N$: it is natural to second
order.
\item A polynomial nonlinearity of degree $d$ followed by projection to $B_K$ is natural if $N>(d+1)K$, and
for generic inputs only then.
\item A rectifier followed by projection to $B_K$ is natural for no $N$; the defect is of order $N^{-2}$.
\end{enumerate}
\end{theorem}
\begin{proof}
(a) Sampling intertwines the characters $e^{ikx}$, $|k|\le K$, with the discrete ones. (b) Apply the stencil to
$e^{ikx}$. (c) The power has band $dK$, and a frequency $k'$ is sampled onto $k\ne k'$ when $N$ divides $k'-k$;
the largest $|k'-k|$ with $|k|\le K$ is $(d+1)K$. (d) A function with a corner has Fourier coefficients of order
$k^{-2}$, and the sum of those at $k+mN$, $m\neq0$, is of order $N^{-2}$.
\end{proof}
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.5)--(0,0)--(6.3,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $8$};
\draw[ink2] (1.26,0)--(1.26,-0.07) node[below,inner sep=1.5pt]{\scriptsize $16$};
\draw[ink2] (2.52,0)--(2.52,-0.07) node[below,inner sep=1.5pt]{\scriptsize $32$};
\draw[ink2] (3.78,0)--(3.78,-0.07) node[below,inner sep=1.5pt]{\scriptsize $64$};
\draw[ink2] (5.04,0)--(5.04,-0.07) node[below,inner sep=1.5pt]{\scriptsize $128$};
\draw[ink2] (6.3,0)--(6.3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $256$};
\draw[ink2] (0,0)--(-0.07,0) node[left,inner sep=1.5pt]{\scriptsize $10^{-5}$};
\draw[ink2] (0,0.814)--(-0.07,0.814) node[left,inner sep=1.5pt]{\scriptsize $10^{-4}$};
\draw[ink2] (0,1.628)--(-0.07,1.628) node[left,inner sep=1.5pt]{\scriptsize $10^{-3}$};
\draw[ink2] (0,2.441)--(-0.07,2.441) node[left,inner sep=1.5pt]{\scriptsize $10^{-2}$};
\draw[ink2] (0,3.255)--(-0.07,3.255) node[left,inner sep=1.5pt]{\scriptsize $10^{-1}$};
\node[below] at (3.15,-0.38) {\scriptsize grid points $N$};
\draw[ink2,densely dashed] plot coordinates {(0,3.319) (6.3,0.87)};
\draw[thick,fwd] plot coordinates {(0,2.619) (1.26,2.126) (2.52,1.636) (3.78,1.146) (5.04,0.656) (6.3,0.166)};
\draw[thick,bwd] plot coordinates {(1.26,3.149) (2.52,2.434) (3.78,1.89) (5.04,1.321) (6.3,0.933)};
\fill[fwd] (0,2.619) circle (1.6pt);
\fill[fwd] (1.26,2.126) circle (1.6pt);
\fill[fwd] (2.52,1.636) circle (1.6pt);
\fill[fwd] (3.78,1.146) circle (1.6pt);
\fill[fwd] (5.04,0.656) circle (1.6pt);
\fill[fwd] (6.3,0.166) circle (1.6pt);
\fill[bwd] (1.26,3.149) circle (1.6pt);
\fill[bwd] (2.52,2.434) circle (1.6pt);
\fill[bwd] (3.78,1.89) circle (1.6pt);
\fill[bwd] (5.04,1.321) circle (1.6pt);
\fill[bwd] (6.3,0.933) circle (1.6pt);
\node[fwd] at (1.666,1.092) {\scriptsize mean of $\relu(\sin x)$};
\node[bwd] at (4.923,2.261) {\scriptsize random input, band $4$};
\node[ink2] at (2.926,2.751) {\scriptsize slope $-2$};
\end{scope}
\end{tikzpicture}
\caption{Aliasing error of a rectifier against the number of grid points, on logarithmic scales: the closed-form case of \cref{thm:alias}(d) and the largest in-band coefficient error for a random input of band $4$. Both follow the slope $-2$.}\label{fig:alias}
\end{figure}
\noindent For a random input of band $5$, the multiplier $(1+k^2)^{-1}$ computed on $16$ and on $64$ points agrees
with the exact result to $10^{-14}$; the stencil is in relative error by $0.221$ and $0.0153$, its symbol at $k=5$
being $0.7173$ and $0.9801$ of the true one. \chk{A1} For band $4$ the in-band coefficients of the square are
exact on $13$ points and wrong by $1.4$ on $12$: the ``two-thirds rule'' of spectral methods \cite{Ors71}.
\chk{A2} For the rectifier one case is closed-form: the mean of $\relu(\sin x)$ sampled on $N$ points is
$\frac1N\cot\frac\pi N$ instead of $\frac1\pi$, an error of $-\pi/3N^2$ to leading order; $N^2$ times the error is
$-1.058$, $-1.050$, $-1.048$, $-1.047$ for $N=8,\dots,64$. \chk{A3} For a random input of band $4$ the largest
in-band error of the rectified signal is $7.4\times10^{-2}$, $9.8\times10^{-3}$, $2.1\times10^{-3}$,
$4.2\times10^{-4}$, $1.4\times10^{-4}$ on $16,\dots,256$ points (\cref{fig:alias}). \chk{A4} This is the quantitative content of the
practice of applying nonlinearities on an oversampled grid and filtering \cite{KALHHLA21}: oversampling by a
factor $M$ buys $M^{2}$, and no finite factor buys exactness. An architecture built from (a) alone is
resolution-independent \cite{LKALBSA21}; one with stencils is so to the order of the stencil; one with
rectifiers is so only approximately, at every resolution.
\section{Diffusion models}\label{sec:diffusion}
Let data $X_0\sim p_0$ on the line be noised by $X_t=X_0+W_t$, so that the density solves the heat equation
$\pd_tp=\frac12\pd_x^2p$, and write $s=\pd_x\log p_t$ for the \term{score}. Our example is a mixture of two
Gaussians, for which everything is closed-form. \chk{S1}
\begin{proposition}\label{prop:tweedie}
\Scl[\cite{Efr11,Hyv05,Vin11}] \emph{(a)} $\E[X_0\mid X_t=x]=x+t\,s(x,t)$. \emph{(b)} For every candidate $\sigma$,
\begin{multline*}
\E\bigl|\sigma(X_t)+\tfrac{X_t-X_0}t\bigr|^2\\=\E|\sigma(X_t)-s(X_t,t)|^2+\tfrac1t-J(p_t),
\end{multline*}
where $J$ is the Fisher information: denoising and explicit score matching differ by a constant.
\end{proposition}
\noindent Both hold to rounding in the example; for two-point data at $t=\frac12$ the constant is $0.924073$ for
each of two test functions. \chk{S2, S8} Since $\frac12\pd_x^2p=-\pd_x\bigl(p\cdot(-\frac12s)\bigr)$, the heat equation
is also a continuity equation, and its characteristics are the \term{probability flow}
$\dot x=-\frac12s(x,t)$ \cite{SSKKEP21}.
\begin{theorem}[Where the ill-posedness goes]\label{thm:illposed}
\Srf[\cite{PDEHC}, Ex.~2.5; \cite{And82,SSKKEP21}]
\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]
\item The backward heat equation multiplies the $k$-th Fourier mode of $p_T$ by $e^{k^2T/2}$: recovering $p_0$
from $p_T$ is ill-posed in every Sobolev norm.
\item Given the score on $[t_0,T]$ with $t_0>0$, the probability flow is an ordinary differential equation with
a Lipschitz field; it transports $p_T$ to $p_{t_0}$, and an error $\epsilon$ in the score, in the supremum norm,
moves each sample by at most $C(t_0)\epsilon$.
\item For data on the two points $\pm a$, $\pd_xs(0,t)=-1/t+a^2/t^2$. The Lipschitz constant of the field is not
integrable at $t=0$, and $C(t_0)\to\infty$ as $t_0\to0$.
\end{enumerate}
\end{theorem}
\begin{proof}
(a) is Fourier analysis. (b) is the uniqueness theorem for characteristics together with Gr\"onwall's
inequality. (c) Here $s=-x/t+(a/t)\tanh(ax/t)$.
\end{proof}
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.4)--(0,0)--(6,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $-2$};
\draw[ink2] (1.5,0)--(1.5,-0.07) node[below,inner sep=1.5pt]{\scriptsize $-1$};
\draw[ink2] (3,0)--(3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (4.5,0)--(4.5,-0.07) node[below,inner sep=1.5pt]{\scriptsize $1$};
\draw[ink2] (6,0)--(6,-0.07) node[below,inner sep=1.5pt]{\scriptsize $2$};
\draw[ink2] (0,0.283)--(-0.07,0.283) node[left,inner sep=1.5pt]{\scriptsize $-10$};
\draw[ink2] (0,1.7)--(-0.07,1.7) node[left,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (0,3.117)--(-0.07,3.117) node[left,inner sep=1.5pt]{\scriptsize $10$};
\node[below] at (3,-0.38) {\scriptsize $x$};
\draw[ink2,very thin] (0,1.7)--(6,1.7);
\draw[thick,ink2] plot coordinates {(0,1.847) (0.015,1.845) (0.03,1.844) (0.045,1.843) (0.06,1.842) (0.075,1.84) (0.09,1.839) (0.105,1.838) (0.12,1.836) (0.135,1.835) (0.15,1.834) (0.165,1.832) (0.18,1.831) (0.195,1.83) (0.21,1.829) (0.225,1.827) (0.24,1.826) (0.255,1.825) (0.27,1.823) (0.285,1.822) (0.3,1.821) (0.315,1.82) (0.33,1.818) (0.345,1.817) (0.36,1.816) (0.375,1.815) (0.39,1.813) (0.405,1.812) (0.42,1.811) (0.435,1.81) (0.45,1.808) (0.465,1.807) (0.48,1.806) (0.495,1.805) (0.51,1.803) (0.525,1.802) (0.54,1.801) (0.555,1.8) (0.57,1.799) (0.585,1.797) (0.6,1.796) (0.615,1.795) (0.63,1.794) (0.645,1.793) (0.66,1.791) (0.675,1.79) (0.69,1.789) (0.705,1.788) (0.72,1.787) (0.735,1.785) (0.75,1.784) (0.765,1.783) (0.78,1.782) (0.795,1.781) (0.81,1.78) (0.825,1.779) (0.84,1.777) (0.855,1.776) (0.87,1.775) (0.885,1.774) (0.9,1.773) (0.915,1.772) (0.93,1.771) (0.945,1.77) (0.96,1.769) (0.975,1.767) (0.99,1.766) (1.005,1.765) (1.02,1.764) (1.035,1.763) (1.05,1.762) (1.065,1.761) (1.08,1.76) (1.095,1.759) (1.11,1.758) (1.125,1.757) (1.14,1.756) (1.155,1.755) (1.17,1.754) (1.185,1.753) (1.2,1.752) (1.215,1.751) (1.23,1.75) (1.245,1.749) (1.26,1.748) (1.275,1.747) (1.29,1.746) (1.305,1.745) (1.32,1.744) (1.335,1.743) (1.35,1.742) (1.365,1.742) (1.38,1.741) (1.395,1.74) (1.41,1.739) (1.425,1.738) (1.44,1.737) (1.455,1.736) (1.47,1.735) (1.485,1.735) (1.5,1.734) (1.515,1.733) (1.53,1.732) (1.545,1.731) (1.56,1.731) (1.575,1.73) (1.59,1.729) (1.605,1.728) (1.62,1.727) (1.635,1.727) (1.65,1.726) (1.665,1.725) (1.68,1.725) (1.695,1.724) (1.71,1.723) (1.725,1.723) (1.74,1.722) (1.755,1.721) (1.77,1.721) (1.785,1.72) (1.8,1.719) (1.815,1.719) (1.83,1.718) (1.845,1.717) (1.86,1.717) (1.875,1.716) (1.89,1.716) (1.905,1.715) (1.92,1.715) (1.935,1.714) (1.95,1.714) (1.965,1.713) (1.98,1.713) (1.995,1.712) (2.01,1.712) (2.025,1.711) (2.04,1.711) (2.055,1.71) (2.07,1.71) (2.085,1.709) (2.1,1.709) (2.115,1.709) (2.13,1.708) (2.145,1.708) (2.16,1.707) (2.175,1.707) (2.19,1.707) (2.205,1.706) (2.22,1.706) (2.235,1.706) (2.25,1.705) (2.265,1.705) (2.28,1.705) (2.295,1.705) (2.31,1.704) (2.325,1.704) (2.34,1.704) (2.355,1.703) (2.37,1.703) (2.385,1.703) (2.4,1.703) (2.415,1.703) (2.43,1.702) (2.445,1.702) (2.46,1.702) (2.475,1.702) (2.49,1.702) (2.505,1.702) (2.52,1.701) (2.535,1.701) (2.55,1.701) (2.565,1.701) (2.58,1.701) (2.595,1.701) (2.61,1.701) (2.625,1.701) (2.64,1.701) (2.655,1.701) (2.67,1.7) (2.685,1.7) (2.7,1.7) (2.715,1.7) (2.73,1.7) (2.745,1.7) (2.76,1.7) (2.775,1.7) (2.79,1.7) (2.805,1.7) (2.82,1.7) (2.835,1.7) (2.85,1.7) (2.865,1.7) (2.88,1.7) (2.895,1.7) (2.91,1.7) (2.925,1.7) (2.94,1.7) (2.955,1.7) (2.97,1.7) (2.985,1.7) (3,1.7) (3.015,1.7) (3.03,1.7) (3.045,1.7) (3.06,1.7) (3.075,1.7) (3.09,1.7) (3.105,1.7) (3.12,1.7) (3.135,1.7) (3.15,1.7) (3.165,1.7) (3.18,1.7) (3.195,1.7) (3.21,1.7) (3.225,1.7) (3.24,1.7) (3.255,1.7) (3.27,1.7) (3.285,1.7) (3.3,1.7) (3.315,1.7) (3.33,1.7) (3.345,1.699) (3.36,1.699) (3.375,1.699) (3.39,1.699) (3.405,1.699) (3.42,1.699) (3.435,1.699) (3.45,1.699) (3.465,1.699) (3.48,1.699) (3.495,1.698) (3.51,1.698) (3.525,1.698) (3.54,1.698) (3.555,1.698) (3.57,1.698) (3.585,1.697) (3.6,1.697) (3.615,1.697) (3.63,1.697) (3.645,1.697) (3.66,1.696) (3.675,1.696) (3.69,1.696) (3.705,1.695) (3.72,1.695) (3.735,1.695) (3.75,1.695) (3.765,1.694) (3.78,1.694) (3.795,1.694) (3.81,1.693) (3.825,1.693) (3.84,1.693) (3.855,1.692) (3.87,1.692) (3.885,1.691) (3.9,1.691) (3.915,1.691) (3.93,1.69) (3.945,1.69) (3.96,1.689) (3.975,1.689) (3.99,1.688) (4.005,1.688) (4.02,1.687) (4.035,1.687) (4.05,1.686) (4.065,1.686) (4.08,1.685) (4.095,1.685) (4.11,1.684) (4.125,1.684) (4.14,1.683) (4.155,1.683) (4.17,1.682) (4.185,1.681) (4.2,1.681) (4.215,1.68) (4.23,1.679) (4.245,1.679) (4.26,1.678) (4.275,1.677) (4.29,1.677) (4.305,1.676) (4.32,1.675) (4.335,1.675) (4.35,1.674) (4.365,1.673) (4.38,1.673) (4.395,1.672) (4.41,1.671) (4.425,1.67) (4.44,1.669) (4.455,1.669) (4.47,1.668) (4.485,1.667) (4.5,1.666) (4.515,1.665) (4.53,1.665) (4.545,1.664) (4.56,1.663) (4.575,1.662) (4.59,1.661) (4.605,1.66) (4.62,1.659) (4.635,1.658) (4.65,1.658) (4.665,1.657) (4.68,1.656) (4.695,1.655) (4.71,1.654) (4.725,1.653) (4.74,1.652) (4.755,1.651) (4.77,1.65) (4.785,1.649) (4.8,1.648) (4.815,1.647) (4.83,1.646) (4.845,1.645) (4.86,1.644) (4.875,1.643) (4.89,1.642) (4.905,1.641) (4.92,1.64) (4.935,1.639) (4.95,1.638) (4.965,1.637) (4.98,1.636) (4.995,1.635) (5.01,1.634) (5.025,1.633) (5.04,1.631) (5.055,1.63) (5.07,1.629) (5.085,1.628) (5.1,1.627) (5.115,1.626) (5.13,1.625) (5.145,1.624) (5.16,1.623) (5.175,1.621) (5.19,1.62) (5.205,1.619) (5.22,1.618) (5.235,1.617) (5.25,1.616) (5.265,1.615) (5.28,1.613) (5.295,1.612) (5.31,1.611) (5.325,1.61) (5.34,1.609) (5.355,1.607) (5.37,1.606) (5.385,1.605) (5.4,1.604) (5.415,1.603) (5.43,1.601) (5.445,1.6) (5.46,1.599) (5.475,1.598) (5.49,1.597) (5.505,1.595) (5.52,1.594) (5.535,1.593) (5.55,1.592) (5.565,1.59) (5.58,1.589) (5.595,1.588) (5.61,1.587) (5.625,1.585) (5.64,1.584) (5.655,1.583) (5.67,1.582) (5.685,1.58) (5.7,1.579) (5.715,1.578) (5.73,1.577) (5.745,1.575) (5.76,1.574) (5.775,1.573) (5.79,1.571) (5.805,1.57) (5.82,1.569) (5.835,1.568) (5.85,1.566) (5.865,1.565) (5.88,1.564) (5.895,1.562) (5.91,1.561) (5.925,1.56) (5.94,1.558) (5.955,1.557) (5.97,1.556) (5.985,1.555) (6,1.553)};
\draw[thick,fwd] plot coordinates {(0,2.172) (0.015,2.168) (0.03,2.163) (0.045,2.158) (0.06,2.153) (0.075,2.149) (0.09,2.144) (0.105,2.139) (0.12,2.134) (0.135,2.13) (0.15,2.125) (0.165,2.12) (0.18,2.116) (0.195,2.111) (0.21,2.106) (0.225,2.101) (0.24,2.097) (0.255,2.092) (0.27,2.087) (0.285,2.083) (0.3,2.078) (0.315,2.073) (0.33,2.068) (0.345,2.064) (0.36,2.059) (0.375,2.054) (0.39,2.049) (0.405,2.045) (0.42,2.04) (0.435,2.035) (0.45,2.031) (0.465,2.026) (0.48,2.021) (0.495,2.016) (0.51,2.012) (0.525,2.007) (0.54,2.002) (0.555,1.998) (0.57,1.993) (0.585,1.988) (0.6,1.983) (0.615,1.979) (0.63,1.974) (0.645,1.969) (0.66,1.964) (0.675,1.96) (0.69,1.955) (0.705,1.95) (0.72,1.946) (0.735,1.941) (0.75,1.936) (0.765,1.931) (0.78,1.927) (0.795,1.922) (0.81,1.917) (0.825,1.913) (0.84,1.908) (0.855,1.903) (0.87,1.898) (0.885,1.894) (0.9,1.889) (0.915,1.884) (0.93,1.88) (0.945,1.875) (0.96,1.87) (0.975,1.865) (0.99,1.861) (1.005,1.856) (1.02,1.851) (1.035,1.847) (1.05,1.842) (1.065,1.837) (1.08,1.832) (1.095,1.828) (1.11,1.823) (1.125,1.818) (1.14,1.814) (1.155,1.809) (1.17,1.804) (1.185,1.799) (1.2,1.795) (1.215,1.79) (1.23,1.785) (1.245,1.781) (1.26,1.776) (1.275,1.771) (1.29,1.767) (1.305,1.762) (1.32,1.757) (1.335,1.753) (1.35,1.748) (1.365,1.743) (1.38,1.738) (1.395,1.734) (1.41,1.729) (1.425,1.724) (1.44,1.72) (1.455,1.715) (1.47,1.71) (1.485,1.706) (1.5,1.701) (1.515,1.697) (1.53,1.692) (1.545,1.687) (1.56,1.683) (1.575,1.678) (1.59,1.673) (1.605,1.669) (1.62,1.664) (1.635,1.66) (1.65,1.655) (1.665,1.651) (1.68,1.646) (1.695,1.641) (1.71,1.637) (1.725,1.632) (1.74,1.628) (1.755,1.623) (1.77,1.619) (1.785,1.615) (1.8,1.61) (1.815,1.606) (1.83,1.601) (1.845,1.597) (1.86,1.593) (1.875,1.588) (1.89,1.584) (1.905,1.58) (1.92,1.575) (1.935,1.571) (1.95,1.567) (1.965,1.563) (1.98,1.559) (1.995,1.555) (2.01,1.551) (2.025,1.547) (2.04,1.543) (2.055,1.539) (2.07,1.535) (2.085,1.532) (2.1,1.528) (2.115,1.525) (2.13,1.521) (2.145,1.518) (2.16,1.514) (2.175,1.511) (2.19,1.508) (2.205,1.505) (2.22,1.502) (2.235,1.499) (2.25,1.496) (2.265,1.494) (2.28,1.491) (2.295,1.489) (2.31,1.487) (2.325,1.485) (2.34,1.483) (2.355,1.482) (2.37,1.48) (2.385,1.479) (2.4,1.478) (2.415,1.477) (2.43,1.477) (2.445,1.476) (2.46,1.476) (2.475,1.477) (2.49,1.477) (2.505,1.478) (2.52,1.479) (2.535,1.48) (2.55,1.482) (2.565,1.484) (2.58,1.486) (2.595,1.489) (2.61,1.492) (2.625,1.496) (2.64,1.5) (2.655,1.504) (2.67,1.509) (2.685,1.514) (2.7,1.519) (2.715,1.525) (2.73,1.531) (2.745,1.538) (2.76,1.545) (2.775,1.553) (2.79,1.56) (2.805,1.569) (2.82,1.577) (2.835,1.586) (2.85,1.595) (2.865,1.605) (2.88,1.615) (2.895,1.625) (2.91,1.635) (2.925,1.646) (2.94,1.656) (2.955,1.667) (2.97,1.678) (2.985,1.689) (3,1.7) (3.015,1.711) (3.03,1.722) (3.045,1.733) (3.06,1.744) (3.075,1.754) (3.09,1.765) (3.105,1.775) (3.12,1.785) (3.135,1.795) (3.15,1.805) (3.165,1.814) (3.18,1.823) (3.195,1.831) (3.21,1.84) (3.225,1.847) (3.24,1.855) (3.255,1.862) (3.27,1.869) (3.285,1.875) (3.3,1.881) (3.315,1.886) (3.33,1.891) (3.345,1.896) (3.36,1.9) (3.375,1.904) (3.39,1.908) (3.405,1.911) (3.42,1.914) (3.435,1.916) (3.45,1.918) (3.465,1.92) (3.48,1.921) (3.495,1.922) (3.51,1.923) (3.525,1.923) (3.54,1.924) (3.555,1.924) (3.57,1.923) (3.585,1.923) (3.6,1.922) (3.615,1.921) (3.63,1.92) (3.645,1.918) (3.66,1.917) (3.675,1.915) (3.69,1.913) (3.705,1.911) (3.72,1.909) (3.735,1.906) (3.75,1.904) (3.765,1.901) (3.78,1.898) (3.795,1.895) (3.81,1.892) (3.825,1.889) (3.84,1.886) (3.855,1.882) (3.87,1.879) (3.885,1.875) (3.9,1.872) (3.915,1.868) (3.93,1.865) (3.945,1.861) (3.96,1.857) (3.975,1.853) (3.99,1.849) (4.005,1.845) (4.02,1.841) (4.035,1.837) (4.05,1.833) (4.065,1.829) (4.08,1.825) (4.095,1.82) (4.11,1.816) (4.125,1.812) (4.14,1.807) (4.155,1.803) (4.17,1.799) (4.185,1.794) (4.2,1.79) (4.215,1.785) (4.23,1.781) (4.245,1.777) (4.26,1.772) (4.275,1.768) (4.29,1.763) (4.305,1.759) (4.32,1.754) (4.335,1.749) (4.35,1.745) (4.365,1.74) (4.38,1.736) (4.395,1.731) (4.41,1.727) (4.425,1.722) (4.44,1.717) (4.455,1.713) (4.47,1.708) (4.485,1.703) (4.5,1.699) (4.515,1.694) (4.53,1.69) (4.545,1.685) (4.56,1.68) (4.575,1.676) (4.59,1.671) (4.605,1.666) (4.62,1.662) (4.635,1.657) (4.65,1.652) (4.665,1.647) (4.68,1.643) (4.695,1.638) (4.71,1.633) (4.725,1.629) (4.74,1.624) (4.755,1.619) (4.77,1.615) (4.785,1.61) (4.8,1.605) (4.815,1.601) (4.83,1.596) (4.845,1.591) (4.86,1.586) (4.875,1.582) (4.89,1.577) (4.905,1.572) (4.92,1.568) (4.935,1.563) (4.95,1.558) (4.965,1.553) (4.98,1.549) (4.995,1.544) (5.01,1.539) (5.025,1.535) (5.04,1.53) (5.055,1.525) (5.07,1.52) (5.085,1.516) (5.1,1.511) (5.115,1.506) (5.13,1.502) (5.145,1.497) (5.16,1.492) (5.175,1.487) (5.19,1.483) (5.205,1.478) (5.22,1.473) (5.235,1.469) (5.25,1.464) (5.265,1.459) (5.28,1.454) (5.295,1.45) (5.31,1.445) (5.325,1.44) (5.34,1.436) (5.355,1.431) (5.37,1.426) (5.385,1.421) (5.4,1.417) (5.415,1.412) (5.43,1.407) (5.445,1.402) (5.46,1.398) (5.475,1.393) (5.49,1.388) (5.505,1.384) (5.52,1.379) (5.535,1.374) (5.55,1.369) (5.565,1.365) (5.58,1.36) (5.595,1.355) (5.61,1.351) (5.625,1.346) (5.64,1.341) (5.655,1.336) (5.67,1.332) (5.685,1.327) (5.7,1.322) (5.715,1.317) (5.73,1.313) (5.745,1.308) (5.76,1.303) (5.775,1.299) (5.79,1.294) (5.805,1.289) (5.82,1.284) (5.835,1.28) (5.85,1.275) (5.865,1.27) (5.88,1.266) (5.895,1.261) (5.91,1.256) (5.925,1.251) (5.94,1.247) (5.955,1.242) (5.97,1.237) (5.985,1.232) (6,1.228)};
\draw[thick,bwd] plot coordinates {(0,3.117) (0.015,3.102) (0.03,3.088) (0.045,3.074) (0.06,3.06) (0.075,3.046) (0.09,3.032) (0.105,3.017) (0.12,3.003) (0.135,2.989) (0.15,2.975) (0.165,2.961) (0.18,2.947) (0.195,2.933) (0.21,2.918) (0.225,2.904) (0.24,2.89) (0.255,2.876) (0.27,2.862) (0.285,2.848) (0.3,2.833) (0.315,2.819) (0.33,2.805) (0.345,2.791) (0.36,2.777) (0.375,2.763) (0.39,2.748) (0.405,2.734) (0.42,2.72) (0.435,2.706) (0.45,2.692) (0.465,2.678) (0.48,2.663) (0.495,2.649) (0.51,2.635) (0.525,2.621) (0.54,2.607) (0.555,2.593) (0.57,2.578) (0.585,2.564) (0.6,2.55) (0.615,2.536) (0.63,2.522) (0.645,2.508) (0.66,2.493) (0.675,2.479) (0.69,2.465) (0.705,2.451) (0.72,2.437) (0.735,2.423) (0.75,2.408) (0.765,2.394) (0.78,2.38) (0.795,2.366) (0.81,2.352) (0.825,2.338) (0.84,2.323) (0.855,2.309) (0.87,2.295) (0.885,2.281) (0.9,2.267) (0.915,2.253) (0.93,2.238) (0.945,2.224) (0.96,2.21) (0.975,2.196) (0.99,2.182) (1.005,2.168) (1.02,2.153) (1.035,2.139) (1.05,2.125) (1.065,2.111) (1.08,2.097) (1.095,2.083) (1.11,2.068) (1.125,2.054) (1.14,2.04) (1.155,2.026) (1.17,2.012) (1.185,1.998) (1.2,1.983) (1.215,1.969) (1.23,1.955) (1.245,1.941) (1.26,1.927) (1.275,1.913) (1.29,1.898) (1.305,1.884) (1.32,1.87) (1.335,1.856) (1.35,1.842) (1.365,1.828) (1.38,1.813) (1.395,1.799) (1.41,1.785) (1.425,1.771) (1.44,1.757) (1.455,1.743) (1.47,1.728) (1.485,1.714) (1.5,1.7) (1.515,1.686) (1.53,1.672) (1.545,1.658) (1.56,1.643) (1.575,1.629) (1.59,1.615) (1.605,1.601) (1.62,1.587) (1.635,1.573) (1.65,1.558) (1.665,1.544) (1.68,1.53) (1.695,1.516) (1.71,1.502) (1.725,1.488) (1.74,1.473) (1.755,1.459) (1.77,1.445) (1.785,1.431) (1.8,1.417) (1.815,1.403) (1.83,1.388) (1.845,1.374) (1.86,1.36) (1.875,1.346) (1.89,1.332) (1.905,1.318) (1.92,1.303) (1.935,1.289) (1.95,1.275) (1.965,1.261) (1.98,1.247) (1.995,1.233) (2.01,1.218) (2.025,1.204) (2.04,1.19) (2.055,1.176) (2.07,1.162) (2.085,1.148) (2.1,1.133) (2.115,1.119) (2.13,1.105) (2.145,1.091) (2.16,1.077) (2.175,1.063) (2.19,1.048) (2.205,1.034) (2.22,1.02) (2.235,1.006) (2.25,0.992) (2.265,0.978) (2.28,0.964) (2.295,0.949) (2.31,0.935) (2.325,0.921) (2.34,0.907) (2.355,0.893) (2.37,0.879) (2.385,0.865) (2.4,0.851) (2.415,0.837) (2.43,0.823) (2.445,0.809) (2.46,0.795) (2.475,0.782) (2.49,0.768) (2.505,0.755) (2.52,0.741) (2.535,0.728) (2.55,0.715) (2.565,0.703) (2.58,0.69) (2.595,0.679) (2.61,0.667) (2.625,0.656) (2.64,0.646) (2.655,0.637) (2.67,0.629) (2.685,0.623) (2.7,0.618) (2.715,0.614) (2.73,0.614) (2.745,0.616) (2.76,0.621) (2.775,0.63) (2.79,0.644) (2.805,0.663) (2.82,0.689) (2.835,0.722) (2.85,0.763) (2.865,0.813) (2.88,0.873) (2.895,0.943) (2.91,1.024) (2.925,1.116) (2.94,1.218) (2.955,1.33) (2.97,1.449) (2.985,1.573) (3,1.7) (3.015,1.827) (3.03,1.951) (3.045,2.07) (3.06,2.182) (3.075,2.284) (3.09,2.376) (3.105,2.457) (3.12,2.527) (3.135,2.587) (3.15,2.637) (3.165,2.678) (3.18,2.711) (3.195,2.737) (3.21,2.756) (3.225,2.77) (3.24,2.779) (3.255,2.784) (3.27,2.786) (3.285,2.786) (3.3,2.782) (3.315,2.777) (3.33,2.771) (3.345,2.763) (3.36,2.754) (3.375,2.744) (3.39,2.733) (3.405,2.721) (3.42,2.71) (3.435,2.697) (3.45,2.685) (3.465,2.672) (3.48,2.659) (3.495,2.645) (3.51,2.632) (3.525,2.618) (3.54,2.605) (3.555,2.591) (3.57,2.577) (3.585,2.563) (3.6,2.549) (3.615,2.535) (3.63,2.521) (3.645,2.507) (3.66,2.493) (3.675,2.479) (3.69,2.465) (3.705,2.451) (3.72,2.436) (3.735,2.422) (3.75,2.408) (3.765,2.394) (3.78,2.38) (3.795,2.366) (3.81,2.352) (3.825,2.337) (3.84,2.323) (3.855,2.309) (3.87,2.295) (3.885,2.281) (3.9,2.267) (3.915,2.252) (3.93,2.238) (3.945,2.224) (3.96,2.21) (3.975,2.196) (3.99,2.182) (4.005,2.167) (4.02,2.153) (4.035,2.139) (4.05,2.125) (4.065,2.111) (4.08,2.097) (4.095,2.082) (4.11,2.068) (4.125,2.054) (4.14,2.04) (4.155,2.026) (4.17,2.012) (4.185,1.997) (4.2,1.983) (4.215,1.969) (4.23,1.955) (4.245,1.941) (4.26,1.927) (4.275,1.912) (4.29,1.898) (4.305,1.884) (4.32,1.87) (4.335,1.856) (4.35,1.842) (4.365,1.827) (4.38,1.813) (4.395,1.799) (4.41,1.785) (4.425,1.771) (4.44,1.757) (4.455,1.742) (4.47,1.728) (4.485,1.714) (4.5,1.7) (4.515,1.686) (4.53,1.672) (4.545,1.657) (4.56,1.643) (4.575,1.629) (4.59,1.615) (4.605,1.601) (4.62,1.587) (4.635,1.572) (4.65,1.558) (4.665,1.544) (4.68,1.53) (4.695,1.516) (4.71,1.502) (4.725,1.487) (4.74,1.473) (4.755,1.459) (4.77,1.445) (4.785,1.431) (4.8,1.417) (4.815,1.402) (4.83,1.388) (4.845,1.374) (4.86,1.36) (4.875,1.346) (4.89,1.332) (4.905,1.317) (4.92,1.303) (4.935,1.289) (4.95,1.275) (4.965,1.261) (4.98,1.247) (4.995,1.232) (5.01,1.218) (5.025,1.204) (5.04,1.19) (5.055,1.176) (5.07,1.162) (5.085,1.147) (5.1,1.133) (5.115,1.119) (5.13,1.105) (5.145,1.091) (5.16,1.077) (5.175,1.062) (5.19,1.048) (5.205,1.034) (5.22,1.02) (5.235,1.006) (5.25,0.992) (5.265,0.977) (5.28,0.963) (5.295,0.949) (5.31,0.935) (5.325,0.921) (5.34,0.907) (5.355,0.892) (5.37,0.878) (5.385,0.864) (5.4,0.85) (5.415,0.836) (5.43,0.822) (5.445,0.807) (5.46,0.793) (5.475,0.779) (5.49,0.765) (5.505,0.751) (5.52,0.737) (5.535,0.722) (5.55,0.708) (5.565,0.694) (5.58,0.68) (5.595,0.666) (5.61,0.652) (5.625,0.637) (5.64,0.623) (5.655,0.609) (5.67,0.595) (5.685,0.581) (5.7,0.567) (5.715,0.552) (5.73,0.538) (5.745,0.524) (5.76,0.51) (5.775,0.496) (5.79,0.482) (5.805,0.467) (5.82,0.453) (5.835,0.439) (5.85,0.425) (5.865,0.411) (5.88,0.397) (5.895,0.382) (5.91,0.368) (5.925,0.354) (5.94,0.34) (5.955,0.326) (5.97,0.312) (5.985,0.297) (6,0.283)};
\draw[accent,very thin,densely dashed] (1.5,0)--(1.5,3.4);
\draw[accent,very thin,densely dashed] (4.5,0)--(4.5,3.4);
\node[bwd,right] at (6.075,0.283) {\scriptsize $t=0.1$};
\node[fwd,right] at (6.075,1.133) {\scriptsize $0.3$};
\node[ink2,right] at (6.075,1.615) {\scriptsize $1$};
\end{scope}
\end{tikzpicture}
\caption{The score $s(x,t)=-x/t+\tanh(x/t)/t$ of data on the two points $\pm1$ (dashed), at three noise levels. As $t\to0$ it becomes a restoring force towards each point, joined by a transition of slope $1/t^2$ at the midpoint.}\label{fig:score}
\end{figure}
\noindent For $T=1$ and $k=10$ the factor in (a) is $5.2\times10^{21}$. In the example the flow from $T=1$ to $0$
preserves the value of the distribution function along five trajectories to $10^{-14}$, and a perturbation
$\epsilon\sin3x$ of the score displaces the samples by $0.264\,\epsilon$. \chk{S3, S4} The slope in (c) is $90$ at
$t=0.1$ and $9900$ at $t=0.01$ (\cref{fig:score}). \chk{S5} The three statements together say that a diffusion model does not solve
the backward heat equation. It replaces an ill-posed linear problem for the density by a well-posed nonlinear
transport problem for samples, whose coefficient, the score, carries the whole difficulty, concentrated at small
times and near the boundaries between modes. Statement (c) is the sample-level form of Hadamard's example.
\begin{proposition}[Entropy]\label{prop:entropy}
\Scl[\cite{Sta59,BaEm85}] \emph{(a)} Under the heat flow $\frac d{dt}h(p_t)=\frac12J(p_t)$. \emph{(b)} Under the
Ornstein--Uhlenbeck flow $dX=-X\,dt+\sqrt2\,dW$, with Gaussian limit $\gamma$,
$\KL{p_t}{\gamma}\le e^{-2t}\KL{p_0}{\gamma}$, and $e^{2t}\KL{p_t}{\gamma}\to\frac12(\E X_0)^2$.
\end{proposition}
\noindent In the example $dh/dt=0.535607$ and $J/2=0.535606$ at $t=0.3$. \chk{S6} The ratio
$\KL{p_t}{\gamma}/e^{-2t}\KL{p_0}{\gamma}$ is $0.353$, $0.204$, $0.119$, $0.088$ at $t=\frac14,\frac12,1,2$, and
$e^{2t}\KL{p_t}{\gamma}=0.03128$ at $t=4$ against the limit $0.03125$. \chk{S7} The rate $2$ is twice the spectral
gap of the generator, and it fixes the noising time needed to make the terminal law Gaussian to a given
accuracy: $\frac12\log(\KL{p_0}{\gamma}/\delta)$.
\section{Flows and characteristics}\label{sec:flows}
A continuous normalizing flow \cite{CRBD18} moves samples by $\dot x=v(x,t)$; the density then solves the
continuity equation $\pd_tp+\pd_x(pv)=0$, of which the trajectories are the characteristics
\cite[Prop.~1.8]{PDEHC}.
\begin{proposition}\label{prop:cnf}
\Scl[\cite{CRBD18}] Along a trajectory $\frac d{dt}\log p(x(t),t)=-\pd_xv(x(t),t)$.
\end{proposition}
\noindent For $v=0.8\tanh(1.3x+0.2)$ the exponential of the integrated divergence equals the derivative of the
flow map to $10^{-8}$. \chk{F1} \term{Flow matching} \cite{LCBNL23,LGL23} chooses a coupling of $X_0\sim p_0$ and
$X_1\sim p_1$, sets $X_t=(1-t)X_0+tX_1$, and regresses onto $v(x,t)=\E[X_1-X_0\mid X_t=x]$.
\begin{theorem}[One dimension]\label{thm:fm}
\Sob On the line, if $v$ is regular enough for trajectories to be unique, the flow of the flow-matching field
is the increasing rearrangement of $p_0$ onto $p_1$, whatever the coupling.
\end{theorem}
\begin{proof}
The marginal field transports $p_0$ to $p_1$ by construction. Trajectories of a scalar equation cannot cross,
so the flow map is increasing, and an increasing map carrying one atomless law to another is unique.
\end{proof}
\noindent For $p_0=N(0,1)$, $p_1=N(2,\frac14)$ and the independent coupling the field is linear in $x$ with
coefficients rational in $t$, and its flow is $x\mapsto2+\frac12x$ to $10^{-13}$. \chk{F2} The conditional paths
are straight lines that cross; the marginal characteristics do not. In higher dimensions non-crossing no longer
determines the map, and the flow depends on the coupling.
\begin{proposition}[A pitchfork in time]\label{prop:pitchfork}
\Sob For $p_0=N(0,1)$ and $p_1$ uniform on $\pm1$, with the independent coupling,
\[ v(x,t)=\frac1{1-t}\Bigl[\tanh\frac{tx}{(1-t)^2}-x\Bigr]. \]
The midpoint $x=0$ is an attracting equilibrium for $t<t^*=\frac{3-\sqrt5}2=0.382$ and a repelling one
afterwards, and $\pd_xv(0,t)\sim(1-t)^{-3}$ as $t\to1$.
\end{proposition}
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.8)--(0,0)--(6.3,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (3.15,0)--(3.15,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0.5$};
\draw[ink2] (6.3,0)--(6.3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $1$};
\draw[ink2] (0,0.173)--(-0.07,0.173) node[left,inner sep=1.5pt]{\scriptsize $-2$};
\draw[ink2] (0,1.036)--(-0.07,1.036) node[left,inner sep=1.5pt]{\scriptsize $-1$};
\draw[ink2] (0,1.9)--(-0.07,1.9) node[left,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (0,2.764)--(-0.07,2.764) node[left,inner sep=1.5pt]{\scriptsize $1$};
\draw[ink2] (0,3.627)--(-0.07,3.627) node[left,inner sep=1.5pt]{\scriptsize $2$};
\node[below] at (3.15,-0.38) {\scriptsize $t$};
\node[above,inner sep=1pt] at (0,3.85) {\scriptsize $x$};
\draw[accent,densely dashed] (2.406,0)--(2.406,3.8);
\draw[fwd] plot coordinates {(0,0.173) (0.076,0.193) (0.151,0.214) (0.227,0.234) (0.302,0.254) (0.378,0.273) (0.454,0.292) (0.529,0.311) (0.605,0.33) (0.68,0.348) (0.756,0.366) (0.832,0.384) (0.907,0.401) (0.983,0.418) (1.058,0.434) (1.134,0.45) (1.21,0.466) (1.285,0.481) (1.361,0.496) (1.436,0.511) (1.512,0.525) (1.588,0.538) (1.663,0.551) (1.739,0.564) (1.814,0.576) (1.89,0.588) (1.966,0.6) (2.041,0.611) (2.117,0.621) (2.192,0.632) (2.268,0.642) (2.344,0.651) (2.419,0.66) (2.495,0.669) (2.57,0.678) (2.646,0.686) (2.722,0.694) (2.797,0.702) (2.873,0.71) (2.948,0.718) (3.024,0.725) (3.1,0.733) (3.175,0.74) (3.251,0.748) (3.326,0.755) (3.402,0.762) (3.478,0.769) (3.553,0.776) (3.629,0.784) (3.704,0.791) (3.78,0.798) (3.856,0.805) (3.931,0.812) (4.007,0.819) (4.082,0.827) (4.158,0.834) (4.234,0.841) (4.309,0.848) (4.385,0.855) (4.46,0.862) (4.536,0.869) (4.612,0.877) (4.687,0.884) (4.763,0.891) (4.838,0.898) (4.914,0.905) (4.99,0.912) (5.065,0.92) (5.141,0.927) (5.216,0.934) (5.292,0.941) (5.368,0.948) (5.443,0.955) (5.519,0.962) (5.594,0.97) (5.67,0.977) (5.746,0.984) (5.821,0.991) (5.897,0.998) (5.972,1.005) (6.048,1.013) (6.124,1.02)};
\draw[fwd] plot coordinates {(0,0.389) (0.076,0.407) (0.151,0.424) (0.227,0.442) (0.302,0.459) (0.378,0.476) (0.454,0.493) (0.529,0.51) (0.605,0.526) (0.68,0.542) (0.756,0.558) (0.832,0.573) (0.907,0.588) (0.983,0.603) (1.058,0.617) (1.134,0.631) (1.21,0.645) (1.285,0.658) (1.361,0.671) (1.436,0.684) (1.512,0.696) (1.588,0.707) (1.663,0.719) (1.739,0.729) (1.814,0.74) (1.89,0.75) (1.966,0.759) (2.041,0.768) (2.117,0.777) (2.192,0.785) (2.268,0.793) (2.344,0.801) (2.419,0.808) (2.495,0.814) (2.57,0.821) (2.646,0.827) (2.722,0.833) (2.797,0.838) (2.873,0.843) (2.948,0.848) (3.024,0.853) (3.1,0.858) (3.175,0.862) (3.251,0.867) (3.326,0.871) (3.402,0.876) (3.478,0.88) (3.553,0.884) (3.629,0.888) (3.704,0.892) (3.78,0.897) (3.856,0.901) (3.931,0.905) (4.007,0.909) (4.082,0.913) (4.158,0.918) (4.234,0.922) (4.309,0.926) (4.385,0.93) (4.46,0.934) (4.536,0.939) (4.612,0.943) (4.687,0.947) (4.763,0.951) (4.838,0.955) (4.914,0.96) (4.99,0.964) (5.065,0.968) (5.141,0.972) (5.216,0.976) (5.292,0.981) (5.368,0.985) (5.443,0.989) (5.519,0.993) (5.594,0.997) (5.67,1.001) (5.746,1.006) (5.821,1.01) (5.897,1.014) (5.972,1.018) (6.048,1.022) (6.124,1.027)};
\draw[fwd] plot coordinates {(0,0.605) (0.076,0.62) (0.151,0.635) (0.227,0.65) (0.302,0.665) (0.378,0.68) (0.454,0.694) (0.529,0.708) (0.605,0.722) (0.68,0.736) (0.756,0.749) (0.832,0.763) (0.907,0.775) (0.983,0.788) (1.058,0.8) (1.134,0.812) (1.21,0.824) (1.285,0.835) (1.361,0.846) (1.436,0.857) (1.512,0.867) (1.588,0.877) (1.663,0.886) (1.739,0.895) (1.814,0.904) (1.89,0.912) (1.966,0.92) (2.041,0.927) (2.117,0.934) (2.192,0.941) (2.268,0.947) (2.344,0.952) (2.419,0.957) (2.495,0.962) (2.57,0.966) (2.646,0.97) (2.722,0.973) (2.797,0.977) (2.873,0.979) (2.948,0.982) (3.024,0.984) (3.1,0.986) (3.175,0.988) (3.251,0.989) (3.326,0.991) (3.402,0.992) (3.478,0.994) (3.553,0.995) (3.629,0.996) (3.704,0.997) (3.78,0.999) (3.856,1) (3.931,1.001) (4.007,1.002) (4.082,1.003) (4.158,1.004) (4.234,1.005) (4.309,1.006) (4.385,1.008) (4.46,1.009) (4.536,1.01) (4.612,1.011) (4.687,1.012) (4.763,1.013) (4.838,1.014) (4.914,1.016) (4.99,1.017) (5.065,1.018) (5.141,1.019) (5.216,1.02) (5.292,1.021) (5.368,1.022) (5.443,1.024) (5.519,1.025) (5.594,1.026) (5.67,1.027) (5.746,1.028) (5.821,1.029) (5.897,1.03) (5.972,1.031) (6.048,1.033) (6.124,1.034)};
\draw[fwd] plot coordinates {(0,0.82) (0.076,0.833) (0.151,0.846) (0.227,0.859) (0.302,0.871) (0.378,0.883) (0.454,0.895) (0.529,0.907) (0.605,0.919) (0.68,0.93) (0.756,0.941) (0.832,0.952) (0.907,0.963) (0.983,0.973) (1.058,0.984) (1.134,0.993) (1.21,1.003) (1.285,1.012) (1.361,1.022) (1.436,1.03) (1.512,1.039) (1.588,1.047) (1.663,1.055) (1.739,1.062) (1.814,1.069) (1.89,1.075) (1.966,1.082) (2.041,1.087) (2.117,1.092) (2.192,1.097) (2.268,1.102) (2.344,1.106) (2.419,1.109) (2.495,1.112) (2.57,1.114) (2.646,1.116) (2.722,1.118) (2.797,1.119) (2.873,1.119) (2.948,1.119) (3.024,1.119) (3.1,1.119) (3.175,1.118) (3.251,1.117) (3.326,1.115) (3.402,1.114) (3.478,1.112) (3.553,1.11) (3.629,1.109) (3.704,1.107) (3.78,1.105) (3.856,1.103) (3.931,1.101) (4.007,1.099) (4.082,1.097) (4.158,1.095) (4.234,1.092) (4.309,1.09) (4.385,1.088) (4.46,1.086) (4.536,1.084) (4.612,1.082) (4.687,1.08) (4.763,1.078) (4.838,1.076) (4.914,1.074) (4.99,1.072) (5.065,1.07) (5.141,1.068) (5.216,1.066) (5.292,1.064) (5.368,1.062) (5.443,1.06) (5.519,1.058) (5.594,1.056) (5.67,1.053) (5.746,1.051) (5.821,1.049) (5.897,1.047) (5.972,1.045) (6.048,1.043) (6.124,1.041)};
\draw[fwd] plot coordinates {(0,1.036) (0.076,1.047) (0.151,1.057) (0.227,1.067) (0.302,1.077) (0.378,1.087) (0.454,1.096) (0.529,1.106) (0.605,1.115) (0.68,1.124) (0.756,1.133) (0.832,1.142) (0.907,1.15) (0.983,1.159) (1.058,1.167) (1.134,1.175) (1.21,1.182) (1.285,1.19) (1.361,1.197) (1.436,1.204) (1.512,1.211) (1.588,1.217) (1.663,1.223) (1.739,1.229) (1.814,1.234) (1.89,1.239) (1.966,1.244) (2.041,1.248) (2.117,1.252) (2.192,1.256) (2.268,1.258) (2.344,1.261) (2.419,1.263) (2.495,1.265) (2.57,1.265) (2.646,1.266) (2.722,1.266) (2.797,1.265) (2.873,1.264) (2.948,1.262) (3.024,1.26) (3.1,1.257) (3.175,1.254) (3.251,1.251) (3.326,1.247) (3.402,1.242) (3.478,1.238) (3.553,1.233) (3.629,1.228) (3.704,1.223) (3.78,1.217) (3.856,1.212) (3.931,1.207) (4.007,1.201) (4.082,1.196) (4.158,1.19) (4.234,1.185) (4.309,1.18) (4.385,1.174) (4.46,1.169) (4.536,1.163) (4.612,1.158) (4.687,1.152) (4.763,1.147) (4.838,1.142) (4.914,1.136) (4.99,1.131) (5.065,1.125) (5.141,1.12) (5.216,1.114) (5.292,1.109) (5.368,1.103) (5.443,1.098) (5.519,1.093) (5.594,1.087) (5.67,1.082) (5.746,1.076) (5.821,1.071) (5.897,1.065) (5.972,1.06) (6.048,1.054) (6.124,1.049)};
\draw[fwd] plot coordinates {(0,1.252) (0.076,1.26) (0.151,1.268) (0.227,1.275) (0.302,1.283) (0.378,1.29) (0.454,1.297) (0.529,1.304) (0.605,1.311) (0.68,1.318) (0.756,1.325) (0.832,1.331) (0.907,1.338) (0.983,1.344) (1.058,1.35) (1.134,1.356) (1.21,1.362) (1.285,1.367) (1.361,1.373) (1.436,1.378) (1.512,1.383) (1.588,1.388) (1.663,1.392) (1.739,1.396) (1.814,1.4) (1.89,1.404) (1.966,1.407) (2.041,1.41) (2.117,1.413) (2.192,1.415) (2.268,1.417) (2.344,1.419) (2.419,1.42) (2.495,1.42) (2.57,1.42) (2.646,1.42) (2.722,1.418) (2.797,1.417) (2.873,1.414) (2.948,1.412) (3.024,1.408) (3.1,1.404) (3.175,1.399) (3.251,1.393) (3.326,1.387) (3.402,1.38) (3.478,1.373) (3.553,1.366) (3.629,1.357) (3.704,1.349) (3.78,1.34) (3.856,1.332) (3.931,1.323) (4.007,1.314) (4.082,1.305) (4.158,1.295) (4.234,1.286) (4.309,1.277) (4.385,1.268) (4.46,1.259) (4.536,1.25) (4.612,1.241) (4.687,1.231) (4.763,1.222) (4.838,1.213) (4.914,1.204) (4.99,1.195) (5.065,1.186) (5.141,1.177) (5.216,1.167) (5.292,1.158) (5.368,1.149) (5.443,1.14) (5.519,1.131) (5.594,1.122) (5.67,1.113) (5.746,1.103) (5.821,1.094) (5.897,1.085) (5.972,1.076) (6.048,1.067) (6.124,1.058)};
\draw[fwd] plot coordinates {(0,1.468) (0.076,1.473) (0.151,1.478) (0.227,1.483) (0.302,1.488) (0.378,1.493) (0.454,1.498) (0.529,1.503) (0.605,1.507) (0.68,1.512) (0.756,1.516) (0.832,1.521) (0.907,1.525) (0.983,1.529) (1.058,1.533) (1.134,1.537) (1.21,1.541) (1.285,1.545) (1.361,1.548) (1.436,1.552) (1.512,1.555) (1.588,1.558) (1.663,1.561) (1.739,1.564) (1.814,1.567) (1.89,1.569) (1.966,1.571) (2.041,1.573) (2.117,1.575) (2.192,1.576) (2.268,1.577) (2.344,1.578) (2.419,1.578) (2.495,1.578) (2.57,1.578) (2.646,1.577) (2.722,1.576) (2.797,1.574) (2.873,1.571) (2.948,1.568) (3.024,1.564) (3.1,1.56) (3.175,1.554) (3.251,1.548) (3.326,1.541) (3.402,1.533) (3.478,1.525) (3.553,1.515) (3.629,1.505) (3.704,1.493) (3.78,1.482) (3.856,1.469) (3.931,1.457) (4.007,1.444) (4.082,1.431) (4.158,1.417) (4.234,1.404) (4.309,1.391) (4.385,1.377) (4.46,1.364) (4.536,1.35) (4.612,1.337) (4.687,1.323) (4.763,1.31) (4.838,1.296) (4.914,1.283) (4.99,1.269) (5.065,1.256) (5.141,1.243) (5.216,1.229) (5.292,1.216) (5.368,1.202) (5.443,1.189) (5.519,1.175) (5.594,1.162) (5.67,1.148) (5.746,1.135) (5.821,1.122) (5.897,1.108) (5.972,1.095) (6.048,1.081) (6.124,1.068)};
\draw[fwd] plot coordinates {(0,1.684) (0.076,1.687) (0.151,1.689) (0.227,1.692) (0.302,1.694) (0.378,1.697) (0.454,1.699) (0.529,1.701) (0.605,1.704) (0.68,1.706) (0.756,1.708) (0.832,1.71) (0.907,1.713) (0.983,1.715) (1.058,1.717) (1.134,1.719) (1.21,1.721) (1.285,1.722) (1.361,1.724) (1.436,1.726) (1.512,1.728) (1.588,1.729) (1.663,1.731) (1.739,1.732) (1.814,1.733) (1.89,1.734) (1.966,1.735) (2.041,1.736) (2.117,1.737) (2.192,1.738) (2.268,1.738) (2.344,1.739) (2.419,1.739) (2.495,1.739) (2.57,1.738) (2.646,1.738) (2.722,1.737) (2.797,1.735) (2.873,1.734) (2.948,1.732) (3.024,1.729) (3.1,1.726) (3.175,1.722) (3.251,1.718) (3.326,1.713) (3.402,1.707) (3.478,1.7) (3.553,1.691) (3.629,1.682) (3.704,1.671) (3.78,1.659) (3.856,1.645) (3.931,1.63) (4.007,1.614) (4.082,1.597) (4.158,1.579) (4.234,1.56) (4.309,1.541) (4.385,1.522) (4.46,1.503) (4.536,1.484) (4.612,1.465) (4.687,1.446) (4.763,1.426) (4.838,1.407) (4.914,1.388) (4.99,1.369) (5.065,1.35) (5.141,1.33) (5.216,1.311) (5.292,1.292) (5.368,1.273) (5.443,1.254) (5.519,1.235) (5.594,1.215) (5.67,1.196) (5.746,1.177) (5.821,1.158) (5.897,1.139) (5.972,1.119) (6.048,1.1) (6.124,1.081)};
\draw[bwd,densely dotted] plot coordinates {(0,1.9) (0.076,1.9) (0.151,1.9) (0.227,1.9) (0.302,1.9) (0.378,1.9) (0.454,1.9) (0.529,1.9) (0.605,1.9) (0.68,1.9) (0.756,1.9) (0.832,1.9) (0.907,1.9) (0.983,1.9) (1.058,1.9) (1.134,1.9) (1.21,1.9) (1.285,1.9) (1.361,1.9) (1.436,1.9) (1.512,1.9) (1.588,1.9) (1.663,1.9) (1.739,1.9) (1.814,1.9) (1.89,1.9) (1.966,1.9) (2.041,1.9) (2.117,1.9) (2.192,1.9) (2.268,1.9) (2.344,1.9) (2.419,1.9) (2.495,1.9) (2.57,1.9) (2.646,1.9) (2.722,1.9) (2.797,1.9) (2.873,1.9) (2.948,1.9) (3.024,1.9) (3.1,1.9) (3.175,1.9) (3.251,1.9) (3.326,1.9) (3.402,1.9) (3.478,1.9) (3.553,1.9) (3.629,1.9) (3.704,1.9) (3.78,1.9) (3.856,1.9) (3.931,1.9) (4.007,1.9) (4.082,1.9) (4.158,1.9) (4.234,1.9) (4.309,1.9) (4.385,1.9) (4.46,1.9) (4.536,1.9) (4.612,1.9) (4.687,1.9) (4.763,1.9) (4.838,1.9) (4.914,1.9) (4.99,1.9) (5.065,1.9) (5.141,1.9) (5.216,1.9) (5.292,1.9) (5.368,1.9) (5.443,1.9) (5.519,1.9) (5.594,1.9) (5.67,1.9) (5.746,1.9) (5.821,1.9) (5.897,1.9) (5.972,1.9) (6.048,1.9) (6.124,1.9)};
\draw[fwd] plot coordinates {(0,2.116) (0.076,2.113) (0.151,2.111) (0.227,2.108) (0.302,2.106) (0.378,2.103) (0.454,2.101) (0.529,2.099) (0.605,2.096) (0.68,2.094) (0.756,2.092) (0.832,2.09) (0.907,2.087) (0.983,2.085) (1.058,2.083) (1.134,2.081) (1.21,2.079) (1.285,2.078) (1.361,2.076) (1.436,2.074) (1.512,2.072) (1.588,2.071) (1.663,2.069) (1.739,2.068) (1.814,2.067) (1.89,2.066) (1.966,2.065) (2.041,2.064) (2.117,2.063) (2.192,2.062) (2.268,2.062) (2.344,2.061) (2.419,2.061) (2.495,2.061) (2.57,2.062) (2.646,2.062) (2.722,2.063) (2.797,2.065) (2.873,2.066) (2.948,2.068) (3.024,2.071) (3.1,2.074) (3.175,2.078) (3.251,2.082) (3.326,2.087) (3.402,2.093) (3.478,2.1) (3.553,2.109) (3.629,2.118) (3.704,2.129) (3.78,2.141) (3.856,2.155) (3.931,2.17) (4.007,2.186) (4.082,2.203) (4.158,2.221) (4.234,2.24) (4.309,2.259) (4.385,2.278) (4.46,2.297) (4.536,2.316) (4.612,2.335) (4.687,2.354) (4.763,2.374) (4.838,2.393) (4.914,2.412) (4.99,2.431) (5.065,2.45) (5.141,2.47) (5.216,2.489) (5.292,2.508) (5.368,2.527) (5.443,2.546) (5.519,2.565) (5.594,2.585) (5.67,2.604) (5.746,2.623) (5.821,2.642) (5.897,2.661) (5.972,2.681) (6.048,2.7) (6.124,2.719)};
\draw[fwd] plot coordinates {(0,2.332) (0.076,2.327) (0.151,2.322) (0.227,2.317) (0.302,2.312) (0.378,2.307) (0.454,2.302) (0.529,2.297) (0.605,2.293) (0.68,2.288) (0.756,2.284) (0.832,2.279) (0.907,2.275) (0.983,2.271) (1.058,2.267) (1.134,2.263) (1.21,2.259) (1.285,2.255) (1.361,2.252) (1.436,2.248) (1.512,2.245) (1.588,2.242) (1.663,2.239) (1.739,2.236) (1.814,2.233) (1.89,2.231) (1.966,2.229) (2.041,2.227) (2.117,2.225) (2.192,2.224) (2.268,2.223) (2.344,2.222) (2.419,2.222) (2.495,2.222) (2.57,2.222) (2.646,2.223) (2.722,2.224) (2.797,2.226) (2.873,2.229) (2.948,2.232) (3.024,2.236) (3.1,2.24) (3.175,2.246) (3.251,2.252) (3.326,2.259) (3.402,2.267) (3.478,2.275) (3.553,2.285) (3.629,2.295) (3.704,2.307) (3.78,2.318) (3.856,2.331) (3.931,2.343) (4.007,2.356) (4.082,2.369) (4.158,2.383) (4.234,2.396) (4.309,2.409) (4.385,2.423) (4.46,2.436) (4.536,2.45) (4.612,2.463) (4.687,2.477) (4.763,2.49) (4.838,2.504) (4.914,2.517) (4.99,2.531) (5.065,2.544) (5.141,2.557) (5.216,2.571) (5.292,2.584) (5.368,2.598) (5.443,2.611) (5.519,2.625) (5.594,2.638) (5.67,2.652) (5.746,2.665) (5.821,2.678) (5.897,2.692) (5.972,2.705) (6.048,2.719) (6.124,2.732)};
\draw[fwd] plot coordinates {(0,2.548) (0.076,2.54) (0.151,2.532) (0.227,2.525) (0.302,2.517) (0.378,2.51) (0.454,2.503) (0.529,2.496) (0.605,2.489) (0.68,2.482) (0.756,2.475) (0.832,2.469) (0.907,2.462) (0.983,2.456) (1.058,2.45) (1.134,2.444) (1.21,2.438) (1.285,2.433) (1.361,2.427) (1.436,2.422) (1.512,2.417) (1.588,2.412) (1.663,2.408) (1.739,2.404) (1.814,2.4) (1.89,2.396) (1.966,2.393) (2.041,2.39) (2.117,2.387) (2.192,2.385) (2.268,2.383) (2.344,2.381) (2.419,2.38) (2.495,2.38) (2.57,2.38) (2.646,2.38) (2.722,2.382) (2.797,2.383) (2.873,2.386) (2.948,2.388) (3.024,2.392) (3.1,2.396) (3.175,2.401) (3.251,2.407) (3.326,2.413) (3.402,2.42) (3.478,2.427) (3.553,2.434) (3.629,2.443) (3.704,2.451) (3.78,2.46) (3.856,2.468) (3.931,2.477) (4.007,2.486) (4.082,2.495) (4.158,2.505) (4.234,2.514) (4.309,2.523) (4.385,2.532) (4.46,2.541) (4.536,2.55) (4.612,2.559) (4.687,2.569) (4.763,2.578) (4.838,2.587) (4.914,2.596) (4.99,2.605) (5.065,2.614) (5.141,2.623) (5.216,2.633) (5.292,2.642) (5.368,2.651) (5.443,2.66) (5.519,2.669) (5.594,2.678) (5.67,2.687) (5.746,2.697) (5.821,2.706) (5.897,2.715) (5.972,2.724) (6.048,2.733) (6.124,2.742)};
\draw[fwd] plot coordinates {(0,2.764) (0.076,2.753) (0.151,2.743) (0.227,2.733) (0.302,2.723) (0.378,2.713) (0.454,2.704) (0.529,2.694) (0.605,2.685) (0.68,2.676) (0.756,2.667) (0.832,2.658) (0.907,2.65) (0.983,2.641) (1.058,2.633) (1.134,2.625) (1.21,2.618) (1.285,2.61) (1.361,2.603) (1.436,2.596) (1.512,2.589) (1.588,2.583) (1.663,2.577) (1.739,2.571) (1.814,2.566) (1.89,2.561) (1.966,2.556) (2.041,2.552) (2.117,2.548) (2.192,2.544) (2.268,2.542) (2.344,2.539) (2.419,2.537) (2.495,2.535) (2.57,2.535) (2.646,2.534) (2.722,2.534) (2.797,2.535) (2.873,2.536) (2.948,2.538) (3.024,2.54) (3.1,2.543) (3.175,2.546) (3.251,2.549) (3.326,2.553) (3.402,2.558) (3.478,2.562) (3.553,2.567) (3.629,2.572) (3.704,2.577) (3.78,2.583) (3.856,2.588) (3.931,2.593) (4.007,2.599) (4.082,2.604) (4.158,2.61) (4.234,2.615) (4.309,2.62) (4.385,2.626) (4.46,2.631) (4.536,2.637) (4.612,2.642) (4.687,2.648) (4.763,2.653) (4.838,2.658) (4.914,2.664) (4.99,2.669) (5.065,2.675) (5.141,2.68) (5.216,2.686) (5.292,2.691) (5.368,2.697) (5.443,2.702) (5.519,2.707) (5.594,2.713) (5.67,2.718) (5.746,2.724) (5.821,2.729) (5.897,2.735) (5.972,2.74) (6.048,2.746) (6.124,2.751)};
\draw[fwd] plot coordinates {(0,2.98) (0.076,2.967) (0.151,2.954) (0.227,2.941) (0.302,2.929) (0.378,2.917) (0.454,2.905) (0.529,2.893) (0.605,2.881) (0.68,2.87) (0.756,2.859) (0.832,2.848) (0.907,2.837) (0.983,2.827) (1.058,2.816) (1.134,2.807) (1.21,2.797) (1.285,2.788) (1.361,2.778) (1.436,2.77) (1.512,2.761) (1.588,2.753) (1.663,2.745) (1.739,2.738) (1.814,2.731) (1.89,2.725) (1.966,2.718) (2.041,2.713) (2.117,2.708) (2.192,2.703) (2.268,2.698) (2.344,2.694) (2.419,2.691) (2.495,2.688) (2.57,2.686) (2.646,2.684) (2.722,2.682) (2.797,2.681) (2.873,2.681) (2.948,2.681) (3.024,2.681) (3.1,2.681) (3.175,2.682) (3.251,2.683) (3.326,2.685) (3.402,2.686) (3.478,2.688) (3.553,2.69) (3.629,2.691) (3.704,2.693) (3.78,2.695) (3.856,2.697) (3.931,2.699) (4.007,2.701) (4.082,2.703) (4.158,2.705) (4.234,2.708) (4.309,2.71) (4.385,2.712) (4.46,2.714) (4.536,2.716) (4.612,2.718) (4.687,2.72) (4.763,2.722) (4.838,2.724) (4.914,2.726) (4.99,2.728) (5.065,2.73) (5.141,2.732) (5.216,2.734) (5.292,2.736) (5.368,2.738) (5.443,2.74) (5.519,2.742) (5.594,2.744) (5.67,2.747) (5.746,2.749) (5.821,2.751) (5.897,2.753) (5.972,2.755) (6.048,2.757) (6.124,2.759)};
\draw[fwd] plot coordinates {(0,3.195) (0.076,3.18) (0.151,3.165) (0.227,3.15) (0.302,3.135) (0.378,3.12) (0.454,3.106) (0.529,3.092) (0.605,3.078) (0.68,3.064) (0.756,3.051) (0.832,3.037) (0.907,3.025) (0.983,3.012) (1.058,3) (1.134,2.988) (1.21,2.976) (1.285,2.965) (1.361,2.954) (1.436,2.943) (1.512,2.933) (1.588,2.923) (1.663,2.914) (1.739,2.905) (1.814,2.896) (1.89,2.888) (1.966,2.88) (2.041,2.873) (2.117,2.866) (2.192,2.859) (2.268,2.853) (2.344,2.848) (2.419,2.843) (2.495,2.838) (2.57,2.834) (2.646,2.83) (2.722,2.827) (2.797,2.823) (2.873,2.821) (2.948,2.818) (3.024,2.816) (3.1,2.814) (3.175,2.812) (3.251,2.811) (3.326,2.809) (3.402,2.808) (3.478,2.806) (3.553,2.805) (3.629,2.804) (3.704,2.803) (3.78,2.801) (3.856,2.8) (3.931,2.799) (4.007,2.798) (4.082,2.797) (4.158,2.796) (4.234,2.795) (4.309,2.794) (4.385,2.792) (4.46,2.791) (4.536,2.79) (4.612,2.789) (4.687,2.788) (4.763,2.787) (4.838,2.786) (4.914,2.784) (4.99,2.783) (5.065,2.782) (5.141,2.781) (5.216,2.78) (5.292,2.779) (5.368,2.778) (5.443,2.776) (5.519,2.775) (5.594,2.774) (5.67,2.773) (5.746,2.772) (5.821,2.771) (5.897,2.77) (5.972,2.769) (6.048,2.767) (6.124,2.766)};
\draw[fwd] plot coordinates {(0,3.411) (0.076,3.393) (0.151,3.376) (0.227,3.358) (0.302,3.341) (0.378,3.324) (0.454,3.307) (0.529,3.29) (0.605,3.274) (0.68,3.258) (0.756,3.242) (0.832,3.227) (0.907,3.212) (0.983,3.197) (1.058,3.183) (1.134,3.169) (1.21,3.155) (1.285,3.142) (1.361,3.129) (1.436,3.116) (1.512,3.104) (1.588,3.093) (1.663,3.081) (1.739,3.071) (1.814,3.06) (1.89,3.05) (1.966,3.041) (2.041,3.032) (2.117,3.023) (2.192,3.015) (2.268,3.007) (2.344,2.999) (2.419,2.992) (2.495,2.986) (2.57,2.979) (2.646,2.973) (2.722,2.967) (2.797,2.962) (2.873,2.957) (2.948,2.952) (3.024,2.947) (3.1,2.942) (3.175,2.938) (3.251,2.933) (3.326,2.929) (3.402,2.924) (3.478,2.92) (3.553,2.916) (3.629,2.912) (3.704,2.908) (3.78,2.903) (3.856,2.899) (3.931,2.895) (4.007,2.891) (4.082,2.887) (4.158,2.882) (4.234,2.878) (4.309,2.874) (4.385,2.87) (4.46,2.866) (4.536,2.861) (4.612,2.857) (4.687,2.853) (4.763,2.849) (4.838,2.845) (4.914,2.84) (4.99,2.836) (5.065,2.832) (5.141,2.828) (5.216,2.824) (5.292,2.819) (5.368,2.815) (5.443,2.811) (5.519,2.807) (5.594,2.803) (5.67,2.799) (5.746,2.794) (5.821,2.79) (5.897,2.786) (5.972,2.782) (6.048,2.778) (6.124,2.773)};
\draw[fwd] plot coordinates {(0,3.627) (0.076,3.607) (0.151,3.586) (0.227,3.566) (0.302,3.546) (0.378,3.527) (0.454,3.508) (0.529,3.489) (0.605,3.47) (0.68,3.452) (0.756,3.434) (0.832,3.416) (0.907,3.399) (0.983,3.382) (1.058,3.366) (1.134,3.35) (1.21,3.334) (1.285,3.319) (1.361,3.304) (1.436,3.289) (1.512,3.275) (1.588,3.262) (1.663,3.249) (1.739,3.236) (1.814,3.224) (1.89,3.212) (1.966,3.2) (2.041,3.189) (2.117,3.179) (2.192,3.168) (2.268,3.158) (2.344,3.149) (2.419,3.14) (2.495,3.131) (2.57,3.122) (2.646,3.114) (2.722,3.106) (2.797,3.098) (2.873,3.09) (2.948,3.082) (3.024,3.075) (3.1,3.067) (3.175,3.06) (3.251,3.052) (3.326,3.045) (3.402,3.038) (3.478,3.031) (3.553,3.024) (3.629,3.016) (3.704,3.009) (3.78,3.002) (3.856,2.995) (3.931,2.988) (4.007,2.981) (4.082,2.973) (4.158,2.966) (4.234,2.959) (4.309,2.952) (4.385,2.945) (4.46,2.938) (4.536,2.931) (4.612,2.923) (4.687,2.916) (4.763,2.909) (4.838,2.902) (4.914,2.895) (4.99,2.888) (5.065,2.88) (5.141,2.873) (5.216,2.866) (5.292,2.859) (5.368,2.852) (5.443,2.845) (5.519,2.838) (5.594,2.83) (5.67,2.823) (5.746,2.816) (5.821,2.809) (5.897,2.802) (5.972,2.795) (6.048,2.787) (6.124,2.78)};
\fill[black] (6.3,2.764) circle (1.8pt);
\fill[black] (6.3,1.036) circle (1.8pt);
\node[accent] at (2.406,4.016) {\scriptsize $t^*$};
\end{scope}
\end{tikzpicture}
\caption{Characteristics of the flow-matching field from $N(0,1)$ to the two points $\pm1$ (\cref{prop:pitchfork}). The midpoint is attracting until $t^*=0.382$ and repelling afterwards; the dotted line is the unstable trajectory. No two characteristics cross.}\label{fig:pitchfork}
\end{figure}
\noindent The field satisfies the continuity equation with $p_t$ to $10^{-13}$; the slope at the midpoint changes
sign at $0.381966$ and is $890$ at $t=0.9$ (\cref{fig:pitchfork}). \chk{F3} This is the transport-side picture of
\cref{thm:illposed}(c): mass begins to be committed to one of the two points at a definite time $t^*$, by the bifurcation
that \cite[Thm.~14.7]{PDEHC} describes for a buckling rod, and the field then steepens without bound as the
characteristics focus. Such transition times have been studied for diffusion models in high dimension
\cite{BBDM24}.
\section{Graph networks}\label{sec:graphs}
Let $L$ be the Laplacian of a connected graph with eigenvalues $0=\lambda_1<\lambda_2\le\dots\le\lambda_n$. The
linear part of a message-passing layer is $X\mapsto(I-\tau L)X$, one explicit Euler step of the heat equation on
the graph \cite{CRGBB21}.
\begin{theorem}[Oversmoothing is the spectral gap]\label{thm:oversmooth}
\Scl[\cite{LHW18,OoSu20}] For $0<\tau\le1/\lambda_n$ the non-constant part of $X$ is multiplied by at most
$1-\tau\lambda_2$ per layer, with equality in the limit, and the Dirichlet energy $X^{\T}LX$ by at most
$(1-\tau\lambda_2)^2$. The depth at which features become indistinguishable is of order $1/\tau\lambda_2$.
\end{theorem}
\noindent On a random connected graph with $12$ nodes, $\lambda_2=0.6251$, $\lambda_n=8.2655$ and the measured
factor is $0.92437$, as predicted; $99\%$ of the Dirichlet energy of a random input is gone after $15$ layers,
the bound from $\lambda_2$ alone being $29$. \chk{G1} The depth scale varies by orders of magnitude with the
graph: on $32$ nodes, with $\tau$ at its largest stable value, it is $415$ layers for a path
($\lambda_2=0.00963$), $5$ for the hypercube ($\lambda_2=2$, $\lambda_n=10$) and $1$ for the complete graph,
where a single layer returns the mean exactly (\cref{fig:smooth}). \chk{G2} Graphs that are good for propagating information are
the ones that oversmooth fastest; the two are the same eigenvalue.
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.6)--(0,0)--(6.3,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (2.1,0)--(2.1,-0.07) node[below,inner sep=1.5pt]{\scriptsize $20$};
\draw[ink2] (4.2,0)--(4.2,-0.07) node[below,inner sep=1.5pt]{\scriptsize $40$};
\draw[ink2] (6.3,0)--(6.3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $60$};
\draw[ink2] (0,0)--(-0.07,0) node[left,inner sep=1.5pt]{\scriptsize $10^{-8}$};
\draw[ink2] (0,0.774)--(-0.07,0.774) node[left,inner sep=1.5pt]{\scriptsize $10^{-6}$};
\draw[ink2] (0,1.548)--(-0.07,1.548) node[left,inner sep=1.5pt]{\scriptsize $10^{-4}$};
\draw[ink2] (0,2.322)--(-0.07,2.322) node[left,inner sep=1.5pt]{\scriptsize $10^{-2}$};
\draw[ink2] (0,3.096)--(-0.07,3.096) node[left,inner sep=1.5pt]{\scriptsize $1$};
\node[below] at (3.15,-0.38) {\scriptsize layers};
\draw[thick,accent] plot coordinates {(0,3.096) (0.105,3.318) (0.21,3.323) (0.315,3.125) (0.42,3.116) (0.525,3.297) (0.63,3.315) (0.735,3.189) (0.84,3.118) (0.945,3.285) (1.05,3.305) (1.155,3.206) (1.26,3.149) (1.365,3.277) (1.47,3.293) (1.575,3.222) (1.68,3.161) (1.785,3.264) (1.89,3.302) (1.995,3.215) (2.1,3.168) (2.205,3.264) (2.31,3.293) (2.415,3.222) (2.52,3.201) (2.625,3.253) (2.73,3.265) (2.835,3.248) (2.94,3.234) (3.045,3.237) (3.15,3.237) (3.255,3.249) (3.36,3.263) (3.465,3.249) (3.57,3.206) (3.675,3.219) (3.78,3.298) (3.885,3.255) (3.99,3.183) (4.095,3.206) (4.2,3.291) (4.305,3.289) (4.41,3.172) (4.515,3.166) (4.62,3.298) (4.725,3.307) (4.83,3.122) (4.935,3.166) (5.04,3.311) (5.145,3.303) (5.25,3.103) (5.355,3.16) (5.46,3.322) (5.565,3.302) (5.67,3.089) (5.775,3.138) (5.88,3.326) (5.985,3.311) (6.09,3.088) (6.195,3.134) (6.3,3.311)};
\draw[thick,fwd] plot coordinates {(0,3.096) (0.105,2.805) (0.21,2.707) (0.315,2.632) (0.42,2.569) (0.525,2.514) (0.63,2.466) (0.735,2.424) (0.84,2.388) (0.945,2.356) (1.05,2.329) (1.155,2.305) (1.26,2.284) (1.365,2.266) (1.47,2.249) (1.575,2.234) (1.68,2.22) (1.785,2.206) (1.89,2.194) (1.995,2.182) (2.1,2.17) (2.205,2.159) (2.31,2.148) (2.415,2.137) (2.52,2.127) (2.625,2.116) (2.73,2.106) (2.835,2.096) (2.94,2.087) (3.045,2.077) (3.15,2.067) (3.255,2.058) (3.36,2.049) (3.465,2.04) (3.57,2.031) (3.675,2.022) (3.78,2.014) (3.885,2.005) (3.99,1.997) (4.095,1.988) (4.2,1.98) (4.305,1.972) (4.41,1.964) (4.515,1.956) (4.62,1.949) (4.725,1.941) (4.83,1.934) (4.935,1.926) (5.04,1.919) (5.145,1.912) (5.25,1.905) (5.355,1.898) (5.46,1.891) (5.565,1.884) (5.67,1.878) (5.775,1.871) (5.88,1.865) (5.985,1.858) (6.09,1.852) (6.195,1.846) (6.3,1.84)};
\draw[thick,bwd] plot coordinates {(0,3.096) (0.105,2.849) (0.21,2.662) (0.315,2.52) (0.42,2.407) (0.525,2.31) (0.63,2.223) (0.735,2.141) (0.84,2.062) (0.945,1.984) (1.05,1.908) (1.155,1.832) (1.26,1.757) (1.365,1.682) (1.47,1.606) (1.575,1.531) (1.68,1.456) (1.785,1.381) (1.89,1.306) (1.995,1.231) (2.1,1.156) (2.205,1.081) (2.31,1.006) (2.415,0.931) (2.52,0.856) (2.625,0.781) (2.73,0.706) (2.835,0.631) (2.94,0.556) (3.045,0.481) (3.15,0.406) (3.255,0.331) (3.36,0.256) (3.465,0.181) (3.57,0.106) (3.675,0.031)};
\node[accent] at (4.62,2.74) {\scriptsize wave layer, path};
\node[fwd] at (4.41,2.089) {\scriptsize heat layer, path};
\node[bwd,right] at (2.835,0.959) {\scriptsize heat layer, hypercube};
\end{scope}
\end{tikzpicture}
\caption{Dirichlet energy of a random signal on $32$ nodes against depth, relative to its initial value. Heat-type layers lose it at a rate set by the spectral gap, slowly on a path and quickly on a hypercube; a wave-type layer on the path (\cref{thm:wavegnn}) keeps it.}\label{fig:smooth}
\end{figure}
A layer of second order in depth behaves as a wave equation \cite{RCRMB22}.
\begin{theorem}[Wave layers]\label{thm:wavegnn}
\Srf[\cite{PDEHC}, Thm.~12.3] The recursion $X_{m+1}=2X_m-X_{m-1}-\tau^2LX_m$ conserves
\[ \tau^{-2}\lVert X_{m+1}-X_m\rVert^2+X_{m+1}^{\T}LX_m, \]
which is positive definite on non-constant signals for $\tau^2\lambda_n<4$. For $\tau^2\lambda_n>4$ the top mode
grows per layer by the modulus of the larger root of $\xi^2+(\tau^2\lambda_n-2)\xi+1=0$.
\end{theorem}
\noindent Over $500$ layers at $\tau^2\lambda_n=3.6$ the invariant drifts by $2\times10^{-15}$ and the mean
Dirichlet energy of the last hundred layers is $1.001$ times that of the first hundred: nothing is smoothed. At
$\tau^2\lambda_n=4.4$ the growth factor is $1.8633$. \chk{G3} The bound is the Courant--Friedrichs--Lewy
condition, and the dichotomy between the two theorems is that of \cite[Ch.~4]{PDEHC}: a Markov semigroup
forgets, a symplectic group does not.
Both layers assume that neighbouring nodes should agree. A \term{cellular sheaf} \cite{HaGh19} replaces
agreement by agreement after transport: each edge carries linear maps from the data at its ends, and the sheaf
Laplacian $L_{\mathcal F}$ penalizes the mismatch. Its kernel is the space $H^0$ of global sections. The simplest
case is a sign $\sigma_e=\pm1$ on each edge, with $x^{\T}L_{\mathcal F}x=\sum_{e=uv}(x_u-\sigma_ex_v)^2$.
\begin{theorem}[Sheaf diffusion]\label{thm:sheaf}
\Sli[\cite{HaGh19,BDCLB22}] Diffusion by $L_{\mathcal F}$ converges to the orthogonal projection of the input
onto $H^0$. For a sign sheaf on a connected graph, $H^0$ has dimension one if the signs are a coboundary,
$\sigma_{uv}=\ell_u\ell_v$, in which case it is spanned by $\ell$; otherwise $H^0=0$. The obstruction is the class
of $\sigma$ in $H^1(G;\Z/2)$, a group of order $2^{E-V+1}$.
\end{theorem}
\noindent On a random graph with ten nodes, two classes, and signs $+$ within and $-$ between classes, sheaf
diffusion converges to the class labelling times $\langle\ell,X_0\rangle/n$, to rounding, while ordinary diffusion
converges to a constant and loses the classes. Reversing one edge that lies on a cycle makes the smallest
eigenvalue $0.1606$ and the limit zero; for a cycle of length $7$ with one negative edge it is
$4\sin^2\frac\pi{14}=0.19806$, the antiperiodic Laplacian. \chk{G4} On a graph with $5$ nodes and $7$ edges,
exactly $16=2^{V-1}$ of the $128$ sign assignments have a non-zero section, and they are exactly those whose
Laplacian is singular over the rationals; the $128$ assignments fall into $8=2^{E-V+1}$ classes. \chk{G5} This
is the same first cohomology that classifies flat gauge fields in \cite[Prop.~13.1]{PDEHC}: choosing the
transport maps of a sheaf network is choosing a connection, and heterophily is non-trivial holonomy. A sign
sheaf whose class is non-zero cannot be made consistent by relabelling nodes, and its diffusion has no
non-trivial limit.
\section{Depth as time}\label{sec:depth}
A residual layer $x\mapsto x+hf(x)$ is one step of forward Euler for $\dot x=f(x)$
\cite{E17,HaRu17}. The type of the linearization of $f$ decides which schemes are admissible, as for partial
differential equations \cite[Ch.~12]{PDEHC}.
\begin{theorem}[Layers by type]\label{thm:layers}
\Srf[\cite{HaRu17,CMHRBH18}; \cite{PDEHC}, Thm.~12.2, 12.3]
\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]
\item \emph{Parabolic}, $f=-Ax$ with $A$ positive definite: forward Euler is stable if and only if
$h\le2/\lambda_{\max}(A)$.
\item \emph{Hyperbolic}, $f=Jx$ with $J$ antisymmetric with eigenvalues $\pm i\omega$: forward Euler multiplies
the norm by $\sqrt{1+h^2\omega^2}$ per layer, for every $h$.
\item For $h,\omega>0$, the leapfrog layer $q\mapsto q+hp$ followed by $p\mapsto p-h\omega^2q$, with the new $q$,
has determinant one, conserves $p^2+\omega^2q^2+h\omega^2pq$, and has bounded powers exactly when $h\omega<2$.
\item In this stable range, the Jacobian of $N$ leapfrog layers in coordinates $(\omega q,p)$ has condition number at most
$(1+\frac{h\omega}2)/(1-\frac{h\omega}2)$, for every $N$. The Jacobian of $N$ parabolic layers with
$h\lambda_{\max}<1$ has condition number $\bigl((1-h\lambda_{\min})/(1-h\lambda_{\max})\bigr)^N$.
\end{enumerate}
\end{theorem}
\begin{proof}
(a) and (b) follow from the eigenvalues of $I+hDf$. For (c), direct computation gives the invariant,
which is positive definite when $h\omega<2$. The update matrix has trace $2-(h\omega)^2$ and determinant one.
At $h\omega=2$ it has a nontrivial Jordan block at $-1$, giving linear growth for some data.
For $h\omega>2$ an eigenvalue has modulus greater than one.
(d): a map that preserves a positive definite quadratic form $Q$ is conjugate to an orthogonal one by $Q^{1/2}$,
so its powers have condition number at most that of $Q$.
\end{proof}
\noindent For a random positive definite $A$ of size $6$ the per-layer factor is $0.9814$ at
$h=1.9/\lambda_{\max}$ and $1.1000$ at $2.1/\lambda_{\max}$. \chk{R1} Forward Euler on a rotation grows by
$1.6446$ over time $10$ with $h=0.1$ and by $1.0513$ with $h=0.01$, that is by $e^{h\omega^2T/2}$: the instability
vanishes only in the limit of infinitely many layers. \chk{R2} Leapfrog conserves its invariant to $10^{-14}$ at
$h\omega=1.9$ and grows by $1.8773$ per layer at $2.1$. \chk{R3} With $h\omega=\frac12$ the condition number of the
product of $50$, $200$ and $800$ leapfrog Jacobians is $1.07$, $1.30$ and $1.53$, below the bound $1.667$; for
fifty Euler layers on eigenvalues $1$ and $10$ with $h=0.05$ it is $8.7\times10^{13}$. \chk{R4} Since
back-propagation multiplies by the transposed Jacobians, (d) is a statement about vanishing and exploding
gradients: a symplectic architecture has neither, by the conservation law that a wave equation has and a heat
equation lacks. The price is \cref{thm:wavegnn}: it does not forget either.
\section{Kernels and losses}\label{sec:kernels}
In the kernel regime, gradient descent on the squared loss moves the function by
$f\mapsto f-\eta K(f-y)$ for a positive kernel operator $K$ \cite{JGH18}. For a translation-invariant kernel
whose Fourier coefficients decay like $k^{-2}$, as for the simplest rectifier networks on the circle
\cite{RBADLHBC19}, the model case is $K=(1-\pd_x^2)^{-1}$, the Green's operator of an elliptic equation.
\begin{theorem}[Spectral bias, with aliases]\label{thm:bias}
\Srf[\cite{RBADLHBC19,JGH18}] \emph{(a)} The residual in mode $k$ after $m$ steps is $(1-\eta\mu_k)^m$ times
its initial value, with $\mu_k=(1+k^2)^{-1}$. \emph{(b)} On $N$ equally spaced samples the eigenvalue is
\[ \mu^N_k=\sum_{m\in\Z}\frac1{1+(k+mN)^2}=\frac{\frac\pi N\sinh\frac{2\pi}N}{\cosh\frac{2\pi}N-\cos\frac{2\pi k}N}. \]
\emph{(c)} The interpolant $u=\sum_ic_iK(\cdot-x_i)$ solves $u-u''=\sum_ic_i\delta_{x_i}$.
\end{theorem}
\noindent Statement (b) is the Poisson summation formula of \cite[Thm.~4.6]{PDEHC}: sampling in space is
summation over the dual lattice. On $64$ points $\mu^N_k/\mu_k$ is $1.0016$, $1.0852$ and $1.2346$ at
$k=1,10,16$. \chk{K1} With $\eta=1$ and target $\cos x+\cos8x$, after $200$ steps mode $1$ is fitted to rounding
and $0.0380$ of mode $8$ remains; the steps needed to reach one per cent are $6.6$, $282$ and $956$ for
$k=1,8,16$ (\cref{fig:bias}). \chk{K2} For (c), with five data points the interpolant satisfies the homogeneous equation away from
the data to $10^{-9}$ and its derivative jumps by $-c_i$ at $x_i$: the representer theorem is the statement
that the minimizer is a combination of fundamental solutions. \chk{K5}
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.4)--(0,0)--(6.3,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $0$};
\draw[ink2] (1.575,0)--(1.575,-0.07) node[below,inner sep=1.5pt]{\scriptsize $100$};
\draw[ink2] (3.15,0)--(3.15,-0.07) node[below,inner sep=1.5pt]{\scriptsize $200$};
\draw[ink2] (4.725,0)--(4.725,-0.07) node[below,inner sep=1.5pt]{\scriptsize $300$};
\draw[ink2] (6.3,0)--(6.3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $400$};
\draw[ink2] (0,0)--(-0.07,0) node[left,inner sep=1.5pt]{\scriptsize $10^{-3}$};
\draw[ink2] (0,1.092)--(-0.07,1.092) node[left,inner sep=1.5pt]{\scriptsize $10^{-2}$};
\draw[ink2] (0,2.184)--(-0.07,2.184) node[left,inner sep=1.5pt]{\scriptsize $10^{-1}$};
\draw[ink2] (0,3.276)--(-0.07,3.276) node[left,inner sep=1.5pt]{\scriptsize $1$};
\node[below] at (3.15,-0.38) {\scriptsize gradient steps};
\draw[thick,ink2] plot coordinates {(0,3.276) (0.032,2.617) (0.063,1.958) (0.094,1.299) (0.126,0.64)};
\draw[thick,accent] plot coordinates {(0,3.276) (0.032,3.217) (0.063,3.159) (0.094,3.101) (0.126,3.042) (0.158,2.984) (0.189,2.926) (0.221,2.867) (0.252,2.809) (0.283,2.751) (0.315,2.692) (0.346,2.634) (0.378,2.576) (0.409,2.518) (0.441,2.459) (0.472,2.401) (0.504,2.343) (0.535,2.284) (0.567,2.226) (0.599,2.168) (0.63,2.109) (0.661,2.051) (0.693,1.993) (0.725,1.934) (0.756,1.876) (0.787,1.818) (0.819,1.76) (0.85,1.701) (0.882,1.643) (0.913,1.585) (0.945,1.526) (0.976,1.468) (1.008,1.41) (1.04,1.351) (1.071,1.293) (1.103,1.235) (1.134,1.176) (1.165,1.118) (1.197,1.06) (1.228,1.001) (1.26,0.943) (1.292,0.885) (1.323,0.827) (1.354,0.768) (1.386,0.71) (1.417,0.652) (1.449,0.593) (1.48,0.535) (1.512,0.477) (1.543,0.418) (1.575,0.36) (1.607,0.302) (1.638,0.243) (1.669,0.185) (1.701,0.127) (1.732,0.068) (1.764,0.01)};
\draw[thick,fwd] plot coordinates {(0,3.276) (0.032,3.26) (0.063,3.245) (0.094,3.229) (0.126,3.214) (0.158,3.198) (0.189,3.183) (0.221,3.167) (0.252,3.152) (0.283,3.136) (0.315,3.121) (0.346,3.105) (0.378,3.09) (0.409,3.074) (0.441,3.059) (0.472,3.043) (0.504,3.028) (0.535,3.012) (0.567,2.997) (0.599,2.981) (0.63,2.966) (0.661,2.95) (0.693,2.935) (0.725,2.919) (0.756,2.904) (0.787,2.888) (0.819,2.873) (0.85,2.857) (0.882,2.842) (0.913,2.826) (0.945,2.811) (0.976,2.795) (1.008,2.78) (1.04,2.764) (1.071,2.749) (1.103,2.733) (1.134,2.718) (1.165,2.702) (1.197,2.687) (1.228,2.671) (1.26,2.656) (1.292,2.64) (1.323,2.624) (1.354,2.609) (1.386,2.593) (1.417,2.578) (1.449,2.562) (1.48,2.547) (1.512,2.531) (1.543,2.516) (1.575,2.5) (1.607,2.485) (1.638,2.469) (1.669,2.454) (1.701,2.438) (1.732,2.423) (1.764,2.407) (1.795,2.392) (1.827,2.376) (1.859,2.361) (1.89,2.345) (1.921,2.33) (1.953,2.314) (1.984,2.299) (2.016,2.283) (2.047,2.268) (2.079,2.252) (2.111,2.237) (2.142,2.221) (2.173,2.206) (2.205,2.19) (2.236,2.175) (2.268,2.159) (2.3,2.144) (2.331,2.128) (2.362,2.113) (2.394,2.097) (2.425,2.082) (2.457,2.066) (2.489,2.051) (2.52,2.035) (2.551,2.02) (2.583,2.004) (2.615,1.989) (2.646,1.973) (2.678,1.958) (2.709,1.942) (2.74,1.927) (2.772,1.911) (2.803,1.896) (2.835,1.88) (2.866,1.865) (2.898,1.849) (2.929,1.834) (2.961,1.818) (2.993,1.803) (3.024,1.787) (3.056,1.772) (3.087,1.756) (3.118,1.741) (3.15,1.725) (3.181,1.71) (3.213,1.694) (3.244,1.679) (3.276,1.663) (3.308,1.648) (3.339,1.632) (3.371,1.617) (3.402,1.601) (3.433,1.586) (3.465,1.57) (3.496,1.555) (3.528,1.539) (3.559,1.524) (3.591,1.508) (3.623,1.493) (3.654,1.477) (3.686,1.462) (3.717,1.446) (3.748,1.431) (3.78,1.415) (3.811,1.4) (3.843,1.384) (3.874,1.369) (3.906,1.353) (3.938,1.338) (3.969,1.322) (4,1.307) (4.032,1.291) (4.063,1.276) (4.095,1.26) (4.127,1.245) (4.158,1.229) (4.189,1.214) (4.221,1.198) (4.253,1.183) (4.284,1.167) (4.316,1.152) (4.347,1.136) (4.378,1.121) (4.41,1.105) (4.441,1.09) (4.473,1.074) (4.505,1.059) (4.536,1.043) (4.567,1.028) (4.599,1.012) (4.631,0.997) (4.662,0.981) (4.693,0.966) (4.725,0.95) (4.756,0.935) (4.788,0.919) (4.819,0.904) (4.851,0.888) (4.883,0.873) (4.914,0.857) (4.946,0.842) (4.977,0.826) (5.008,0.811) (5.04,0.795) (5.071,0.78) (5.103,0.764) (5.134,0.749) (5.166,0.733) (5.197,0.718) (5.229,0.702) (5.26,0.687) (5.292,0.671) (5.324,0.656) (5.355,0.64) (5.386,0.625) (5.418,0.609) (5.449,0.594) (5.481,0.578) (5.513,0.563) (5.544,0.547) (5.575,0.532) (5.607,0.516) (5.639,0.501) (5.67,0.485) (5.701,0.47) (5.733,0.454) (5.764,0.439) (5.796,0.423) (5.827,0.408) (5.859,0.392) (5.89,0.377) (5.922,0.361) (5.954,0.346) (5.985,0.33) (6.016,0.315) (6.048,0.299) (6.079,0.284) (6.111,0.268) (6.143,0.253) (6.174,0.237) (6.205,0.222) (6.237,0.206) (6.269,0.191) (6.3,0.175)};
\draw[thick,bwd] plot coordinates {(0,3.276) (0.032,3.271) (0.063,3.266) (0.094,3.262) (0.126,3.257) (0.158,3.253) (0.189,3.248) (0.221,3.244) (0.252,3.239) (0.283,3.234) (0.315,3.23) (0.346,3.225) (0.378,3.221) (0.409,3.216) (0.441,3.212) (0.472,3.207) (0.504,3.203) (0.535,3.198) (0.567,3.193) (0.599,3.189) (0.63,3.184) (0.661,3.18) (0.693,3.175) (0.725,3.171) (0.756,3.166) (0.787,3.161) (0.819,3.157) (0.85,3.152) (0.882,3.148) (0.913,3.143) (0.945,3.139) (0.976,3.134) (1.008,3.129) (1.04,3.125) (1.071,3.12) (1.103,3.116) (1.134,3.111) (1.165,3.107) (1.197,3.102) (1.228,3.097) (1.26,3.093) (1.292,3.088) (1.323,3.084) (1.354,3.079) (1.386,3.075) (1.417,3.07) (1.449,3.066) (1.48,3.061) (1.512,3.056) (1.543,3.052) (1.575,3.047) (1.607,3.043) (1.638,3.038) (1.669,3.034) (1.701,3.029) (1.732,3.024) (1.764,3.02) (1.795,3.015) (1.827,3.011) (1.859,3.006) (1.89,3.002) (1.921,2.997) (1.953,2.992) (1.984,2.988) (2.016,2.983) (2.047,2.979) (2.079,2.974) (2.111,2.97) (2.142,2.965) (2.173,2.96) (2.205,2.956) (2.236,2.951) (2.268,2.947) (2.3,2.942) (2.331,2.938) (2.362,2.933) (2.394,2.929) (2.425,2.924) (2.457,2.919) (2.489,2.915) (2.52,2.91) (2.551,2.906) (2.583,2.901) (2.615,2.897) (2.646,2.892) (2.678,2.887) (2.709,2.883) (2.74,2.878) (2.772,2.874) (2.803,2.869) (2.835,2.865) (2.866,2.86) (2.898,2.855) (2.929,2.851) (2.961,2.846) (2.993,2.842) (3.024,2.837) (3.056,2.833) (3.087,2.828) (3.118,2.823) (3.15,2.819) (3.181,2.814) (3.213,2.81) (3.244,2.805) (3.276,2.801) (3.308,2.796) (3.339,2.792) (3.371,2.787) (3.402,2.782) (3.433,2.778) (3.465,2.773) (3.496,2.769) (3.528,2.764) (3.559,2.76) (3.591,2.755) (3.623,2.75) (3.654,2.746) (3.686,2.741) (3.717,2.737) (3.748,2.732) (3.78,2.728) (3.811,2.723) (3.843,2.718) (3.874,2.714) (3.906,2.709) (3.938,2.705) (3.969,2.7) (4,2.696) (4.032,2.691) (4.063,2.686) (4.095,2.682) (4.127,2.677) (4.158,2.673) (4.189,2.668) (4.221,2.664) (4.253,2.659) (4.284,2.655) (4.316,2.65) (4.347,2.645) (4.378,2.641) (4.41,2.636) (4.441,2.632) (4.473,2.627) (4.505,2.623) (4.536,2.618) (4.567,2.613) (4.599,2.609) (4.631,2.604) (4.662,2.6) (4.693,2.595) (4.725,2.591) (4.756,2.586) (4.788,2.581) (4.819,2.577) (4.851,2.572) (4.883,2.568) (4.914,2.563) (4.946,2.559) (4.977,2.554) (5.008,2.549) (5.04,2.545) (5.071,2.54) (5.103,2.536) (5.134,2.531) (5.166,2.527) (5.197,2.522) (5.229,2.518) (5.26,2.513) (5.292,2.508) (5.324,2.504) (5.355,2.499) (5.386,2.495) (5.418,2.49) (5.449,2.486) (5.481,2.481) (5.513,2.476) (5.544,2.472) (5.575,2.467) (5.607,2.463) (5.639,2.458) (5.67,2.454) (5.701,2.449) (5.733,2.444) (5.764,2.44) (5.796,2.435) (5.827,2.431) (5.859,2.426) (5.89,2.422) (5.922,2.417) (5.954,2.412) (5.985,2.408) (6.016,2.403) (6.048,2.399) (6.079,2.394) (6.111,2.39) (6.143,2.385) (6.174,2.381) (6.205,2.376) (6.237,2.371) (6.269,2.367) (6.3,2.362)};
\node[ink2,right] at (0.346,0.521) {\scriptsize $k=1$};
\node[accent,right] at (1.496,1.178) {\scriptsize $4$};
\node[fwd] at (4.567,1.466) {\scriptsize $8$};
\node[bwd] at (5.197,2.75) {\scriptsize $16$};
\end{scope}
\end{tikzpicture}
\caption{Spectral bias in the kernel regime: the remaining fraction of mode $k$ of the target against the number of gradient steps, for the sampled $H^1$ kernel on $64$ points (\cref{thm:bias}).}\label{fig:bias}
\end{figure}
A physics-informed loss \cite{RPK19} for $-u''=f$ is the squared residual $\lVert Au-f\rVert^2$ with
$A=-\pd_x^2$; the energy, or Ritz, loss \cite{EYu18} is $\frac12\langle u,Au\rangle-\langle f,u\rangle$.
\begin{theorem}[Squaring the operator]\label{thm:pinn}
\Scl[cf.\ \cite{KGZKM21,WTP21}] \emph{(a)} The Hessian of the strong-form loss is $A^{\T}A$ and that of the
energy loss is $A$: the condition numbers are $\kappa^2$ and $\kappa$, with $\kappa\sim4N^2/\pi^2$ for $N$
unknowns. \emph{(b)} Composed with the kernel of \cref{thm:bias}, the strong-form loss has mode rates
$\mu_kk^4=k^4/(1+k^2)$, increasing in $k$.
\end{theorem}
\noindent On $31$ unknowns $\kappa=414.3$ and $\kappa^2=171{,}682$; gradient descent with the optimal step needs
$2862$ and $1{,}185{,}936$ iterations to gain six digits. \chk{K3} On $64$ points the regression rates at $k=1$ and
$16$ are $0.5008$ and $0.0048$, a ratio of $1/104$, and the strong-form rates are $0.5000$ and $206.8$, a ratio of
$414$. \chk{K4} The bias is reversed and the spread of rates is four times as large: the low modes, which carry the solution,
are now the slow ones relative to the step that the high modes allow. This
is a linear model of the training difficulties reported for physics-informed networks \cite{KGZKM21,WTP21}, and
it is the reason the weak formulation is the standard one in numerical analysis
\cite[Ch.~12]{PDEHC}.
\section{Noise}\label{sec:noise}
Gradient descent with small isotropic noise on a loss $V$ is modelled by the Langevin equation
$dX=-V'(X)\,dt+\sqrt{2/\beta}\,dW$, whose density solves a Fokker--Planck equation with stationary solution
proportional to $e^{-\beta V}$ \cite{MHB17}.
\begin{proposition}\label{prop:langevin}
\Scl[\cite{Kra40,BaEm85}] \emph{(a)} For $V=\frac12\lambda x^2$ the spectral gap is $\lambda$ for every $\beta$.
\emph{(b)} For a symmetric double well with minima $\pm a$ and barrier $\Delta V$ at $0$, the gap is
$\frac1\pi\sqrt{V''(a)|V''(0)|}\,e^{-\beta\Delta V}(1+O(\beta^{-1}))$. \emph{(c)} Constant-step stochastic gradient
descent on $\frac12\lambda x^2$ with gradient noise of variance $\sigma^2$ has stationary variance
$\eta\sigma^2/\lambda(2-\eta\lambda)$.
\end{proposition}
\noindent A reversible discretization has the Gibbs density as its exact stationary vector, and for $\lambda=1$ its
gap is $0.9997$ at $\beta=1$ and at $\beta=4$. \chk{L1} For $V=(x^2-1)^2$ the gap divided by
$\frac{\sqrt{32}}\pi e^{-\beta}$ is $0.903$, $0.928$, $0.947$, $0.959$ at $\beta=4,6,8,10$, and the gap at
$\beta=10$ is $7.8\times10^{-5}$. \chk{L2} The recursion for the variance converges to $0.0625$ for $\eta=0.1$,
$\lambda=2$, $\sigma=1.5$; the first-order value $\eta\sigma^2/2\lambda$ is $0.05625$, corresponding to the
temperature $\beta^{-1}=\eta\sigma^2/2$. \chk{L3} Within a basin, mixing is as fast as the curvature and does not
depend on the noise; between basins it is exponentially slow in the ratio of barrier height to noise. The
exponent in (b) is where a claim that noise selects wide minima has to be made quantitative.
\section{Control}\label{sec:control}
\begin{proposition}\label{prop:bellman}
\Scl[\cite{CrLi83,BaSo91}] The Bellman operator $TV=\min_a(c_a+\gamma P_aV)$ is monotone and a contraction of
factor $\gamma$ in the supremum norm. A monotone, stable and consistent scheme for a Hamilton--Jacobi equation
converges to its viscosity solution.
\end{proposition}
\noindent For a random problem with $6$ states and $3$ actions, monotonicity and the contraction hold on $300$
random pairs, with largest ratio $0.7072$ at $\gamma=0.9$, and value iteration obeys the geometric bound.
\chk{Q1} For $|u'|=1$ on the unit interval with zero boundary values the scheme
$u_i=\min(u_{i-1},u_{i+1})+h$ converges to the distance to the boundary exactly, and has residual $2h$ on a
sawtooth and on minus the distance, which satisfy the equation almost everywhere. \chk{Q2} Monotonicity is the
discrete maximum principle: the Bellman operator is to control what a Markov kernel is to diffusion, and the
selection of the viscosity solution is the selection of the entropy solution of a conservation law
\cite[Thm.~14.3]{PDEHC}.
\section{How many numbers an operator needs}\label{sec:rank}
Operator learning approximates a solution map by a finite-rank one, composed with nonlinear maps
\cite{LJPZK21,KLLABSA23}. For linear problems the best rank-$r$ error is the $(r+1)$-st singular value.
\begin{theorem}[Rank by type]\label{thm:rank}
\Scl On the unit interval with Dirichlet conditions:
\begin{enumerate}[label=(\alph*),leftmargin=1.6em,itemsep=1pt,topsep=2pt]
\item \emph{Elliptic.} $(-\pd_x^2)^{-1}$ has singular values $(\pi k)^{-2}$: relative accuracy $\epsilon$ needs
$\epsilon^{-1/2}$ modes.
\item \emph{Parabolic.} $e^{t\pd_x^2}$ has singular values $e^{-\pi^2k^2t}$: accuracy $\epsilon$ needs
$\sqrt{1+\log(1/\epsilon)/\pi^2t}$ modes.
\item \emph{Hyperbolic.} The wave propagator is an isometry of the energy: every mode is needed.
\item Translates of a step function have covariance eigenvalues $((k-\frac12)\pi)^{-2}$, so the best
$r$-dimensional linear approximation of the family has relative error $\sim(\pi^2r/2)^{-1/2}$.
\end{enumerate}
\end{theorem}
\noindent At $\epsilon=10^{-6}$ this is $1000$ modes for the elliptic inverse and $11$ for the heat flow at
$t=0.01$ ($12$ on a grid of $63$ points, whose stencil underestimates the higher eigenvalues as in
\cref{thm:alias}(b)); the discrete wave propagator preserves the energy form to $10^{-15}$. \chk{O1} For the
steps, the first three eigenvalues are $0.40531$, $0.04505$, $0.01623$ on $60$ points, and the relative error
is $0.142$ with ten modes and $0.045$ with a hundred (\cref{fig:rank}). \chk{O2} A linear-reduction architecture is therefore
well matched to parabolic problems, adequate for elliptic ones, and badly matched to transport, where the
natural coordinates are the characteristics of \cref{sec:flows} and not a fixed basis. This is the Kolmogorov
barrier of reduced-order modelling \cite{OhRa16}, read off the type of the equation.
\begin{figure}[t]
\centering
\begin{tikzpicture}
\begin{scope}[shift={(0,0)}]
\draw[ink2] (0,3.6)--(0,0)--(6.3,0);
\draw[ink2] (0,0)--(0,-0.07) node[below,inner sep=1.5pt]{\scriptsize $1$};
\draw[ink2] (1.454,0)--(1.454,-0.07) node[below,inner sep=1.5pt]{\scriptsize $10$};
\draw[ink2] (3.069,0)--(3.069,-0.07) node[below,inner sep=1.5pt]{\scriptsize $20$};
\draw[ink2] (4.685,0)--(4.685,-0.07) node[below,inner sep=1.5pt]{\scriptsize $30$};
\draw[ink2] (6.3,0)--(6.3,-0.07) node[below,inner sep=1.5pt]{\scriptsize $40$};
\draw[ink2] (0,0)--(-0.07,0) node[left,inner sep=1.5pt]{\scriptsize $10^{-6}$};
\draw[ink2] (0,1.112)--(-0.07,1.112) node[left,inner sep=1.5pt]{\scriptsize $10^{-4}$};
\draw[ink2] (0,2.223)--(-0.07,2.223) node[left,inner sep=1.5pt]{\scriptsize $10^{-2}$};
\draw[ink2] (0,3.335)--(-0.07,3.335) node[left,inner sep=1.5pt]{\scriptsize $1$};
\node[below] at (3.15,-0.38) {\scriptsize mode $k$};
\draw[thick,accent] plot coordinates {(0,3.335) (0.162,3.335) (0.323,3.335) (0.485,3.335) (0.646,3.335) (0.808,3.335) (0.969,3.335) (1.131,3.335) (1.292,3.335) (1.454,3.335) (1.615,3.335) (1.777,3.335) (1.938,3.335) (2.1,3.335) (2.262,3.335) (2.423,3.335) (2.585,3.335) (2.746,3.335) (2.908,3.335) (3.069,3.335) (3.231,3.335) (3.392,3.335) (3.554,3.335) (3.715,3.335) (3.877,3.335) (4.038,3.335) (4.2,3.335) (4.362,3.335) (4.523,3.335) (4.685,3.335) (4.846,3.335) (5.008,3.335) (5.169,3.335) (5.331,3.335) (5.492,3.335) (5.654,3.335) (5.815,3.335) (5.977,3.335) (6.138,3.335) (6.3,3.335)};
\draw[thick,ink2,densely dashed] plot coordinates {(0,3.335) (0.162,3.07) (0.323,2.946) (0.485,2.865) (0.646,2.804) (0.808,2.756) (0.969,2.716) (1.131,2.681) (1.292,2.651) (1.454,2.624) (1.615,2.6) (1.777,2.578) (1.938,2.558) (2.1,2.539) (2.262,2.522) (2.423,2.506) (2.585,2.491) (2.746,2.477) (2.908,2.463) (3.069,2.45) (3.231,2.438) (3.392,2.427) (3.554,2.416) (3.715,2.405) (3.877,2.395) (4.038,2.386) (4.2,2.376) (4.362,2.368) (4.523,2.359) (4.685,2.351) (4.846,2.343) (5.008,2.335) (5.169,2.327) (5.331,2.32) (5.492,2.313) (5.654,2.306) (5.815,2.299) (5.977,2.293) (6.138,2.286) (6.3,2.28)};
\draw[thick,fwd] plot coordinates {(0,3.335) (0.162,3) (0.323,2.804) (0.485,2.666) (0.646,2.558) (0.808,2.47) (0.969,2.395) (1.131,2.331) (1.292,2.274) (1.454,2.223) (1.615,2.177) (1.777,2.135) (1.938,2.097) (2.1,2.061) (2.262,2.027) (2.423,1.996) (2.585,1.967) (2.746,1.939) (2.908,1.913) (3.069,1.889) (3.231,1.865) (3.392,1.843) (3.554,1.821) (3.715,1.801) (3.877,1.781) (4.038,1.762) (4.2,1.744) (4.362,1.726) (4.523,1.709) (4.685,1.693) (4.846,1.677) (5.008,1.662) (5.169,1.647) (5.331,1.632) (5.492,1.618) (5.654,1.605) (5.815,1.592) (5.977,1.579) (6.138,1.566) (6.3,1.554)};
\draw[thick,bwd] plot coordinates {(0,3.335) (0.162,3.263) (0.323,3.144) (0.485,2.977) (0.646,2.763) (0.808,2.501) (0.969,2.191) (1.131,1.834) (1.292,1.429) (1.454,0.976) (1.615,0.476)};
\node[accent] at (4.685,3.502) {\scriptsize wave propagator};
\node[ink2] at (4.685,2.656) {\scriptsize translated steps};
\node[fwd] at (4.846,1.342) {\scriptsize elliptic inverse};
\node[bwd,right] at (2.1,0.723) {\scriptsize heat flow};
\end{scope}
\end{tikzpicture}
\caption{Singular values of solution operators on the unit interval, relative to the first, by type (\cref{thm:rank}): none decay for the wave equation, they decay algebraically for the elliptic inverse and for translates of a step, and faster than exponentially for the heat flow at $t=0.01$.}\label{fig:rank}
\end{figure}
\section{Summary of results}\label{sec:summary}
\begin{table*}[t]
\centering\small
\begin{tabular}{@{}>{\raggedright}p{0.15\textwidth}>{\raggedright}p{0.31\textwidth}>{\raggedright}p{0.36\textwidth}l@{}}
\toprule
Topic & Qualitative statement & Quantitative statement & Checks\\
\midrule
Padding & reflect $=$ Neumann by images, iff kernels symmetric; zero $=$ Dirichlet only for radius one & defects $1.18$, $0.100$, $0.160$; mass $0.532$ left after $200$ zero-padded layers & N1--N6\\
Resolution & multipliers natural; stencils to second order; polynomials iff $N>(d+1)K$; rectifiers never & symbol $0.717$, $0.980$ of $k^2$; rectifier error $-\pi/3N^2$ & A1--A4\\
Diffusion models & reversal is ill-posed for densities, well-posed for samples given the score & $5.2\times10^{21}$ against $0.264$; score slope $-1/t+a^2/t^2$ & S3--S5\\
Score matching, entropy & denoising $=$ explicit $+$ constant; de Bruijn; contraction & constant $1/t-J=0.924$; rate $2$, limit $\frac12(\E X_0)^2$ & S2, S6--S8\\
Flow matching & in one dimension, the monotone map for any coupling; commitment by a pitchfork & error $10^{-13}$; $t^*=0.382$; slope $890$ at $t=0.9$ & F1--F3\\
Message passing & oversmoothing is the spectral gap & factor $0.92437$; depth scales $415$, $5$, $1$ & G1, G2\\
Wave layers & conserve an energy; CFL bound & drift $2\times10^{-15}$; growth $1.8633$ beyond & G3\\
Sheaf diffusion & limit is the projection to $H^0$; obstruction in $H^1(G;\Z/2)$ & $16$ of $128$; gaps $0.1606$, $0.19806$ & G4, G5\\
Residual layers & Euler unstable on antisymmetric layers; leapfrog bounded gradients & $1.6446$, $1.0513$; condition $\le1.667$ against $8.7\times10^{13}$ & R1--R4\\
Kernel regression & low modes first; sampled spectrum is a Poisson sum & ratios $1.0016$, $1.0852$, $1.2346$; steps $6.6$, $282$, $956$ & K1, K2, K5\\
Physics-informed loss & squares the condition number and reverses the bias & $414$ against $171{,}682$; rate ratios $1/104$ and $414$ & K3, K4\\
Noise & gap $=$ curvature in a basin, Kramers between basins & $0.9997$; ratios $0.903,\dots,0.959$ & L1--L3\\
Control & Bellman operator monotone contraction; monotone scheme selects the viscosity solution & ratio $\le0.7072$; residual $2h$ on the wrong solutions & Q1, Q2\\
Operator rank & algebraic, exponential, none, by type & $1000$, $11$, all; steps: $0.142$ with ten modes & O1, O2\\
\bottomrule
\end{tabular}
\caption{Qualitative statements with their quantitative counterparts. Every number is recomputed by the companion script.}\label{tab:summary}
\end{table*}
\Cref{tab:summary} pairs the statements. Read by column of the title, the entries on padding, resolution, flow
matching and sheaves are about naturality; those on diffusion models, residual layers, physics-informed losses
and control are about well-posedness; and those on message passing, kernels, noise and rank are about rates. The
divisions are not sharp, because in each case the three questions are answered by the same object: a group
acting on a domain and the spectrum of an operator that commutes with it.
\section{Limits}\label{sec:limits}
\emph{Linearity.} Most results concern the linear part of a layer or the kernel regime of training. Nonlinear
layers enter only where they can be treated exactly: pointwise maps commute with even extension
(\cref{thm:reflect}), and their aliasing is computed (\cref{thm:alias}). Nothing is proved about the training
dynamics of a finite network.
\emph{Dimension.} The examples are one-dimensional or on small graphs. \Cref{thm:reflect} extends to boxes and
to the other chambers of reflection groups \cite[Rem.~3.6]{PDEHC}; \cref{thm:fm} does not extend, and says so.
\emph{The score is assumed.} \Cref{thm:illposed} takes the score as given with a uniform error. How well a
network estimates it, and in which norm, is the statistical question that the theorem isolates and does not
answer.
\emph{Models of noise.} \Cref{prop:langevin} models stochastic gradient descent by isotropic Langevin dynamics,
which is known to be a rough approximation: real gradient noise is anisotropic and state-dependent.
\emph{Checks.} The script verifies instances with a fixed seed. A check that passes on a random instance
supports a statement and does not prove it; the proofs are those given or cited.
\emph{Novelty.} We know of no result here that a specialist in the relevant area would find new. The statements
tagged as observations (\cref{prop:zero}, \cref{thm:fm}, \cref{prop:pitchfork}) are elementary and may well be
in the literature in other words.
\emph{What is higher.} As in \cite{PDEHC}, the categorical content is modest: naturality squares, a group
action with its characters, and cohomology in degree one for sheaves on graphs. The one place where a higher
structure is forced, gauge fields forming a groupoid \cite[Rem.~13.2]{PDEHC}, has a counterpart here in
\cref{thm:sheaf}, where relabelling nodes is a gauge transformation; we have not developed it.
\appendix
\section{The companion script}\label{app:checks}
\texttt{verify-pde-learning.py} runs in about a second and prints one line per check. The checks are:
\textbf{N1--N6} padding rules, their eigenbases, the image constructions and their failures, mass and the
maximum principle;
\textbf{A1--A4} resampling of multipliers, stencils, squares and rectifiers;
\textbf{S1--S8} the heat equation for a Gaussian mixture, Tweedie's formula, the probability flow, sensitivity
to the score, the two-point score, de Bruijn's identity, contraction of relative entropy, and the
score-matching constant;
\textbf{F1--F3} change of variables along characteristics, flow matching between Gaussians, and the two-point
field with its bifurcation time;
\textbf{G1--G5} oversmoothing rates, depth scales on three graphs, the wave layer, sheaf diffusion, and the
count of balanced sign sheaves;
\textbf{R1--R4} stability of Euler and leapfrog layers and the conditioning of their Jacobians;
\textbf{K1--K5} the sampled kernel spectrum, kernel gradient descent, the two condition numbers, the reversed
rates, and the interpolant as a sum of fundamental solutions;
\textbf{L1--L3} Fokker--Planck gaps and the variance of stochastic gradient descent;
\textbf{Q1--Q2} the Bellman operator and the monotone scheme for the eikonal equation;
\textbf{O1--O2} ranks of solution operators and the spectrum of translated steps.
Random inputs are drawn with seed $20261117$. Tolerances are stated in the script next to each check. The curves in the figures were computed from the formulas of the text by a separate plotting script and are embedded in the source as coordinates; the points of \cref{fig:alias} for the random input are the values printed by check A4, and the signal of \cref{fig:smooth} is a random one drawn for the figure.
\begin{thebibliography}{RBADLHBC19}\footnotesize\setlength{\itemsep}{0.5pt}
\newcommand{\bi}[1]{\bibitem[#1]{#1}}
\bi{And82} B.~D.~O.~Anderson, Reverse-time diffusion equation models, \emph{Stochastic Process.\ Appl.} 12 (1982), 313--326.
\bi{BaEm85} D.~Bakry, M.~\'Emery, Diffusions hypercontractives, in \emph{S\'eminaire de Probabilit\'es XIX}, Lecture Notes in Math.\ 1123, Springer, 1985, 177--206.
\bi{BaSo91} G.~Barles, P.~E.~Souganidis, Convergence of approximation schemes for fully nonlinear second order equations, \emph{Asymptotic Anal.} 4 (1991), 271--283.
\bi{BBCV21} M.~M.~Bronstein, J.~Bruna, T.~Cohen, P.~Veli\v{c}kovi\'c, Geometric deep learning: grids, groups, graphs, geodesics, and gauges, arXiv:2104.13478, 2021.
\bi{BBDM24} G.~Biroli, T.~Bonnaire, V.~de Bortoli, M.~M\'ezard, Dynamical regimes of diffusion models, \emph{Nature Commun.} 15 (2024).
\bi{BBRMMA23} F.~Bartolucci, E.~de B\'ezenac, B.~Raoni\'c, R.~Molinaro, S.~Mishra, R.~Alaifari, Representation equivalent neural operators: a framework for alias-free operator learning, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 36 (2023); arXiv:2305.19913. Checked.
\bi{BDCLB22} C.~Bodnar, F.~Di Giovanni, B.~P.~Chamberlain, P.~Li\`o, M.~M.~Bronstein, Neural sheaf diffusion: a topological perspective on heterophily and oversmoothing in GNNs, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 35 (2022); arXiv:2202.04579. Checked.
\bi{CMHRBH18} B.~Chang, L.~Meng, E.~Haber, L.~Ruthotto, D.~Begert, E.~Holtham, Reversible architectures for arbitrarily deep residual neural networks, \emph{Proc.\ AAAI} 32 (2018).
\bi{CoWe16} T.~Cohen, M.~Welling, Group equivariant convolutional networks, \emph{Proc.\ ICML} 2016, 2990--2999.
\bi{CRBD18} R.~T.~Q.~Chen, Y.~Rubanova, J.~Bettencourt, D.~Duvenaud, Neural ordinary differential equations, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 31 (2018).
\bi{CRGBB21} B.~P.~Chamberlain, J.~Rowbottom, M.~Gorinova, S.~Webb, E.~Rossi, M.~M.~Bronstein, GRAND: graph neural diffusion, \emph{Proc.\ ICML} 2021, 1407--1418.
\bi{CrLi83} M.~G.~Crandall, P.-L.~Lions, Viscosity solutions of Hamilton--Jacobi equations, \emph{Trans.\ Amer.\ Math.\ Soc.} 277 (1983), 1--42.
\bi{DCWJXKY26} Dai, Chen, Wang, Jia, Xie, Kumar, Yu, Learning PDE solvers with physics and data: a unifying view of physics-informed neural networks and neural operators, arXiv:2601.14517, 2026. Checked; surnames only were recorded.
\bi{E17} W.~E, A proposal on machine learning via dynamical systems, \emph{Commun.\ Math.\ Stat.} 5 (2017), 1--11.
\bi{Efr11} B.~Efron, Tweedie's formula and selection bias, \emph{J.~Amer.\ Statist.\ Assoc.} 106 (2011), 1602--1614.
\bi{EYu18} W.~E, B.~Yu, The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, \emph{Commun.\ Math.\ Stat.} 6 (2018), 1--12.
\bi{HaGh19} J.~Hansen, R.~Ghrist, Toward a spectral theory of cellular sheaves, \emph{J.~Appl.\ Comput.\ Topol.} 3 (2019), 315--358.
\bi{HaRu17} E.~Haber, L.~Ruthotto, Stable architectures for deep neural networks, \emph{Inverse Problems} 34 (2017), 014004.
\bi{HJA20} J.~Ho, A.~Jain, P.~Abbeel, Denoising diffusion probabilistic models, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 33 (2020).
\bi{Hyv05} A.~Hyv\"arinen, Estimation of non-normalized statistical models by score matching, \emph{J.~Mach.\ Learn.\ Res.} 6 (2005), 695--709.
\bi{ILB20} M.~A.~Islam, S.~Jia, N.~D.~B.~Bruce, How much position information do convolutional neural networks encode?, \emph{Proc.\ ICLR} 2020.
\bi{JGH18} A.~Jacot, F.~Gabriel, C.~Hongler, Neural tangent kernel: convergence and generalization in neural networks, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 31 (2018).
\bi{KALHHLA21} T.~Karras, M.~Aittala, S.~Laine, E.~H\"ark\"onen, J.~Hellsten, J.~Lehtinen, T.~Aila, Alias-free generative adversarial networks, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 34 (2021).
\bi{KGZKM21} A.~S.~Krishnapriyan, A.~Gholami, S.~Zhe, R.~M.~Kirby, M.~W.~Mahoney, Characterizing possible failure modes in physics-informed neural networks, \emph{Adv.\ Neural Inf.\ Process.\ Syst.} 34 (2021).
\bi{KLLABSA23} N.~Kovachki, Z.~Li, B.~Liu, K.~Azizzadenesheli, K.~Bhattacharya, A.~Stuart, A.~Anandkumar, Neural operator: learning maps between function spaces with applications to PDEs, \emph{J.~Mach.\ Learn.\ Res.} 24 (2023), no.~89.
\bi{Kra40} H.~A.~Kramers, Brownian motion in a field of force and the diffusion model of chemical reactions, \emph{Physica} 7 (1940), 284--304.
\bi{LCBNL23} Y.~Lipman, R.~T.~Q.~Chen, H.~Ben-Hamu, M.~Nickel, M.~Le, Flow matching for generative modeling, \emph{Proc.\ ICLR} 2023.
\bi{LGL23} X.~Liu, C.~Gong, Q.~Liu, Flow straight and fast: learning to generate and transfer data with rectified flow, \emph{Proc.\ ICLR} 2023.
\bi{LHW18} Q.~Li, Z.~Han, X.-M.~Wu, Deeper insights into graph convolutional networks for semi-supervised learning, \emph{Proc.\ AAAI} 32 (2018).
\bi{LJPZK21} L.~Lu, P.~Jin, G.~Pang, Z.~Zhang, G.~E.~Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, \emph{Nature Mach.\ Intell.} 3 (2021), 218--229.
\bi{LKALBSA21} Z.~Li, N.~Kovachki, K.~Azizzadenesheli, B.~Liu, K.~Bhattacharya, A.~Stuart, A.~Anandkumar, Fourier neural operator for parametric partial differential equations, \emph{Proc.\ ICLR} 2021.
\bi{MHB17} S.~Mandt, M.~D.~Hoffman, D.~M.~Blei, Stochastic gradient descent as approximate Bayesian inference, \emph{J.~Mach.\ Learn.\ Res.} 18 (2017), no.~134.
\bi{OhRa16} M.~Ohlberger, S.~Rave, Reduced basis methods: success, limitations and future challenges, \emph{Proc.\ ALGORITMY} 2016, 1--12.
\bi{OoSu20} K.~Oono, T.~Suzuki, Graph neural networks exponentially lose expressive power for node classification, \emph{Proc.\ ICLR} 2020.
\bi{Ors71} S.~A.~Orszag, On the elimination of aliasing in finite-difference schemes by filtering high-wavenumber components, \emph{J.~Atmos.\ Sci.} 28 (1971), 1074.
\bi{PDEHC} Anonymous, \emph{Partial Differential Equations in Higher Categories}, Higher Categorical Semantics, 2026. Companion volume.
\bi{RBADLHBC19} N.~Rahaman, A.~Baratin, D.~Arpit, F.~Draxler, M.~Lin, F.~Hamprecht, Y.~Bengio, A.~Courville, On the spectral bias of neural networks, \emph{Proc.\ ICML} 2019, 5301--5310.
\bi{RCRMB22} T.~K.~Rusch, B.~P.~Chamberlain, J.~Rowbottom, S.~Mishra, M.~M.~Bronstein, Graph-coupled oscillator networks, \emph{Proc.\ ICML} 2022, 18888--18909.
\bi{RPK19} M.~Raissi, P.~Perdikaris, G.~E.~Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, \emph{J.~Comput.\ Phys.} 378 (2019), 686--707.
\bi{SSKKEP21} Y.~Song, J.~Sohl-Dickstein, D.~P.~Kingma, A.~Kumar, S.~Ermon, B.~Poole, Score-based generative modeling through stochastic differential equations, \emph{Proc.\ ICLR} 2021.
\bi{Sta59} A.~J.~Stam, Some inequalities satisfied by the quantities of information of Fisher and Shannon, \emph{Inform.\ and Control} 2 (1959), 101--112.
\bi{Str08} W.~A.~Strauss, \emph{Partial Differential Equations: An Introduction}, 2nd ed., Wiley, 2008.
\bi{Str99} G.~Strang, The discrete cosine transform, \emph{SIAM Rev.} 41 (1999), 135--147.
\bi{SWMG15} J.~Sohl-Dickstein, E.~Weiss, N.~Maheswaranathan, S.~Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, \emph{Proc.\ ICML} 2015, 2256--2265.
\bi{Vin11} P.~Vincent, A connection between score matching and denoising autoencoders, \emph{Neural Comput.} 23 (2011), 1661--1674.
\bi{WTP21} S.~Wang, Y.~Teng, P.~Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, \emph{SIAM J.~Sci.\ Comput.} 43 (2021), A3055--A3081.
\end{thebibliography}
\end{document}