Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates
Padding as a boundary condition, depth as time, the score as the datum that makes reversal well-posed
Convolutional layers are difference operators, depth is a time variable, generative diffusion is the heat equation, message passing is a graph Laplacian, and a kernel is a Green's function. This paper organizes those identifications by the three questions that the theory of partial differential equations asks of any construction, and attaches a computed constant to each answer. It is the application paper to the companion volume Partial Differential Equations in Higher Categories.
Format. Double-column formal paper, 11 pages, 7 figures, 46 references; status tags on every numbered statement (classical, literature, reformulation, observation, programme); a companion script with 42 computational checks.
Three questions
Naturality. A layer $T$ on signals over a domain is natural for a map $f\colon D\to D'$ (a symmetry, a covering, a refinement of a grid) when
$$\begin{CD} F(D') @>{T_{D'}}>> F(D') \\ @V{f^*}VV @VV{f^*}V \\ F(D) @>>{T_D}> F(D) \end{CD}$$
commutes; the defect is the size of the difference. For symmetries this is equivariance, for refinements discretization invariance, for coverings the method of images.
Well-posedness. Does the output depend continuously on the input (Hadamard)?
Rates. At what rate does the construction act on each mode? For linear constructions the answer is a spectrum.
Results: qualitative statement, then its quantitative form
| Topic | Qualitative | Quantitative |
|---|---|---|
| Reflect padding | half-sample reflection models the Neumann condition. Circular layers preserve the reflected subspace exactly when their periodized kernels are symmetric | exact for symmetric kernels; anti-invariant part $1.18$ and two-layer defect $0.100$ for an asymmetric one |
| Zero padding | the Dirichlet condition only for three-point kernels; breaks translation only near the boundary; leaks mass | defect $0.160$ at radius two; $0.532$ of the mass left after $200$ smoothing layers on $32$ points |
| Resolution | Fourier multipliers commute with resampling; stencils to second order; squaring iff more than $3K$ points; rectifiers never | stencil symbol $0.717$ and $0.980$ of $k^2$ on $16$ and $64$ points; sampled mean of $\mathrm{relu}(\sin x)$ is $\cot(\pi/N)/N$, error $-\pi/3N^2$ |
| Diffusion models | reversing the heat equation is ill-posed for densities and well-posed for samples given the score | mode $10$ amplified by $5.2\times10^{21}$; samples move $0.264$ per unit of score error; score slope $-1/t+a^2/t^2$, i.e. $90$ and $9900$ at $t=0.1,0.01$ |
| Score matching, entropy | denoising equals explicit score matching plus a constant; de Bruijn; contraction | constant $1/t-J=0.924$; rate exactly $2$, with $e^{2t}D(p_t\Vert\gamma)\to\tfrac12(\mathbb E X_0)^2$ |
| Flow matching | on the line, the flow is the monotone transport map for every coupling; commitment to a mode is a pitchfork bifurcation in time | error $10^{-13}$; $t^*=(3-\sqrt5)/2=0.382$; slope $890$ at $t=0.9$ |
| Message passing | oversmoothing is the spectral gap | factor $1-\tau\lambda_2=0.92437$ measured and predicted; depth scales $415$, $5$, $1$ for a path, a hypercube, a complete graph on $32$ nodes |
| Wave-type layers | conserve an energy, do not smooth; stable under a CFL bound | drift $2\times10^{-15}$ over $500$ layers; growth $1.8633$ beyond the bound |
| Sheaf diffusion | converges to the global sections; obstruction in $H^1(G;\mathbb Z/2)$ | $16$ of $128$ sign sheaves on a graph with $5$ nodes and $7$ edges have a section; gaps $0.1606$, $0.19806$ when frustrated |
| Residual layers | forward Euler amplifies nonzero oscillatory modes. Leapfrog has bounded powers and condition numbers when $0<h\omega<2$ | $1.6446$ and $1.0513$ over time $10$; condition at most $1.667$ against $8.7\times10^{13}$ |
| Kernel regression | low modes first; the sampled spectrum is a Poisson sum over aliases | steps to one per cent: $6.6$, $282$, $956$ for $k=1,8,16$ |
| Physics-informed loss | squares the condition number of the energy loss and reverses the spectral bias | $414$ against $171{,}682$ on $31$ unknowns; rate ratios $1/104$ and $414$ |
| Noise | the spectral gap is the curvature in a quadratic well and approaches the Kramers rate at low noise in a double well | $0.9997$; gap over Kramers $0.903, 0.928, 0.947, 0.959$ at $\beta=4,\dots,10$ |
| Control | the Bellman operator is a monotone contraction; a monotone scheme selects the viscosity solution | ratio at most $0.7072$ at $\gamma=0.9$ |
| Operator rank | algebraic for elliptic, exponential for parabolic, none for hyperbolic | $1000$, $11$, all modes at relative accuracy $10^{-6}$; translated steps: $14\%$ error with ten modes |
Three points the organization makes visible
Padding and reflection symmetry. With symmetric kernels, half-sample reflect padding restricts the doubled circular model to signals fixed by a reflection. The odd extension used for Dirichlet data generally differs from wide zero padding, and a rectifier can destroy odd symmetry.
Diffusion models move the ill-posedness; they do not remove it. The backward heat equation is replaced by a transport equation that is well-posed once the score is known. The whole difficulty is then in the score, at small times and between modes, where its slope grows like $1/t^2$.
Heat forgets, waves do not. Oversmoothing, vanishing gradients and mixing are properties of Markov semigroups; conserved energy, bounded gradients and lack of smoothing are properties of symplectic groups. An architecture chooses one, and inherits both the benefit and the cost.
What is and is not claimed
No new theorem. The identifications are from the literature and are cited. Three statements are tagged as observations (zero padding as Dirichlet only for short kernels; flow matching on the line; the bifurcation time for two-point data); they are elementary and may be known in other words. Most models are linear and one-dimensional so that every number is exact or accurate to many digits. Nothing is proved about the training dynamics of a finite network; the score is assumed given; stochastic gradient descent is modelled by isotropic Langevin dynamics, a rough approximation. No experiment on a trained model was run.
Files
partial-differential-equations-in-machine-learning-and-ai.pdf— the paper (11 pages, two columns).partial-differential-equations-in-machine-learning-and-ai.tex— complete LaTeX source as a single file (pdflatex, three passes).verify-pde-learning.py— 42 checks, Python standard library only, about one second; exits 0 if and only if all pass.
Status and authorship
Contribution. Organization of known results around three questions, with exact constants, a small number of observations, and reproducible computations.
Limitations. As listed above; in addition, the bibliography was assembled from memory except for three entries whose details were checked online on 17 November 2026 (for one of them only the authors' surnames were recorded), and nothing has been independently refereed. The script's checks are evidence that the numbers are stated correctly and are not proofs. Corrections are welcome.
AI assistance. The text was drafted and typeset with a large language model (Anthropic's Claude) under the direction of the submitting author, who is responsible for it. Generative AI assisted with drafting, literature recall, mathematical consistency checks, computation and typesetting.