Independent scholarship · Open accessThe monthly digest

Editorially reviewed

Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates

Three questions from PDE theory asked of machine learning: is a construction natural, is it well-posed, at what rate does it act? Padding as a boundary condition, aliasing, diffusion models and the score, flow matching, oversmoothing and sheaf diffusion, residual layers as schemes, physics-informed losses, Langevin noise, operator rank; each with a constant. Two-column paper, 11 pp., 42 checks.

Anonymous← All research

Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates

Padding as a boundary condition, depth as time, the score as the datum that makes reversal well-posed

Convolutional layers are difference operators, depth is a time variable, generative diffusion is the heat equation, message passing is a graph Laplacian, and a kernel is a Green's function. This paper organizes those identifications by the three questions that the theory of partial differential equations asks of any construction, and attaches a computed constant to each answer. It is the application paper to the companion volume Partial Differential Equations in Higher Categories.

Format. Double-column formal paper, 11 pages, 7 figures, 46 references; status tags on every numbered statement (classical, literature, reformulation, observation, programme); a companion script with 42 computational checks.

Three questions

Naturality. A layer $T$ on signals over a domain is natural for a map $f\colon D\to D'$ (a symmetry, a covering, a refinement of a grid) when

$$\begin{CD} F(D') @>{T_{D'}}>> F(D') \\ @V{f^*}VV @VV{f^*}V \\ F(D) @>>{T_D}> F(D) \end{CD}$$

commutes; the defect is the size of the difference. For symmetries this is equivariance, for refinements discretization invariance, for coverings the method of images.

Well-posedness. Does the output depend continuously on the input (Hadamard)?

Rates. At what rate does the construction act on each mode? For linear constructions the answer is a spectrum.

Results: qualitative statement, then its quantitative form

TopicQualitativeQuantitative
Reflect paddinghalf-sample reflection models the Neumann condition. Circular layers preserve the reflected subspace exactly when their periodized kernels are symmetricexact for symmetric kernels; anti-invariant part $1.18$ and two-layer defect $0.100$ for an asymmetric one
Zero paddingthe Dirichlet condition only for three-point kernels; breaks translation only near the boundary; leaks massdefect $0.160$ at radius two; $0.532$ of the mass left after $200$ smoothing layers on $32$ points
ResolutionFourier multipliers commute with resampling; stencils to second order; squaring iff more than $3K$ points; rectifiers neverstencil symbol $0.717$ and $0.980$ of $k^2$ on $16$ and $64$ points; sampled mean of $\mathrm{relu}(\sin x)$ is $\cot(\pi/N)/N$, error $-\pi/3N^2$
Diffusion modelsreversing the heat equation is ill-posed for densities and well-posed for samples given the scoremode $10$ amplified by $5.2\times10^{21}$; samples move $0.264$ per unit of score error; score slope $-1/t+a^2/t^2$, i.e. $90$ and $9900$ at $t=0.1,0.01$
Score matching, entropydenoising equals explicit score matching plus a constant; de Bruijn; contractionconstant $1/t-J=0.924$; rate exactly $2$, with $e^{2t}D(p_t\Vert\gamma)\to\tfrac12(\mathbb E X_0)^2$
Flow matchingon the line, the flow is the monotone transport map for every coupling; commitment to a mode is a pitchfork bifurcation in timeerror $10^{-13}$; $t^*=(3-\sqrt5)/2=0.382$; slope $890$ at $t=0.9$
Message passingoversmoothing is the spectral gapfactor $1-\tau\lambda_2=0.92437$ measured and predicted; depth scales $415$, $5$, $1$ for a path, a hypercube, a complete graph on $32$ nodes
Wave-type layersconserve an energy, do not smooth; stable under a CFL bounddrift $2\times10^{-15}$ over $500$ layers; growth $1.8633$ beyond the bound
Sheaf diffusionconverges to the global sections; obstruction in $H^1(G;\mathbb Z/2)$$16$ of $128$ sign sheaves on a graph with $5$ nodes and $7$ edges have a section; gaps $0.1606$, $0.19806$ when frustrated
Residual layersforward Euler amplifies nonzero oscillatory modes. Leapfrog has bounded powers and condition numbers when $0<h\omega<2$$1.6446$ and $1.0513$ over time $10$; condition at most $1.667$ against $8.7\times10^{13}$
Kernel regressionlow modes first; the sampled spectrum is a Poisson sum over aliasessteps to one per cent: $6.6$, $282$, $956$ for $k=1,8,16$
Physics-informed losssquares the condition number of the energy loss and reverses the spectral bias$414$ against $171{,}682$ on $31$ unknowns; rate ratios $1/104$ and $414$
Noisethe spectral gap is the curvature in a quadratic well and approaches the Kramers rate at low noise in a double well$0.9997$; gap over Kramers $0.903, 0.928, 0.947, 0.959$ at $\beta=4,\dots,10$
Controlthe Bellman operator is a monotone contraction; a monotone scheme selects the viscosity solutionratio at most $0.7072$ at $\gamma=0.9$
Operator rankalgebraic for elliptic, exponential for parabolic, none for hyperbolic$1000$, $11$, all modes at relative accuracy $10^{-6}$; translated steps: $14\%$ error with ten modes

Three points the organization makes visible

Padding and reflection symmetry. With symmetric kernels, half-sample reflect padding restricts the doubled circular model to signals fixed by a reflection. The odd extension used for Dirichlet data generally differs from wide zero padding, and a rectifier can destroy odd symmetry.

Diffusion models move the ill-posedness; they do not remove it. The backward heat equation is replaced by a transport equation that is well-posed once the score is known. The whole difficulty is then in the score, at small times and between modes, where its slope grows like $1/t^2$.

Heat forgets, waves do not. Oversmoothing, vanishing gradients and mixing are properties of Markov semigroups; conserved energy, bounded gradients and lack of smoothing are properties of symplectic groups. An architecture chooses one, and inherits both the benefit and the cost.

What is and is not claimed

No new theorem. The identifications are from the literature and are cited. Three statements are tagged as observations (zero padding as Dirichlet only for short kernels; flow matching on the line; the bifurcation time for two-point data); they are elementary and may be known in other words. Most models are linear and one-dimensional so that every number is exact or accurate to many digits. Nothing is proved about the training dynamics of a finite network; the score is assumed given; stochastic gradient descent is modelled by isotropic Langevin dynamics, a rough approximation. No experiment on a trained model was run.

Files

Status and authorship

Contribution. Organization of known results around three questions, with exact constants, a small number of observations, and reproducible computations.

Limitations. As listed above; in addition, the bibliography was assembled from memory except for three entries whose details were checked online on 17 November 2026 (for one of them only the authors' surnames were recorded), and nothing has been independently refereed. The script's checks are evidence that the numbers are stated correctly and are not proofs. Corrections are welcome.

AI assistance. The text was drafted and typeset with a large language model (Anthropic's Claude) under the direction of the submitting author, who is responsible for it. Generative AI assisted with drafting, literature recall, mathematical consistency checks, computation and typesetting.

Reproducibility artifacts

partial-differential-equations-in-machine-learning-and-ai.pdf · 300,370 bytes · Download
SHA-256 03408eba55b78b6ca3a9e7e0d25bc5cae491242d11251ec4b36184d29249cc09

partial-differential-equations-in-machine-learning-and-ai.tex · 117,704 bytes · Read in browser · Download
Text indexing selected · View indexed source
SHA-256 8d57190061d6f500c0d4c1982b46b9e35b74a73579dc453c1baac8c510bf4724

verify-pde-learning.py · 46,089 bytes · Download
Text indexing selected · View indexed source
SHA-256 377a04d0fdd47fa07dab0db2c76f3ef9fc996eef7d9ff7126345be05f3ea8b5a

Discussion

Open a careful, constructive conversation about this note.

Comments appear after editorial approval.