Independent scholarship · Open accessThe monthly digest

Indexed text attachment

partial-differential-equations-in-machine-learning-and-ai.tex

Source for Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates

View paper · Download original file

Lines 61–160 of 1085. Source text is displayed literally.

61\setlength{\parskip}{0pt}62\frenchspacing63\raggedbottom64\title{\textbf{Partial Differential Equations in Machine Learning and AI:\\ Naturality, Well-Posedness, and Spectral Rates}}65\author{Anonymous}66\date{Higher Categorical Semantics \quad$\cdot$\quad Version 1.0, 18 November 2026}67\begin{document}68\twocolumn[69\maketitle70\begin{center}\begin{minipage}{0.86\textwidth}\small71\textbf{Abstract.} Convolutional layers are difference operators, depth is a time variable, generative diffusion is72the heat equation, message passing is a graph Laplacian, and a kernel is a Green's function. We organize these73identifications by three questions that the theory of partial differential equations asks of any construction.74\emph{Is it natural}, that is, does it commute with the maps of domains, resolutions and symmetries along which75one wants to move it? \emph{Is it well-posed}, that is, does the output depend continuously on the input?76\emph{At what rate} does it act on each mode? Each qualitative answer is paired with a computed one. Naturality:77half-sample reflection models the Neumann condition. Circular layers preserve the reflected subspace exactly78when the periodized kernels are symmetric.79zero padding is the Dirichlet condition only for three-point stencils; Fourier multipliers commute with80resampling, stencils do so to second order, squaring needs more than $3K$ points for band $K$, and a rectifier81is aliased at order $N^{-2}$ on every grid. Well-posedness: reversing diffusion amplifies mode $10$ by82$5\times10^{21}$, while the reverse flow driven by the score moves samples by $0.26$ per unit of score error, the83difficulty reappearing as a score of slope $1/t^2$. Forward Euler amplifies nonzero oscillatory modes. In the84stable range $0<h\omega<2$, the leapfrog condition number is at most $(1+h\omega/2)/(1-h\omega/2)$. A strong-form loss squares the85condition number of the energy loss, $414$ against $171{,}682$ on $31$ unknowns. Rates: oversmoothing proceeds86at $1-\tau\lambda_2$ per layer, with depth scales $415$, $5$ and $1$ on a path, a hypercube and a complete graph87of $32$ nodes; sheaf diffusion converges to global sections, which exist for $16$ of the $128$ sign sheaves on a88small graph; kernel regression learns mode $k$ at rate $1/(1+k^2)$ plus aliases; and the spectral gap of89Langevin dynamics in a double well approaches the Kramers rate at low noise. All numerical statements are recomputed by a companion90script with $42$ checks. No new theorem is claimed; the contribution is the organization, the exact constants,91and the three questions asked uniformly.92\end{minipage}\end{center}93\vspace{1.2em}94]95 96\section{Introduction}97 98The companion volume \cite{PDEHC} retells an introductory course on partial differential equations99\cite{Str08} around one question: which constructions are natural for maps of domains, of equations and of100function spaces. This paper asks the same question of machine learning, where the same equations have been101rediscovered piecemeal. A convolutional layer with a three-point kernel is a difference operator, and its102padding rule is a boundary condition \cite{Str99}. A residual network is a difference scheme in a time variable103called depth \cite{E17,HaRu17,CRBD18}. A diffusion model noises data with the heat equation and samples by104reversing it \cite{SWMG15,HJA20,SSKKEP21}. Message passing on a graph is a discrete diffusion105\cite{CRGBB21}, or a discrete wave equation \cite{RCRMB22}, or a diffusion of sections of a sheaf106\cite{HaGh19,BDCLB22}. Kernel regression inverts an elliptic operator, physics-informed networks minimize the107residual of an equation \cite{RPK19}, and operator learning approximates its solution map108\cite{LKALBSA21,KLLABSA23,LJPZK21}. A recent survey organizes the last two families \cite{DCWJXKY26}.109 110Three questions organize the theory of the equations themselves, and we ask each of them of each construction.111 112\emph{Naturality.} A domain can be reflected, translated, refined or glued, and a construction is natural if it113commutes with the corresponding maps. Equivariance \cite{CoWe16,BBCV21} is naturality for symmetries;114discretization invariance and the absence of aliasing \cite{KALHHLA21,BBRMMA23} are naturality for changes of115resolution.116 117\emph{Well-posedness.} Hadamard's criterion asks that a solution exist, be unique, and depend continuously on118the data. Learning problems fail the third condition in characteristic ways: by running a parabolic equation119backwards, by discretizing a conservative system with a dissipative scheme or the reverse, or by squaring an120operator in the loss.121 122\emph{Rates.} For linear constructions the answer to ``how fast'' is a spectrum. Oversmoothing, spectral bias,123mixing of stochastic gradient descent and the rank of a learned operator are all statements about eigenvalues124of a Laplacian.125 126\paragraph{What is and is not claimed.} Every statement below carries a status tag: classical, literature,127reformulation, observation or programme. Almost all are classical or known; where the form or the constant128seems not to be standard we say ``observation''. The checks, tagged like \chk{N3}, are performed by the companion129script \texttt{verify-pde-learning.py}, which uses only the Python standard library and a fixed seed, and are130listed in \cref{app:checks}. They recompute instances; they are evidence and not proofs. Most models are linear131and one-dimensional, so that every number is exact or accurate to many digits; \cref{sec:limits} says what this132leaves out. The references were assembled from memory, except for \cite{DCWJXKY26,BBRMMA23,BDCLB22}, which were133checked online on 17 November 2026.134 135\section{Setting}\label{sec:setting}136 137Let $\cat{Dom}$ be a category of domains: intervals, circles, grids, graphs, with the maps of interest between138them (inclusions, coverings, symmetries, refinements). A \term{signal space} is a functor139$F\colon\cat{Dom}^{\mathrm{op}}\to\cat{Vect}$, such as functions on the domain with restriction or pullback. A140\term{layer} is a family of maps $T_D\colon F(D)\to F(D)$, not necessarily linear.141 142\begin{definition}\label{def:natural}143The layer $T$ is \term{natural} for a map $f\colon D\to D'$ if the square144\[\begin{tikzcd}[column sep=large]145F(D')\arrow[r,"T_{D'}"]\arrow[d,"f^*"'] & F(D')\arrow[d,"f^*"]\\146F(D)\arrow[r,"T_D"'] & F(D)147\end{tikzcd}\]148commutes. Its \term{defect} on an input $x$ is $\lVert f^*T_{D'}x-T_Df^*x\rVert$, relative to the norm of either149term.150\end{definition}151 152\noindent For $f$ a symmetry of $D$ this is equivariance. For $f$ the inclusion of a coarse grid in a fine one,153with $F$ restricted to band-limited signals, it is discretization invariance. For $f$ a covering map and $F$154replaced by its invariant part, it is the method of images, with which we begin. A \term{depth semigroup} is a155family $T^t$ with $T^{t+s}=T^tT^s$, and a network of $N$ layers of step $h$ is a discretization of $T^{Nh}$;156naturality in $t$ is consistency of a scheme, and well-posedness is its stability \cite[Ch.~12]{PDEHC}.157 158\section{Padding is a boundary condition}\label{sec:padding}159 160Let $x\in\R^n$ and let $w=(w_{-r},\dots,w_r)$ be a kernel. The layer is $(C_wx)_i=\sum_jw_jx_{i+j}$, and a