Indexed text attachment
partial-differential-equations-in-machine-learning-and-ai.tex
Source for Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates
View paper · Download original file
Lines 21–120 of 1085. Source text is displayed literally.
21\usetikzlibrary{arrows.meta,positioning}22\usepackage{booktabs,array}23\usepackage{enumitem}24\usepackage[font=small,labelfont={bf,sf}]{caption}25\usepackage[colorlinks=true,linkcolor=accent,citecolor=accent,urlcolor=accent,26 pdftitle={Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates},27 pdfauthor={Anonymous},28 pdfsubject={Partial differential equations as a framework for convolution, generative diffusion, graph networks and learning dynamics},29 pdfkeywords={partial differential equations, machine learning, boundary conditions, generative diffusion, graph networks, numerical stability, spectral rates}]{hyperref}30\usepackage[nameinlink,capitalise,noabbrev]{cleveref}31\theoremstyle{plain}32\newtheorem{theorem}{Theorem}[section]33\newtheorem{proposition}[theorem]{Proposition}34\newtheorem{corollary}[theorem]{Corollary}35\newtheorem{lemma}[theorem]{Lemma}36\theoremstyle{definition}37\newtheorem{definition}[theorem]{Definition}38\newtheorem{example}[theorem]{Example}39\theoremstyle{remark}40\newtheorem{remark}[theorem]{Remark}41\newcommand{\stag}[2]{{\footnotesize\sffamily\upshape\color{#1}[#2]}\enspace\ignorespaces}42\newcommand{\Scl}[1][]{\stag{ink2}{classical\if\relax\detokenize{#1}\relax\else; #1\fi}}43\newcommand{\Sli}[1][]{\stag{fwd}{literature\if\relax\detokenize{#1}\relax\else; #1\fi}}44\newcommand{\Srf}[1][]{\stag{accent}{reformulation\if\relax\detokenize{#1}\relax\else; cf.\ #1\fi}}45\newcommand{\Sob}{\stag{accent}{observation}}46\newcommand{\Spg}{\stag{bwd}{programme}}47\newcommand{\chk}[1]{{\footnotesize\sffamily\upshape\color{accent}[#1]}}48\newcommand{\cat}[1]{\mathsf{#1}}49\newcommand{\R}{\mathbb{R}}50\newcommand{\Z}{\mathbb{Z}}51\newcommand{\E}{\mathbb{E}}52\newcommand{\one}{\mathbf{1}}53\newcommand{\T}{\mathsf{T}}54\newcommand{\Lap}{\Delta}55\newcommand{\pd}{\partial}56\newcommand{\KL}[2]{D(#1\,\Vert\,#2)}57\newcommand{\Var}{\operatorname{Var}}58\newcommand{\relu}{\operatorname{relu}}59\newcommand{\sinc}{\operatorname{sinc}}60\newcommand{\term}[1]{\textbf{\itshape #1}}61\setlength{\parskip}{0pt}62\frenchspacing63\raggedbottom64\title{\textbf{Partial Differential Equations in Machine Learning and AI:\\ Naturality, Well-Posedness, and Spectral Rates}}65\author{Anonymous}66\date{Higher Categorical Semantics \quad$\cdot$\quad Version 1.0, 18 November 2026}67\begin{document}68\twocolumn[69\maketitle70\begin{center}\begin{minipage}{0.86\textwidth}\small71\textbf{Abstract.} Convolutional layers are difference operators, depth is a time variable, generative diffusion is72the heat equation, message passing is a graph Laplacian, and a kernel is a Green's function. We organize these73identifications by three questions that the theory of partial differential equations asks of any construction.74\emph{Is it natural}, that is, does it commute with the maps of domains, resolutions and symmetries along which75one wants to move it? \emph{Is it well-posed}, that is, does the output depend continuously on the input?76\emph{At what rate} does it act on each mode? Each qualitative answer is paired with a computed one. Naturality:77half-sample reflection models the Neumann condition. Circular layers preserve the reflected subspace exactly78when the periodized kernels are symmetric.79zero padding is the Dirichlet condition only for three-point stencils; Fourier multipliers commute with80resampling, stencils do so to second order, squaring needs more than $3K$ points for band $K$, and a rectifier81is aliased at order $N^{-2}$ on every grid. Well-posedness: reversing diffusion amplifies mode $10$ by82$5\times10^{21}$, while the reverse flow driven by the score moves samples by $0.26$ per unit of score error, the83difficulty reappearing as a score of slope $1/t^2$. Forward Euler amplifies nonzero oscillatory modes. In the84stable range $0<h\omega<2$, the leapfrog condition number is at most $(1+h\omega/2)/(1-h\omega/2)$. A strong-form loss squares the85condition number of the energy loss, $414$ against $171{,}682$ on $31$ unknowns. Rates: oversmoothing proceeds86at $1-\tau\lambda_2$ per layer, with depth scales $415$, $5$ and $1$ on a path, a hypercube and a complete graph87of $32$ nodes; sheaf diffusion converges to global sections, which exist for $16$ of the $128$ sign sheaves on a88small graph; kernel regression learns mode $k$ at rate $1/(1+k^2)$ plus aliases; and the spectral gap of89Langevin dynamics in a double well approaches the Kramers rate at low noise. All numerical statements are recomputed by a companion90script with $42$ checks. No new theorem is claimed; the contribution is the organization, the exact constants,91and the three questions asked uniformly.92\end{minipage}\end{center}93\vspace{1.2em}94]95 96\section{Introduction}97 98The companion volume \cite{PDEHC} retells an introductory course on partial differential equations99\cite{Str08} around one question: which constructions are natural for maps of domains, of equations and of100function spaces. This paper asks the same question of machine learning, where the same equations have been101rediscovered piecemeal. A convolutional layer with a three-point kernel is a difference operator, and its102padding rule is a boundary condition \cite{Str99}. A residual network is a difference scheme in a time variable103called depth \cite{E17,HaRu17,CRBD18}. A diffusion model noises data with the heat equation and samples by104reversing it \cite{SWMG15,HJA20,SSKKEP21}. Message passing on a graph is a discrete diffusion105\cite{CRGBB21}, or a discrete wave equation \cite{RCRMB22}, or a diffusion of sections of a sheaf106\cite{HaGh19,BDCLB22}. Kernel regression inverts an elliptic operator, physics-informed networks minimize the107residual of an equation \cite{RPK19}, and operator learning approximates its solution map108\cite{LKALBSA21,KLLABSA23,LJPZK21}. A recent survey organizes the last two families \cite{DCWJXKY26}.109 110Three questions organize the theory of the equations themselves, and we ask each of them of each construction.111 112\emph{Naturality.} A domain can be reflected, translated, refined or glued, and a construction is natural if it113commutes with the corresponding maps. Equivariance \cite{CoWe16,BBCV21} is naturality for symmetries;114discretization invariance and the absence of aliasing \cite{KALHHLA21,BBRMMA23} are naturality for changes of115resolution.116 117\emph{Well-posedness.} Hadamard's criterion asks that a solution exist, be unique, and depend continuously on118the data. Learning problems fail the third condition in characteristic ways: by running a parabolic equation119backwards, by discretizing a conservative system with a dissipative scheme or the reverse, or by squaring an120operator in the loss.