Independent scholarship · Open accessThe monthly digest

Indexed text attachment

partial-differential-equations-in-machine-learning-and-ai.tex

Source for Partial Differential Equations in Machine Learning and AI: Naturality, Well-Posedness, and Spectral Rates

View paper · Download original file

Lines 121–220 of 1085. Source text is displayed literally.

121 122\emph{Rates.} For linear constructions the answer to ``how fast'' is a spectrum. Oversmoothing, spectral bias,123mixing of stochastic gradient descent and the rank of a learned operator are all statements about eigenvalues124of a Laplacian.125 126\paragraph{What is and is not claimed.} Every statement below carries a status tag: classical, literature,127reformulation, observation or programme. Almost all are classical or known; where the form or the constant128seems not to be standard we say ``observation''. The checks, tagged like \chk{N3}, are performed by the companion129script \texttt{verify-pde-learning.py}, which uses only the Python standard library and a fixed seed, and are130listed in \cref{app:checks}. They recompute instances; they are evidence and not proofs. Most models are linear131and one-dimensional, so that every number is exact or accurate to many digits; \cref{sec:limits} says what this132leaves out. The references were assembled from memory, except for \cite{DCWJXKY26,BBRMMA23,BDCLB22}, which were133checked online on 17 November 2026.134 135\section{Setting}\label{sec:setting}136 137Let $\cat{Dom}$ be a category of domains: intervals, circles, grids, graphs, with the maps of interest between138them (inclusions, coverings, symmetries, refinements). A \term{signal space} is a functor139$F\colon\cat{Dom}^{\mathrm{op}}\to\cat{Vect}$, such as functions on the domain with restriction or pullback. A140\term{layer} is a family of maps $T_D\colon F(D)\to F(D)$, not necessarily linear.141 142\begin{definition}\label{def:natural}143The layer $T$ is \term{natural} for a map $f\colon D\to D'$ if the square144\[\begin{tikzcd}[column sep=large]145F(D')\arrow[r,"T_{D'}"]\arrow[d,"f^*"'] & F(D')\arrow[d,"f^*"]\\146F(D)\arrow[r,"T_D"'] & F(D)147\end{tikzcd}\]148commutes. Its \term{defect} on an input $x$ is $\lVert f^*T_{D'}x-T_Df^*x\rVert$, relative to the norm of either149term.150\end{definition}151 152\noindent For $f$ a symmetry of $D$ this is equivariance. For $f$ the inclusion of a coarse grid in a fine one,153with $F$ restricted to band-limited signals, it is discretization invariance. For $f$ a covering map and $F$154replaced by its invariant part, it is the method of images, with which we begin. A \term{depth semigroup} is a155family $T^t$ with $T^{t+s}=T^tT^s$, and a network of $N$ layers of step $h$ is a discretization of $T^{Nh}$;156naturality in $t$ is consistency of a scheme, and well-posedness is its stability \cite[Ch.~12]{PDEHC}.157 158\section{Padding is a boundary condition}\label{sec:padding}159 160Let $x\in\R^n$ and let $w=(w_{-r},\dots,w_r)$ be a kernel. The layer is $(C_wx)_i=\sum_jw_jx_{i+j}$, and a161\term{padding} rule supplies the values $x_i$ for $i$ outside $0,\dots,n-1$: \emph{zero} ($x_i=0$),162\emph{circular} ($x_{i+n}=x_i$) or \emph{reflect} ($x_{-1-i}=x_i$ and $x_{n+i}=x_{n-1-i}$).163 164\begin{proposition}\label{prop:laplacians}165\Scl[\cite{Str99}] With the kernel $(-1,2,-1)$ the three rules give the Dirichlet, periodic and Neumann166difference Laplacians. They are diagonalized by the sine transform $\sin\frac{\pi k(j+1)}{n+1}$, the discrete167Fourier transform, and the cosine transform $\cos\frac{\pi k(j+1/2)}n$, with eigenvalues168$4\sin^2\frac{\pi k}{2(n+1)}$, $4\sin^2\frac{\pi k}n$ and $4\sin^2\frac{\pi k}{2n}$. \chk{N1, N2}169\end{proposition}170 171\begin{figure}[t]172\centering173\begin{tikzpicture}174\fill[fwd!8] (-0.2,-0.72) rectangle (2.2,0.78);175\draw[ink2] (-3.04,0)--(5.04,0);176\draw[accent,densely dashed] (-0.2,-0.72)--(-0.2,0.78);177\draw[accent,densely dashed] (2.2,-0.72)--(2.2,0.78);178\draw[thick,bwd] (-2.8,0)--(-2.8,0.66);179\fill[bwd] (-2.8,0.66) circle (1.3pt);180\draw[thick,bwd] (-2.4,0)--(-2.4,0.66);181\fill[bwd] (-2.4,0.66) circle (1.3pt);182\draw[thick,bwd] (-2,0)--(-2,0.363);183\fill[bwd] (-2,0.363) circle (1.3pt);184\draw[thick,bwd] (-1.6,0)--(-1.6,0.198);185\fill[bwd] (-1.6,0.198) circle (1.3pt);186\draw[thick,bwd] (-1.2,0)--(-1.2,0.462);187\fill[bwd] (-1.2,0.462) circle (1.3pt);188\draw[thick,bwd] (-0.8,0)--(-0.8,0.594);189\fill[bwd] (-0.8,0.594) circle (1.3pt);190\draw[thick,bwd] (-0.4,0)--(-0.4,0.33);191\fill[bwd] (-0.4,0.33) circle (1.3pt);192\draw[thick,fwd] (0,0)--(0,0.33);193\fill[fwd] (0,0.33) circle (1.3pt);194\draw[thick,fwd] (0.4,0)--(0.4,0.594);195\fill[fwd] (0.4,0.594) circle (1.3pt);196\draw[thick,fwd] (0.8,0)--(0.8,0.462);197\fill[fwd] (0.8,0.462) circle (1.3pt);198\draw[thick,fwd] (1.2,0)--(1.2,0.198);199\fill[fwd] (1.2,0.198) circle (1.3pt);200\draw[thick,fwd] (1.6,0)--(1.6,0.363);201\fill[fwd] (1.6,0.363) circle (1.3pt);202\draw[thick,fwd] (2,0)--(2,0.66);203\fill[fwd] (2,0.66) circle (1.3pt);204\draw[thick,bwd] (2.4,0)--(2.4,0.66);205\fill[bwd] (2.4,0.66) circle (1.3pt);206\draw[thick,bwd] (2.8,0)--(2.8,0.363);207\fill[bwd] (2.8,0.363) circle (1.3pt);208\draw[thick,bwd] (3.2,0)--(3.2,0.198);209\fill[bwd] (3.2,0.198) circle (1.3pt);210\draw[thick,bwd] (3.6,0)--(3.6,0.462);211\fill[bwd] (3.6,0.462) circle (1.3pt);212\draw[thick,bwd] (4,0)--(4,0.594);213\fill[bwd] (4,0.594) circle (1.3pt);214\draw[thick,bwd] (4.4,0)--(4.4,0.33);215\fill[bwd] (4.4,0.33) circle (1.3pt);216\draw[thick,bwd] (4.8,0)--(4.8,0.33);217\fill[bwd] (4.8,0.33) circle (1.3pt);218\node[anchor=west] at (-3.04,1.02) {\scriptsize reflect: even extension};219\fill[fwd!8] (-0.2,-2.72) rectangle (2.2,-1.22);220\draw[ink2] (-3.04,-2)--(5.04,-2);