Overview
We develop a finite, introductory account of learning and AI systems in which the values, comparisons, and placement of a computation are specified explicitly. The accompanying double-column manuscript contains the complete definitions, proofs, illustrations, disclosure, and references. Executable artifacts include a checked training-plan compiler, distributed value iteration, checks of fixed-version replica descent, and sequence-sharded attention.
Models, comparisons, and gradients
A parametric model $X\to Y$ consists of $P$ and a smooth function $f:P\times X\to Y$. For $g:Q\times Y\to Z$, the composite acts by $((q,p),x)\mapsto g(q,f(p,x))$. A 2-morphism from $(P,f)$ to $(P',f')$ is a smooth $r:P\to P'$ with $f=f'\circ(r\times\operatorname{id})$. Vertical composition uses function composition and horizontal composition uses products of parameter maps. The product associator and unitors are coherent because both routes in each coherence diagram select the same projections. The 2-morphism $t\mapsto(t,t)$ ties the weight and bias of $wx+b$ and pulls their cotangents back to their sum. Euclidean optimizer steps can still depend on parameterization.
For a smooth map $f$, the backward map $R[f](x,\bar y)=Df(x)^{\mathsf T}\bar y$ is a cotangent pullback. The reverse chain rule is exactly the lens composition law. Product maps have block-diagonal differential, making the assignment to forward and backward lenses strong symmetric monoidal in the Euclidean setting. The manuscript works through affine regression and a two-class cross-entropy whose logit gradient is $(1/2,-1/2)$.
Synchronous training and compiler correctness
Let $I$ index a nonempty finite batch and suppose all workers start at the same parameter $p$ and optimizer state. Each differentiable per-record loss contributes
$$ \phi_p(i)=\bigl(\nabla L_i(p),L_i(p),1\bigr). $$
The compiler checks that the layout partitions $I$. Each worker computes the sum of these contributions for its own shard. A collective gives the triple of global gradient sum, loss sum, and count. In exact arithmetic these totals are independent of grouping because their operation is a commutative monoid. The global count is positive, and linearity of differentiation shows that the normalized gradient equals the gradient of the serial mean loss. With the same deterministic optimizer transition, the serial and distributed parameter and state agree after a step. Induction gives agreement across steps with the same common inputs.
At $p=(0,0)$ on records $(i,i)$ for $i=0,1,2,3$, shard sizes $1$ and $3$ yield global gradient sum $(-14,-6)$, loss sum $7$, and count $4$. With step size $1/4$, both parameter replicas become $(7/8,3/8)$. The mean loss moves from $7/4$ to $7/256$. Weighting the two local means equally changes the objective. Batch-dependent state and floating-point grouping require additional contracts.
The executable compiler lowers a declarative JSON file to a typed plan with a broadcast, local reverse pass, global sums, and normalized update. The archive runs this plan with Python worker processes, PyTorch autograd on independent CPU processes, and a JAX mesh with explicit collectives. The JAX runner exports the lowered StableHLO-based module. Reproduction commands cover unequal partitions and a three-step run with an empty worker. A Gloo target is supplied. Host interface permissions blocked its initialization during verification.
Probabilistic AI and attention
Finite stochastic kernels compose by summing over their intermediate index. Copying the input of a signal channel retains the source coordinate in its joint law. The worked signal model has posterior probabilities $P(H\mid+)=4/7$ and $P(H\mid-)=1/13$. For a finite discounted decision problem, a synchronous Bellman step computes action values in parallel and takes their statewise maximum. The contraction factor is $\gamma$ in the supremum norm. The example uses two worker processes and changes the maximizing action at the initial state when the delayed value propagates.
For one attention row with a nonempty set of eligible keys, a shard holds a score maximum $m$, the centered total $\ell=\sum_j e^{s_j-m}$, and the centered weighted value sum $u=\sum_j e^{s_j-m}v_j$. To combine two nonempty summaries, take the larger maximum and rescale both pairs $(\ell,u)$ to that common maximum before adding. The change of variables $(m,\ell,u)\mapsto(m,e^m\ell,e^mu)$ identifies this operation with maximum and addition, so it is commutative and associative over the reals. An empty summary acts as the unit. The quotient $u/\ell$ returns the dense attention row output. For scores $\log 2,\log 3,\log5$ and values $(1,3,7)$, the JAX sequence-sharded calculation and dense PyTorch function both return $4.6$ on CPU. The JAX lowered module exposes a maximum reduction followed by a sum reduction.
Layouts and descent
A layout $f:I\to S$ groups a family of values into fiberwise products. For successive layouts $f$ and $g$, the natural isomorphism $g_*f_*\cong(gf)_*$ forgets the extra level of product parentheses. Coherence for refinements follows because each composite selects the same logical leaves. Replicated coordinates give a second condition. At one version, assignments on a cover of a finite coordinate set glue if they agree on every overlap. In the groupoid-valued example overlaps supply isomorphisms and triple intersections check their composition. These criteria govern compatibility of a chosen snapshot. Commit and retry behavior belong to the storage and execution protocols.
Reproducibility and preparation
All executable checks use one host. They establish the stated numerical outputs and exercise both local process boundaries and logical CPU meshes. The archive contains the complete LaTeX sources and commands to regenerate the StableHLO modules. Generative AI assisted drafting, typesetting, and preparation of the examples. The manuscript uses an anonymous draft byline. The submitting author is responsible for the final claims and attribution.
Selected references
- Cruttwell et al., Categorical Foundations of Gradient-Based Learning.
- Fritz, A Synthetic Approach to Markov Kernels.
- Riehl and Verity, Elements of Infinity-Category Theory.
- Dao et al., FlashAttention.
- OpenXLA, StableHLO Specification.
- JAX, Manual Parallelism with shard_map.
- PyTorch, Distributed Communication.