\documentclass[10pt,twocolumn,letterpaper]{article}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage[letterpaper,top=.76in,bottom=.66in,left=.70in,right=.70in,columnsep=.27in]{geometry}
\usepackage{amsmath,amssymb,amsthm,mathtools}
\usepackage{booktabs,array,graphicx,xcolor}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage{microtype}
\usepackage{fancyhdr}
\usepackage{tikz,pgfplots}
\pgfplotsset{compat=1.18}
\usepgfplotslibrary{groupplots}
\usepackage{ragged2e}
\usepackage[colorlinks,linkcolor=inkblue,citecolor=inkblue,urlcolor=inkblue,
  pdftitle={Cotangent Semantics for Distributed Optimization}]{hyperref}
\definecolor{inkblue}{HTML}{1C4A69}
\definecolor{graytext}{HTML}{4C5A63}
\definecolor{gaorange}{HTML}{A34925}
\definecolor{boostteal}{HTML}{13796C}
\definecolor{uniformpurple}{HTML}{76548A}
\color{black}
\setlength{\parindent}{1em}
\setlength{\parskip}{0pt}
\setlength{\headheight}{14pt}
\setlength{\textfloatsep}{13pt plus 3pt minus 3pt}
\setlength{\floatsep}{12pt plus 2pt minus 2pt}
\setlength{\intextsep}{10pt plus 2pt minus 2pt}
\setlength{\abovecaptionskip}{5pt}
\setcounter{topnumber}{3}
\renewcommand{\topfraction}{.88}
\renewcommand{\textfraction}{.08}
\renewcommand{\arraystretch}{1.15}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{\small\textsc{Cotangent semantics}}
\fancyhead[R]{\small\textsc{Distributed optimization}}
\fancyfoot[C]{\small\thepage}
\renewcommand{\headrulewidth}{.3pt}
\fancypagestyle{plain}{\fancyhf{}\fancyfoot[C]{\small\thepage}\renewcommand{\headrulewidth}{0pt}}
\raggedbottom
\lefthyphenmin=3
\righthyphenmin=3
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\input{glyphtounicode}
\pdfgentounicode=1
\newcommand{\R}{\mathbb R}
\newcommand{\N}{\mathbb N}
\newcommand{\id}{\mathrm{id}}
\newcommand{\Spc}{\mathcal S}
\newcommand{\curvekey}[3]{\tikz[baseline=-.45ex]{\draw[#1,#2,very thick] (0,0)--(.44,0);}\,\footnotesize #3}
\newcommand{\Graph}{\mathsf{Circ}}
\newcommand{\Sched}{\mathsf{Sched}}
\newcommand{\Lens}{\mathsf{CotLens}}
\newcommand{\Obj}{\mathcal L}
\newcommand{\KL}{\operatorname{KL}}
\newcommand{\E}{\mathbb E}
\newcommand{\softmax}{\operatorname{softmax}}
\newcommand{\MaxDerivativeError}{3.83\times 10^{-11}}
\newcommand{\FrameCount}{2,934,129}
\newcommand{\ForwardCount}{87,460}
\newcommand{\ReverseCount}{28,808}
\newcommand{\CorrelationCount}{30,720}
\newcommand{\TrafficTotal}{1.922}
\newcommand{\FrozenAccuracy}{88.91}
\newcommand{\FrozenLoss}{0.380}
\begin{document}
\twocolumn[{\begin{@twocolumnfalse}
\noindent{\color{inkblue}\small\textsc{Research manuscript}\par}
\vspace{.55em}
{\LARGE\bfseries Cotangent Semantics for Distributed Optimization\par}
\vspace{.40em}
{\large Schedule coherence and reproducible benchmarks on three QEMU guests\par}
\vspace{.55em}
{\normalsize Anonymous submission\par}
\vspace{.95em}
\noindent{\color{inkblue}\rule{\textwidth}{.55pt}}\par
\vspace{.40em}
\noindent\textbf{Abstract.}\quad
A distributed smooth program can expose its reverse dependency graph as a protocol for cotangent messages. We construct a semantics in which legal schedules of a typed acyclic circuit are related by coherent 2-cells. A symmetric monoidal reverse interpretation sends every legal schedule to the same cotangent lens, with transposed derivatives and additive cotangent fan-in. A finite-fiber base-change identity describes aggregation under reindexing, and the interpretation descends for compatible smooth families over a context site. We derive synchronous minibatch SGD from the distributed pullback and implement the forward and reverse traffic on three QEMU guests. At 18,432 training example evaluations per configuration, SGD gave the lowest mean test log loss on Iris with radial basis features, handwritten digits with radial basis features and with a tanh network, and pooled Fashion-MNIST. The comparison includes a real-coded genetic algorithm, an entropy-maximizing additive boosting rule, and uniform and frozen-feature controls. Two independently booted cluster runs reproduced all logged losses, accuracy counts, derivative checks, and application-byte counts exactly.
\par\vspace{.55em}
\noindent\textbf{Keywords.}\quad Reverse-mode differentiation, categorical semantics, distributed optimization, stochastic gradient descent, entropy regularization, reproducibility
\par\vspace{.95em}
\end{@twocolumnfalse}}]
\thispagestyle{plain}

\section{Introduction}

A distributed implementation exposes more structure than its extensional input--output map. Independent operations can finish in different orders. A feature shard can be moved to a different machine. A parameter can contribute to several outputs, so several messages must be combined before a reverse operation can execute. The semantics has to say which changes preserve the computed differential and which affect the cost of computing it. We address this question for finite smooth programs whose primal trace and vector--Jacobian products are available.

Reverse derivative categories provide an axiomatic setting for reverse differentiation \cite{cockett}. Parametric lenses describe how learning rules act on models \cite{cruttwell}. Adjoinable MPI studies the reversal of communication in numerical programs \cite{utke}. Our construction relates these ideas at the level of typed schedules. It gives a two-dimensional presentation of reorderings, a cotangent correctness theorem with explicit wire occurrences, and a base-change law for finite aggregation. The presentation also specifies what is preserved when a forward and reverse exchange is placed across a virtual network.

The experiments ask two separate questions. First, do actual cross-guest cotangents agree with independent finite differences and survive a change in scheduling? Second, what happens when full SGD, a forward-only genetic algorithm, and a boosting rule using entropy-maximizing example weights are given the same number of forward evaluations? The boosting rule adds to a finite dictionary of weak learners. Its weighting policy has a precise variational characterization, and a uniform-weight ablation isolates that policy.

\section{Typed circuits and their dual}

\subsection{Smooth primitives and occurrences}

An interface is an ordered list of finite-dimensional real vector spaces, interpreted as their product. Execution uses fixed coordinates on each factor. The empty list is interpreted by the zero-dimensional space. An open circuit has a finite acyclic event graph with ordered input and output ports. Each event \(v\) carries a smooth map \(f_v\) from the product of its input types to the product of its output types. Each input port has exactly one source wire. An output wire may be unused or may feed multiple ports, including repeated inputs to one event. Numbering consumers by \emph{port occurrence} retains that multiplicity. Boundary output ports count as consumers. We write \(F_C\colon X\to Y\) for the resulting smooth map of a circuit \(C\).

Parameter and data inputs may be written \(X=P\times A\). Shared parameters are wires with multiple consumers. Batching is a finite product of per-example computations with copies of a shared parameter input and a final loss aggregation. The smoothness requirement applies to parameters for each fixed data record and each fixed branch of the training protocol. We use \(C^1\) for the reverse semantics and additional smoothness assumptions for optimization estimates. Discrete labels and minibatch indices are held fixed during differentiation.

For a smooth map \(f\colon X\to Y\), its reverse at \(x\) is the linear map
\[
  f_x^*\colon Y^*\longrightarrow X^*,
  \qquad f_x^*\bar y=\bar y\circ Df_x .
\]
A reverse lens from \(X\) to \(Y\) consists of a smooth forward map \(f\) and a smoothly varying, fiberwise linear backward map \(r_x\colon Y^*\to X^*\). The composite of \((f,r)\) followed by \((g,s)\) is
\[
    (g,s)\circ(f,r)=(g\circ f,\ x\mapsto r_x\,s_{f(x)}) .
\]
We use smooth maps in the lens category and the corresponding \(C^1\), continuous-backward variant when only first differentiability is assumed. The product of lenses uses the product forward map and the direct sum of backward maps. This defines \(\Lens\), with reverse differentiation \(f\mapsto(f,f^*)\) respecting composition and products by the chain rule. Standard coordinates identify a cotangent with its Euclidean gradient when an optimizer needs a vector.

\begin{definition}[Dual graph]
The dual of an executed circuit has a reverse edge for each primal port occurrence. Its value is the cotangent sent from that consumer to the source. The reverse event uses the primal inputs, or equivalent saved data, to apply \(f_v^*\). At every wire, the event waits for its required cotangents and sums them. A boundary seed contributes once per output occurrence. A wire with no contribution receives zero.
\end{definition}

For example, if a scalar input is used in both arguments of multiplication, the derivative of \(m\circ\Delta\) at \(x\), with \(m(a,b)=ab\), is \(2x\). In the dual graph, each argument sends \(x\bar y\) to the same source. A graph that deduplicates those messages returns \(x\bar y\). If \(x\) is duplicated onto \(n\) destinations, \(\Delta(x)=(x,\ldots,x)\) has pullback \((\bar x_i)_i\mapsto\sum_i\bar x_i\). Conversely the pullback of an addition node sends its output cotangent to every input. Identity transport and wire permutations pull back by the transpose of their routing maps. The operation does not require the forward primitive to be invertible.

\subsection{Correctness and communication}

\begin{theorem}[Distributed pullback]\label{thm:pullback}
Fix a circuit \(C\), input \(x\), and output seed \(\bar y\). Any legal forward evaluation followed by a reverse evaluation that delivers each contribution exactly once returns \(DF_C(x)^*\bar y\). The result is invariant under event placement provided communication implements the specified wire maps.
\end{theorem}

\begin{proof}
For any input variation \(\dot x\), propagate a tangent \(\dot w\) along every wire by applying \(Df_v\) to the tuple of event-input tangents. Pair the resulting boundary output tangents with \(\bar y\). During reverse evaluation, replace each pairing of event-output cotangents with event-output tangents by the corresponding pairing of \(f_v^*\) with event-input tangents. This follows from the definition of the transpose. At each wire collect all pairings with its common tangent by additivity. Reverse acyclicity eventually leaves only the boundary input pairings. The sum of those pairings equals \(\langle\bar y,DF_C(x)\dot x\rangle\) for every \(\dot x\). Nondegeneracy of the pairing gives the claim. A transmission of the same typed value changes none of the pairings. For a nonidentity wire map its transpose enters the same argument.
\end{proof}

This proof models a complete logical trace. Ordered TCP gives the byte stream used by our implementation. Its requests also name a sequence, task, row count, and the forward tape identifier, which is consumed once by a reverse or statistics operation. A lost or stale tape invalidates that iteration. In real arithmetic the sum is independent of its order. The implementation fixes operand order at the coordinator so that changing the worker dispatch order does not change the floating-point sum. An asynchronous system with duplicate retries would need occurrence-level deduplication or a protocol establishing exactly-once contributions.

\section{Schedules and higher coherence}

\subsection{A locally groupoidal 2-category}

Let \(\Graph\) be the category with typed interfaces as objects and open circuits up to boundary-preserving isomorphism as arrows. An isomorphism preserves the primitive labels, port orders, and wiring. Composition glues matching boundary ports, and its isomorphism class is independent of the order of gluing. The tensor is disjoint union of events and concatenation of interfaces, with the symmetry induced by reordering boundary wires. A placement may be carried as a vertex label when the schedule is to be compared at fixed deployment. Erasing that label is a functor.

A schedule is a linear extension of the precedence order on events. Let \(\Sched\) have the same objects and scheduled circuits, up to schedule-preserving isomorphism, as 1-cells. For any two 1-cells with the same underlying circuit in \(\Graph\), give the hom-category exactly one invertible 2-cell between them. It represents the identification of legal reschedulings. For distinct underlying circuits there are no such 2-cells. Concatenation after gluing composes 1-cells. Vertical composition of 2-cells is forced by uniqueness. Horizontal composition is well defined since equality in \(\Graph\) is a congruence for gluing. Interchange, associativity and the unit axioms follow from this description.

\begin{lemma}[Adjacent exchanges]\label{lem:swaps}
Every rescheduling 2-cell is generated by exchanges of adjacent incomparable events.
\end{lemma}

\begin{proof}
Fix a target linear extension. In the current extension move its first event to the first position by adjacent swaps. Each event passed is incomparable with it. If a passed event preceded it in the order, the target could not start with the chosen event. If it followed it, the current extension could not place it earlier. Remove the common first event and repeat on the remaining finite order.
\end{proof}

Equivalently, take the groupoid of paths of such exchanges and identify parallel paths. This quotient records schedule equivalence and omits the history of swaps. Tensoring 1-cells concatenates their independent schedules. Its compositor identifies two legal schedules on the same glued disjoint circuit. The unit and symmetry comparisons use the empty schedule and boundary permutation. The coherence diagrams commute since their comparisons have the same boundary permutation and there is at most one 2-cell between their scheduled circuits.

\begin{theorem}[Coherent reverse interpretation]\label{thm:coherence}
Reverse differentiation induces a symmetric monoidal interpretation of \(\Sched\) in the locally discrete 2-category of cotangent lenses. It factors through \(\Graph\), and forgetting schedules induces an equivalence on the homotopy-coherent nerves of the corresponding categories enriched in spaces.
\end{theorem}

\begin{proof}
For each scheduled circuit \(C\), assign the lens \((F_C,DF_C^*)\). The extensional forward evaluation depends only on wiring. Theorem~\ref{thm:pullback} gives equality of reverse evaluations for any two schedules of that wiring. Gluing is composition of the forward maps, and its reverse is the composite of the transposes in reverse order. Disjoint union gives the product lens, so the tensor and its comparison 2-cells are preserved. The hom-groupoid of \(\Sched\) maps to the discrete hom-set of \(\Graph\). Each fiber is a nonempty groupoid with exactly one arrow between any two objects, hence has contractible nerve. This gives a weak equivalence on every simplicial mapping space and identity on objects. The homotopy-coherent nerve gives the stated equivalence \cite[Sec.~1.1.5]{lurie}.
\end{proof}

This is a particular higher semantics with contractible spaces of schedule choices. We quotient reorderings of a fixed dependency graph. Graph rewrites, timings, partial failure histories and resource allocations can give finer observations. Theorem~\ref{thm:coherence} only identifies the value and cotangent semantics. Our executable audit compares two worker dispatch orders on identical inputs and stores cost measurements independently.

\subsection{Indexed aggregation and descent}

Let \(R\) be a commutative ring and let \(u:I\to J\) be a finite map of indexing sets. On families of \(R\)-modules, reindexing \(u^*\) has a left adjoint \(\Sigma_u\) given by fiberwise direct sums and a right adjoint \(\Pi_u\) given by fiberwise products. The canonical map from sum to product is an isomorphism for finite fibers, including an empty fiber. For a repeated module \(M_j\), the induced replication map has diagonal components \(M_j\to\prod_{u(i)=j}M_j\). When these modules are finite free, the dual map has summation components \(\bigoplus_{u(i)=j}M_j^*\to M_j^*\).

\begin{proposition}[Base change for reverse aggregation]\label{prop:bc}
Given a pullback square of finite sets
\[
\begin{tikzpicture}[baseline=(J.base),scale=.76]
  \node (I1) at (0,1) {\(I'\)}; \node (I) at (2,1) {\(I\)};
  \node (J1) at (0,0) {\(J'\)}; \node (J) at (2,0) {\(J\)};
  \draw[->] (I1)--node[above]{\(\scriptstyle a\)}(I);
  \draw[->] (I1)--node[left]{\(\scriptstyle u'\)}(J1);
  \draw[->] (I)--node[right]{\(\scriptstyle u\)}(J);
  \draw[->] (J1)--node[below]{\(\scriptstyle b\)}(J);
\end{tikzpicture}
\]
the comparison \(b^*\Sigma_u\cong\Sigma_{u'}a^*\) identifies the two reverse aggregations under fiber reindexing. It is compatible with pasting pullback squares.
\end{proposition}

\begin{proof}
At \(j'\in J'\), \(a\) induces a bijection \((u')^{-1}(j')\to u^{-1}(b(j'))\). Reindexing the summands by this bijection gives the comparison. Both folds add exactly those summands, with the empty fold zero. Composing pullback squares composes their fiber bijections. The argument works for weighted linear maps when weights are transported with their occurrences.
\end{proof}

For context variation, take the site of opens of a smooth manifold \(B\). Let each interface over an open \(U\) be a smooth vector bundle, with products taken over \(U\). A primitive is a smooth bundle map over \(U\) that is allowed to be nonlinear on the fibers. Let \(T_v f\) denote its vertical differential and \(T_v f^*\) the pointwise dual of that differential. Both commute with restriction to smaller opens. In overlapping coordinates the usual chain rule transports \(T_v f\) and the dual transition maps transport the cotangents. This construction is interpreted in sheaves over \(B\), or in their embedding as \(0\)-truncated objects of \(\operatorname{Shv}(B;\Spc)\). It gives a topos-theoretic account of compatible local programs, with no derivative assigned to an arbitrary internal morphism.

\begin{proposition}[Descent in the parameter context]\label{prop:descent}
Fix the finite circuit and its port occurrences over \(B\). Compatible smooth local primitive maps on an open cover, with their vector bundle transition functions, define a global forward map and a global vertical cotangent pullback. The global pullback restricts to the composite local dual protocol.
\end{proposition}

\begin{proof}
Vector bundle sections and smooth maps over \(B\) glue uniquely when the local maps agree under the given transitions. In particular the local forward interpretations glue. The vertical chain rule makes the local dual maps agree on each overlap. Finite sums and composition of smooth maps commute with restriction, so the local execution of the dual also glues. On each member of the cover it is the vertical cotangent pullback by Theorem~\ref{thm:pullback}, applied fiberwise. Uniqueness of the glued map identifies the two global maps.
\end{proof}

The finite-fiber law concerns how indexed contributions are regrouped. For example, partitioning a batch into unequal blocks requires multiplying each local mean by its block size before taking the global mean. An unweighted mean of local means changes the objective whenever block sizes differ and the block means are unequal. The experiments instead shard the model and features, and their output reduction is a sum of partial logits.

\section{Optimization from the dual}

Let \(f_\theta(x)\in\R^K\) be the logits and \(\ell_i(\theta)=\operatorname{lse}(f_\theta(x_i))-f_\theta(x_i)_{y_i}\), where \(\operatorname{lse}(z)=\log\sum_c e^{z_c}\). For \(n\) training examples, set
\[
  \Obj(\theta)=\frac{1}{n}\sum_{i=1}^n \ell_i(\theta)
          +\frac{\lambda}{2}\|\theta\|^2 .
\]
A batch \(I=(i_1,\ldots,i_B)\) is sampled independently with replacement from the training indices. The pullback of its mean loss uses seed \(1\) and returns
\[
 g_I(\theta)
   =\frac{1}{B}\sum_{a=1}^{B}\nabla\ell_{i_a}(\theta)+\lambda\theta .
\]
Each worker computes its parameter block at the parameter version from the forward pass. For shared parameter blocks, the result includes the transpose of the replication map. The coordinator then applies \(\theta\leftarrow\theta-\eta g_I(\theta)\). The Euclidean update depends on the chosen metric, even though the cotangent pullback is natural under smooth coordinate changes. A change of coordinates must carry the metric along to compare intrinsic gradient directions.

\begin{proposition}[Synchronous SGD]\label{prop:sgd}
For an independent uniform minibatch and fixed \(\theta\), \(\E[g_I\mid\theta]=\nabla\Obj(\theta)\). If \(\Obj\) is \(L\)-smooth along the update segments and the trace of the single-example gradient covariance at the visited \(\theta\) is at most \(\sigma^2\), then for \(0<\eta\le 1/L\),
\begin{align*}
 \E[\Obj(\theta-\eta g_I)\mid\theta]
 &\le \Obj(\theta)-\frac{\eta}{2}\|\nabla\Obj(\theta)\|^2\\
 &\qquad+\frac{L\eta^2\sigma^2}{2B}.
\end{align*}
\end{proposition}

\begin{proof}
Linearity of expectation gives unbiasedness, and independence gives conditional variance at most \(\sigma^2/B\). Substitute \(v=-\eta g_I\) in
\(\Obj(\theta+v)\le\Obj(\theta)+\langle\nabla\Obj(\theta),v\rangle+L\|v\|^2/2\).
Taking the conditional expectation and using \(L\eta\le 1\) gives the claim.
\end{proof}

The bound uses the stipulated smoothness and step size, as in standard stochastic optimization \cite{bottou}. The benchmark chooses the learning rate on validation loss. For a linear softmax head on fixed features, the Hessian is positive semidefinite before regularization. With ridge on all parameters it is at least \(\lambda I\), and its upper bound is finite on a finite feature matrix. A bound \(L\le\lambda+\max_i\|\tilde x_i\|^2\) follows by using the bias-augmented feature vector \(\tilde x_i\) and the inequality \(0\preceq\operatorname{diag}(p)-pp^\top\preceq I\). The tanh network is nonconvex. Its empirical comparison does not require a global smoothness or global convergence claim.

The optimizer acts on the state after evaluating a circuit and its cotangent. Write \(S\) for a standard Borel space of parameters and optimizer state, and \(\Omega\) for the space of draws. For a schedule \(\alpha\), let \(U_\alpha(s,\omega)\) apply the deterministic update given the objective value and cotangent, with the batch and any other randomness fixed by \(\omega\). The resulting state evolution is the Markov kernel
\[
 K_\alpha(s,A)=\int_\Omega
   \mathbf{1}_{A}\bigl(U_\alpha(s,\omega)\bigr)\,\mu_s(d\omega),
\]
where \(\mu_s\) is the stipulated sampling kernel. The genetic population and mutation draws, and the boosting dictionary index, define analogous state transitions \cite{fritz}. Hard rank selection, tie breaking, and accuracy evaluation are measurable operations. Cotangents differentiate the smooth objective conditional on its inputs.

\begin{proposition}[Schedule invariance of the update law]
For fixed sampling and an update that depends only on the optimizer state, objective and cotangent, two schedules identified in \(\Sched\) yield the same one-step Markov kernel. The joint law of any finite sequence of such updates is also the same when each schedule may be chosen anew at each step.
\end{proposition}

\begin{proof}
For each fixed draw, the forward and backward interpretations agree by Theorem~\ref{thm:coherence}. The state updates are consequently equal pointwise, and integrating against the same sampling kernel gives equal transition probabilities. Coupling the iterates by the same draw conditional on the current state makes their states agree inductively. The same reasoning allows the schedule to depend on earlier draws and observed states as long as it preserves the circuit and the conditional sampling kernel.
\end{proof}

This result concerns the law of the model state. When a scheduler changes the sample stream, accepts stale snapshots, or changes a reduction in finite precision, equality of the ideal lenses alone gives no corresponding runtime guarantee.

\section{Baselines and entropy weighting}

\subsection{Real-coded genetic algorithm}

The genetic baseline uses a population of eight parameter vectors and ranks them by the regularized batch objective, using the same batch for all candidates in a generation. The two best candidates are copied to the next generation. For each of six offspring, two parents are drawn independently from the best four, and a uniformly drawn scalar gives an affine crossover between the parents. Independent normal mutations are scaled by parameter block. Every candidate evaluation uses the same distributed forward program as SGD. It sends no backward request. At a checkpoint the best individual of the latest evaluated generation is used. This is a specific real-coded genetic algorithm in the sense of selection, recombination and mutation \cite{holland}.

\subsection{Entropy-maximizing additive boosting}\label{sec:entropy}

A weak atom \(a_j(x)\) is one scalar feature, one frozen hidden unit, or the constant bias atom. A block \(v_j\in\R^K\) controls the additive logits \(a_j(x)v_j\). At each round, the baseline chooses one atom and adds a vector to its coefficient. Repeated selections add to the same coefficient, so the number of active atoms is bounded by the dictionary size. The linear and RBF tasks use the same hypothesis class as SGD. On the MLP, the weak dictionary freezes the hidden units at the matched initialization. We include SGD with the same hidden blocks frozen as a separate control.

Let \(u\) denote the uniform distribution on the \(B\) batch positions. Given the current losses \(\ell_a\), choose weights
\begin{align}
 q_\tau&=\arg\max_{q\in\Delta_B}
    \bigg\{\sum_a q_a\ell_a-\tau\KL(q\|u)\bigg\},\label{eq:kl}\\
 (q_\tau)_a&=\frac{\exp(\ell_a/\tau)}{\sum_b\exp(\ell_b/\tau)},
    \qquad \tau>0.\nonumber
\end{align}
This tradeoff assigns more weight to high-loss positions and rewards entropy. If one fixes the mean loss at \(\sum_a(q_\tau)_a\ell_a\), then \(q_\tau\) is the unique maximum-entropy distribution for that constraint. Indeed the expression in \eqref{eq:kl} equals
\[
 \tau\log\Big(B^{-1}\textstyle\sum_a e^{\ell_a/\tau}\Big)
          -\tau\KL(q\|q_\tau).
\]
On the constraint set, maximizing it is the same as maximizing entropy. Its maximum value is a smooth batch risk, whose derivative is \(\sum_a(q_\tau)_a\nabla\ell_a\). At fixed losses the derivative of \(H(q_\tau)\) in \(\tau\) is \(\operatorname{Var}_{q_\tau}(\ell)/\tau^3\). Uniform weighting is the \(\tau\to\infty\) control. Minimizing the smoothed batch risk is a different stochastic objective from minimizing the ordinary batch mean. In particular the nonlinear batch weights do not in general give an unbiased gradient for the corresponding full-training-set entropic risk.

With \(p_a=\softmax(f_\theta(x_{i_a}))\) and \(e_y\) the one-hot vector, define
\begin{align*}
 g_j &= \lambda v_j+
          \sum_a (q_\tau)_a a_j(x_{i_a})(p_a-e_{y_{i_a}}),\\
 D_j &= .05+\lambda+\sum_a(q_\tau)_a a_j(x_{i_a})^2 .
\end{align*}
Minimizing the block surrogate \(g_j^\top \delta+D_j\|\delta\|^2/2\) would improve it by \(\|g_j\|^2/(2D_j)\). The implementation chooses the largest \(\|g_j\|^2/D_j\) and takes \(-.75\,g_j/D_j\) on that block. The coordinator sends \(q_\tau\) and the weighted logit residuals to both workers. Each returns its best local atom with score, divisor and gradient. The coordinator also considers the bias. A tie keeps the earlier candidate, with bias first, followed by worker 1 and worker 2, and feature index order within a worker. This damped surrogate is a selection rule. No claim of per-round monotonicity for the full training loss is needed. The method is related to stagewise boosting and entropy-regularized boosting \cite{freund,warmuth}, with a different multiclass objective and update.

\section{Three-guest experiment}

\subsection{Placement and message accounting}

Every run boots three independent Alpine Linux 3.23.8 initramfs guests, one virtual CPU and 192 MiB each, on QEMU 10.0.13 with TCG \cite{qemu}. The coordinator \(C\) owns labels, the global bias, parameter snapshots, the loss and optimizer state. Workers \(W_1,W_2\) own feature shards and their local forward and reverse computations. The host builds the guests and relays Ethernet frames on an isolated virtual LAN. No optimization or scoring runs on the host. Guests exchange TCP messages over emulated e1000 interfaces. Each worker records its own application-byte and operation counts, which the verifier reconciles against the coordinator's records.

\begin{figure}[t]
\centering
\begin{tikzpicture}[x=1cm,y=1cm,
 vm/.style={draw=inkblue,rounded corners=2pt,fill=inkblue!5,minimum width=2.05cm,
             minimum height=.60cm,align=center,font=\footnotesize},
 tag/.style={font=\scriptsize,text=graytext,fill=white,inner sep=1pt}]
 \node[vm] (one) at (-2.13,0) {\(W_1\)\\feature shard 1};
 \node[vm] (two) at (2.13,0) {\(W_2\)\\feature shard 2};
 \node[vm,minimum width=2.3cm] (core) at (0,1.54) {\(C\)\\loss and updates};
 \draw[->,thick,inkblue] (one.north east) -- (core.south west)
       node[tag,pos=.51,above left]{partial logits};
 \draw[->,thick,inkblue] (two.north west) -- (core.south east)
       node[tag,pos=.51,above right]{partial logits};
 \draw[->,thick,boostteal] (core.south) -- (one.east)
       node[tag,pos=.53,below right]{cotangent};
 \draw[->,thick,boostteal] (core.south) -- (two.west)
       node[tag,pos=.53,below left]{cotangent};
\end{tikzpicture}
\caption{Primal outputs and their reverse seeds on three guests. Requests carry batch indices and the parameter snapshot. In the reverse round, the workers return their parameter-block pullbacks.}
\label{fig:cluster}
\end{figure}

Each worker emits \(z_i\in\R^K\) per row. The coordinator uses \(z=(b+z_1)+z_2\). For a linear shard the worker computes \(z_i=W_i^\top x_i\) and its pullback accumulates \(x_i\bar z^\top\). For the MLP it computes \(z_i=V_i^\top\tanh(A_i x+b_i)\), with 16 hidden units per worker. Both MLP workers receive the same preprocessed inputs and own disjoint hidden/output blocks. For \(h_i=\tanh(A_i x+b_i)\), its local formulas are
\begin{align*}
 \bar V_i &= h_i \bar z^\top,\qquad
 t_i = (V_i\bar z)\odot(1-h_i^2),\\
 \bar A_i &= t_i x^\top,\qquad \bar b_i = t_i .
\end{align*}
The coordinator sums its bias cotangents over rows and adds the ridge gradient once, after collecting all parameter blocks. The tape stores hidden activations and the snapshot used for each worker. Cross-guest messages carry a 20-byte header on requests and arrays of indices or binary64 values as appropriate. Bytes reported below are application payload and framing over both links, excluding the TCP/IP and Ethernet headers.

\subsection{Datasets and hypotheses}

\begin{table}[t]
\centering\small
\caption{Fixed training, validation and test splits. Counts refer to the data loaded by the guests.}
\label{tab:data}
\begin{tabular}{@{}lrrr r@{}}
\toprule
Workload & Train & Val. & Test & Params.\\
\midrule
Iris RBF       & 90    & 30    & 30    & 75\\
Digits RBF     & 1,074 & 355   & 368   & 810\\
Digits MLP     & 1,074 & 355   & 368   & 2,410\\
Fashion linear & 4,000 & 1,000 & 2,000 & 500\\
\bottomrule
\end{tabular}
\end{table}

\paragraph{Iris RBF.}
The three-class Iris task uses the corrected UCI rows \cite{iris}. Train/validation/test indices are fixed by a stratified 60/20/20 split. The four input measurements are standardized using training statistics. The head uses eight training landmarks per class, giving 24 Gaussian radial basis features.

\paragraph{Digits RBF and MLP.}
The 1,797 rows of the UCI Optical Recognition of Handwritten Digits test file \cite{digits} are split into new, disjoint 60/20/20 partitions by class. Thus these accuracies are for the supplied split of that file. Inputs are 8-by-8 pixel counts divided by 16. The kernel model uses 80 training landmarks, eight per class. The neural model standardizes its 64 inputs with training statistics and learns 32 tanh units in two shards.

\paragraph{Fashion linear.}
We take 400 training and 100 validation images per class from the official Fashion-MNIST training set and 200 test images per class from its official test set \cite{fashion}. Nonoverlapping \(4\times4\) average pooling produces 49 features from each \(28\times28\) grayscale image. Training-only standardization follows. The head is ten-class softmax with 49 features and a bias. This is an image classification workload with a deliberately small linear head.

The RBF feature for landmark \(c_j\) is \(\exp(-\gamma\|x-c_j\|^2)\). The program sets \(\gamma\) to the reciprocal of the median squared distance of 512 reproducibly sampled training pairs. It standardizes each feature using training statistics. This realizes a finite kernel classifier with \(K_{\mathrm{fin}}(x,x')=\langle\phi(x),\phi(x')\rangle\). It uses regularization in feature coefficient space. Kernel feature methods provide the relevant comparison class \cite{rahimi}, while this construction uses fixed training landmarks. The archive records all split indices, landmark rows and bandwidths, together with the code that regenerates the feature shards. Table~\ref{tab:data} gives the model sizes.

\subsection{Budgets and selection protocol}

We run each workload at seeds \(11,23,37,53,71\) with the fixed splits. A configuration receives 384 forward queries of 48 examples each, or 18,432 example evaluations. GA spends eight queries on one common batch for each of 48 generations. Each other optimizer consumes 384 independently sampled batches and one forward query per batch. Thus the accounting matches forward exposure, including all GA candidate evaluations. It permits different independent draws, backward work and wall time. For all methods \(\lambda=10^{-4}\), and the same seed sets the common initial parameter vector. GA's remaining initial individuals perturb that vector. The two MLP boosting controls and frozen SGD share the same hidden-feature initialization as full SGD.

Each primary method has three hyperparameter choices. SGD starts at \(\eta_0\in\{.04,.12,.36\}\) and uses \(\eta_s=\eta_0/\sqrt{1+s/192}\) at zero-based step \(s\). GA uses initial mutation scale in \(\{.04,.12,.30\}\), annealed by \(1-.6s/384\) where \(s\) is the number of prior candidate evaluations. For the bias and hidden-bias entries the relative mutation scale is 1, for input weights it is the inverse square root of input width, and for output weights it is the inverse square root of total feature or hidden width. The entropy variants use \(\tau\in\{.5,1,2\}\) and the uniform control has one configuration. No test metric enters model selection. For each method and seed we select the lowest validation log loss after query 384. Intermediate validation checkpoints at 96, 192 and 288 show the course of training. The test set is evaluated once per selected configuration. Evaluation, audit and tuning overhead are included in the cluster-wide counts and excluded from the per-configuration training budgets.

\section{Results and replication}

\begin{table*}[t]
\centering\small
\caption{Held-out accuracy in percent. Entries are mean \(\pm\) sample standard deviation across five seeds on fixed splits. The uniform column uses the same weak atoms as entropy boosting.}
\label{tab:accuracy}
\begin{tabular}{@{}lrrrr@{}}
\toprule
Workload & SGD & Genetic algorithm & Entropy boosting & Uniform boosting\\
\midrule
Iris RBF & 96.67\ensuremath{\pm}0.00 & 90.00\ensuremath{\pm}2.36 & 96.00\ensuremath{\pm}1.49 & 96.67\ensuremath{\pm}0.00 \\
Digits RBF & 95.38\ensuremath{\pm}0.43 & 43.75\ensuremath{\pm}2.72 & 93.21\ensuremath{\pm}0.86 & 93.48\ensuremath{\pm}0.43 \\
Digits MLP & 96.09\ensuremath{\pm}0.68 & 26.41\ensuremath{\pm}4.46 & 88.21\ensuremath{\pm}0.87 & 87.83\ensuremath{\pm}0.40 \\
Fashion linear & 78.24\ensuremath{\pm}0.77 & 33.87\ensuremath{\pm}4.12 & 72.95\ensuremath{\pm}0.69 & 72.78\ensuremath{\pm}0.57 \\
\bottomrule
\end{tabular}
\end{table*}

\begin{table}[t]
\centering\small
\caption{Mean test negative log likelihood in nats per example.}
\label{tab:loss}
\setlength{\tabcolsep}{4.5pt}
\begin{tabular}{@{}lrrrr@{}}
\toprule
Workload & SGD & GA & Entropy & Uniform\\
\midrule
Iris RBF & 0.078 & 0.259 & 0.089 & 0.085 \\
Digits RBF & 0.147 & 1.646 & 0.317 & 0.314 \\
Digits MLP & 0.123 & 2.044 & 0.525 & 0.515 \\
Fashion linear & 0.621 & 1.863 & 0.893 & 0.829 \\
\bottomrule
\end{tabular}
\end{table}

\begin{table}[t]
\centering\small
\caption{Median training traffic in MiB at the selected configuration. The count combines both worker links in both directions.}
\label{tab:traffic}
\setlength{\tabcolsep}{4.5pt}
\begin{tabular}{@{}lrrrr@{}}
\toprule
Workload & SGD & GA & Entropy & Uniform\\
\midrule
Iris RBF & 2.28 & 1.21 & 2.38 & 2.38 \\
Digits RBF & 10.48 & 5.31 & 8.49 & 8.49 \\
Digits MLP & 19.86 & 10.00 & 13.18 & 13.18 \\
Fashion linear & 8.67 & 4.40 & 7.58 & 7.58 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Derivative and schedule checks}

Before training, each task tests a regularized objective on eight rows. For four unit directions (coordinator bias, each worker block, and all blocks), the coordinator computes central differences at radius \(10^{-5}\) with distributed forward queries. It compares each estimate with the directional contraction of the distributed cotangent. The largest absolute difference is \(\MaxDerivativeError\). Reversing the two worker dispatch and collection orders, with fixed arithmetic order for reductions, changes neither the computed batch loss nor the returned gradient in these checks. This probes the implementation at finite points and is separate from the real-arithmetic proof.

In the reference run each worker recorded \(\ForwardCount{}\) forward requests, \(\ReverseCount{}\) full reverse requests and \(\CorrelationCount{}\) boosting statistics requests. The reconciled application traffic was \(\TrafficTotal{}\) GiB across both links, including validation and audits. The virtual switch recorded \(\FrameCount{}\) Ethernet frames from three distinct guest MAC addresses. The second run booted fresh guests, built bit-identical guest binaries and initramfs images on the recorded toolchain, and reproduced all 860 validation checkpoints, 85 test records and four derivative audit records exactly apart from the excluded timing fields. The Ethernet frame counts differ between the runs.

\subsection{Accuracy, loss and cost}

Tables~\ref{tab:accuracy} and \ref{tab:loss} give the primary comparisons. Full SGD has the lowest mean held-out log loss on all four workloads. It has the highest mean accuracy on both digit tasks and Fashion-MNIST and ties uniform boosting on Iris. The Fashion accuracy difference between SGD and entropy boosting is \(5.29\) percentage points. For Digits MLP, frozen SGD reaches \(\FrozenAccuracy{}\%\) with mean log loss \(\FrozenLoss{}\), versus \(96.09\%\) and \(0.123\) for full SGD. That control separates changing the hidden representation from changing the optimizer within a fixed dictionary. Frozen SGD uses the full reverse protocol and suppresses hidden updates, so it does not reduce the measured traffic.

Entropy weighting raises mean accuracy by 0.38 points on Digits MLP and 0.17 on Fashion, and lowers it by 0.67 on Iris and 0.27 on Digits RBF. Mean log loss is higher with entropy weighting on each workload. The mean normalized weighting entropy \(H(q)/\log B\) at selected configurations ranges from \(0.926\) to \(0.990\). The accuracy gains on two tasks coexist with higher mean log loss.

The genetic baseline consumes the fewest application bytes and has the shortest selected-configuration median training time on each workload in this run. For Digits MLP the selected medians are 2.60 s for full SGD, 1.94 s for GA and 2.38 s for entropy boosting. These durations describe the emulated single-host deployment and exclude evaluation. Table~\ref{tab:traffic} shows that returning full gradients can be materially more expensive than returning an atom summary. In SGD the reverse round also returns a parameter block on each link. Boosting sends batch weights and returns only the best atom summary from each worker. On Iris, the compact full gradients make the two budgets similar.

\begin{figure*}[t]
\centering
\begin{tikzpicture}
\begin{groupplot}[group style={group size=2 by 2,horizontal sep=1.0cm,vertical sep=.83cm,xlabels at=edge bottom,xticklabels at=edge bottom},
width=.429\textwidth,height=2.85cm,scale only axis,
title style={font=\small\bfseries},tick label style={font=\scriptsize},
label style={font=\footnotesize},xlabel={Forward queries},
ylabel={Validation NLL},xmin=86,xmax=395,xtick={96,192,288,384},
grid=major,grid style={gray!20},axis line style={gray!65}]
\nextgroupplot[title={Iris RBF}]
\addplot[inkblue,very thick,mark=*,mark size=2pt] coordinates {(96,0.15928144) (192,0.12401177) (288,0.10770155) (384,0.09312037)};
\addplot[gaorange,very thick,mark=square*,mark size=2pt] coordinates {(96,0.47877783) (192,0.37839944) (288,0.33426111) (384,0.31004877)};
\addplot[boostteal,very thick,mark=triangle*,mark size=2pt] coordinates {(96,0.14564999) (192,0.09509260) (288,0.07636548) (384,0.06614309)};
\addplot[uniformpurple,very thick,mark=diamond*,mark size=2pt,dashed] coordinates {(96,0.19454720) (192,0.13487515) (288,0.10962411) (384,0.09455036)};
\nextgroupplot[title={Digits RBF},ylabel={}]
\addplot[inkblue,very thick,mark=*,mark size=2pt] coordinates {(96,0.14787566) (192,0.12561968) (288,0.11208309) (384,0.10090086)};
\addplot[gaorange,very thick,mark=square*,mark size=2pt] coordinates {(96,2.06366710) (192,1.91597376) (288,1.74652723) (384,1.59833936)};
\addplot[boostteal,very thick,mark=triangle*,mark size=2pt] coordinates {(96,0.64869758) (192,0.40870034) (288,0.32259549) (384,0.26248846)};
\addplot[uniformpurple,very thick,mark=diamond*,mark size=2pt,dashed] coordinates {(96,0.60117870) (192,0.38101415) (288,0.29842410) (384,0.25427866)};
\nextgroupplot[title={Digits MLP}]
\addplot[inkblue,very thick,mark=*,mark size=2pt] coordinates {(96,0.15703193) (192,0.10624314) (288,0.08940936) (384,0.08233145)};
\addplot[gaorange,very thick,mark=square*,mark size=2pt] coordinates {(96,2.30023223) (192,2.18556125) (288,2.15299041) (384,2.04114718)};
\addplot[boostteal,very thick,mark=triangle*,mark size=2pt] coordinates {(96,0.91123377) (192,0.62583991) (288,0.51249900) (384,0.44916751)};
\addplot[uniformpurple,very thick,mark=diamond*,mark size=2pt,dashed] coordinates {(96,0.89876967) (192,0.61433785) (288,0.50314124) (384,0.43958002)};
\nextgroupplot[title={Fashion linear},ylabel={}]
\addplot[inkblue,very thick,mark=*,mark size=2pt] coordinates {(96,0.66691827) (192,0.62112822) (288,0.60283335) (384,0.58878103)};
\addplot[gaorange,very thick,mark=square*,mark size=2pt] coordinates {(96,2.24039725) (192,2.11511561) (288,1.99523192) (384,1.85760126)};
\addplot[boostteal,very thick,mark=triangle*,mark size=2pt] coordinates {(96,1.12311189) (192,0.96246740) (288,0.92064677) (384,0.87255610)};
\addplot[uniformpurple,very thick,mark=diamond*,mark size=2pt,dashed] coordinates {(96,1.06591535) (192,0.90476077) (288,0.83985042) (384,0.79850268)};
\end{groupplot}
\end{tikzpicture}\par\smallskip
\begin{tabular}{@{}llll@{}}
\curvekey{inkblue}{solid}{SGD} & \curvekey{gaorange}{solid}{Genetic algorithm} &
\curvekey{boostteal}{solid}{Entropy boosting} & \curvekey{uniformpurple}{dashed}{Uniform boosting}\\
\end{tabular}
\caption{Mean validation negative log likelihood across five seeds at four query checkpoints, using the configuration selected by final validation loss for each seed. Each panel uses its own vertical scale. The curves are retrospective summaries of the selected configurations.}
\label{fig:curves}
\end{figure*}

Figure~\ref{fig:curves} shows the validation trajectories in forward queries. The fixed budget captures a regime in which GA has only 48 population updates. The best GA mutation scale and SGD learning rate often lie at the upper grid boundary, so the comparison describes these particular grids. It does not estimate the best attainable score for any algorithm. The five seeds reuse one split, so their spread measures sensitivity to initialization and sampling on that split. Iris has only 30 test examples. Its observed ties and one-example changes should be read at that resolution. The Fashion workload uses pooled pixels and a subset. It does not estimate accuracy for the standard full-resolution benchmark or a modern convolutional model.

\section{Scope and implications}

A reverse interface specifies more than the presence of derivatives. It specifies the cotangent space on each port, the snapshot on which the pullback depends, and the aggregation of all uses of a wire. A synchronous implementation can therefore change its legal schedule or placement while preserving the semantic update, provided it preserves those data. A cost analysis must still retain the trace of transmissions and placement. The same lens admits different deployments with different message traffic and runtime.

The experiments use model sharding over one emulated LAN. There is no comparison against multiple physical hosts, decentralized consensus, stale gradient protocols, or fault-tolerant recovery. The raw transcripts are generated by the provided program, so their hashes and counter reconciliation establish repeatability and internal consistency. The rebuild and repeat-run comparison provide a direct check of the reported results. A program with data-dependent branching can use a fixed executed trace for conditional differentiation, with separate conditions needed near switching boundaries. Feedback can be unrolled to a finite acyclic program at a fixed horizon. Neither construction licenses transposing a nondifferentiable branch decision or ignoring the state needed to reverse a loop.

\section{Reproduction and disclosures}

The companion archive includes the C implementation, a Python 3 standard-library guest builder and runner, SHA-256 pins, checksum-verified prepared data shards, both runs' serial logs and manifests, the verifier, and the double-column LaTeX source. On a Linux x86-64 host, QEMU system emulation and a C compiler with static libc suffice to build and run the experiments. The archive's README gives the commands for a full cluster run, the cross-run comparison and regeneration of the manuscript tables and curves. The data shards can be regenerated from the pinned upstream datasets. The verifier treats guest identities, Ethernet frame counts and wall-clock measurements as execution evidence whose values can vary, and checks the stable numerical and application-counter fields against the reference run. No accelerator or external service is used by the guests.

The commissioning researcher is the responsible contributor. The manuscript uses an anonymous byline. An AI coding and writing assistant developed the manuscript and the experimental software, ran the cluster twice, checked the quantitative results and assembled the replication artifacts under the researcher's direction. The mathematical results are supported by prose proofs.

\begin{thebibliography}{99}
\small\RaggedRight
\bibitem{cockett} R. Cockett, G. Cruttwell, J. Gallagher, J.-S. P. Lemay,
B. MacAdam, G. Plotkin, and D. Pronk.
Reverse derivative categories.
In \emph{CSL 2020}, LIPIcs 152, article 18.
\href{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.CSL.2020.18}{doi:10.4230/LIPIcs.CSL.2020.18}.

\bibitem{cruttwell} G. S. H. Cruttwell, B. Gavranović, N. Ghani, P. Wilson, and F. Zanasi.
Categorical foundations of gradient-based learning.
In \emph{ESOP 2022}, LNCS 13240, pp.~1--28.
\href{https://link.springer.com/chapter/10.1007/978-3-030-99336-8_1}{doi:10.1007/978-3-030-99336-8\_1}.

\bibitem{utke} J. Utke, L. Hascoët, C. Hill, P. Hovland, J. Riehme, and U. Naumann.
Toward adjoinable MPI.
In \emph{PDSEC}, 2009.
\href{https://www-sop.inria.fr/tropics/papers/Utke2009TAM.html}{INRIA record}.

\bibitem{lurie} J. Lurie. \emph{Higher Topos Theory}.
Annals of Mathematics Studies 170, Princeton University Press, 2009.
\href{https://arxiv.org/abs/math/0608040}{arXiv:math/0608040}.

\bibitem{bottou} L. Bottou, F. E. Curtis, and J. Nocedal.
Optimization methods for large-scale machine learning.
\emph{SIAM Review} 60(2), pp.~223--311, 2018.
\href{https://arxiv.org/abs/1606.04838}{doi:10.1137/16M1080173}.

\bibitem{fritz} T. Fritz.
A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics.
\emph{Advances in Mathematics} 370, 107239, 2020.
\href{https://arxiv.org/abs/1908.07021}{arXiv:1908.07021}.

\bibitem{holland} J. H. Holland.
\emph{Adaptation in Natural and Artificial Systems}.
MIT Press, 1992.
\href{https://doi.org/10.7551/MITPRESS/1090.001.0001}{MIT Press edition}.

\bibitem{freund} Y. Freund and R. E. Schapire.
A decision-theoretic generalization of on-line learning and an application to boosting.
\emph{Journal of Computer and System Sciences} 55(1), pp.~119--139, 1997.
\href{https://doi.org/10.1006/jcss.1997.1504}{doi:10.1006/jcss.1997.1504}.

\bibitem{warmuth} M. K. Warmuth, K. A. Glocer, and S. V. N. Vishwanathan.
Entropy regularized LPBoost.
In \emph{ALT 2008}, LNCS 5254, pp.~256--271.
\href{https://link.springer.com/chapter/10.1007/978-3-540-87987-9_23}{doi:10.1007/978-3-540-87987-9\_23}.

\bibitem{iris} R. A. Fisher.
Iris. UCI Machine Learning Repository, 1988.
\href{https://doi.org/10.24432/C56C76}{doi:10.24432/C56C76}.
\href{https://archive.ics.uci.edu/ml/machine-learning-databases/iris/bezdekIris.data}{Corrected data file}.

\bibitem{digits} E. Alpaydın and C. Kaynak.
Optical Recognition of Handwritten Digits.
UCI Machine Learning Repository, 1998.
\href{https://doi.org/10.24432/C50P49}{doi:10.24432/C50P49}.
\href{https://archive.ics.uci.edu/ml/machine-learning-databases/optdigits/optdigits.tes}{Data file}.

\bibitem{fashion} H. Xiao, K. Rasul, and R. Vollgraf.
Fashion-MNIST, a novel image dataset for benchmarking machine learning algorithms, 2017.
\href{https://arxiv.org/abs/1708.07747}{arXiv:1708.07747}.

\bibitem{rahimi} A. Rahimi and B. Recht.
Random features for large-scale kernel machines.
In \emph{Advances in Neural Information Processing Systems 20}, 2007.
\href{https://papers.neurips.cc/paper_files/paper/2007/hash/013a006f03dbc5392effeb8f18fda755-Abstract.html}{NeurIPS proceedings}.

\bibitem{qemu} QEMU Project.
\emph{Direct Linux Boot} and \emph{Network emulation}.
\href{https://www.qemu.org/docs/master/system/linuxboot.html}{Boot documentation}
and \href{https://www.qemu.org/docs/master/system/devices/net.html}{network documentation}.
\end{thebibliography}
\end{document}