Independent scholarship · Open accessThe monthly digest

Indexed text attachment

operads-modules-machine-learning.tex

Source for Operads, Modules, and Differentiation in Machine Learning

View paper · Download original file

Lines 861–960 of 1174. Source text is displayed literally.

861exactly the next kernels. Their composites vanish since \(u^p=0\).862Tensoring with the trivial module \(k\) makes both differentials863zero, whence \(\operatorname{Tor}^{A}_q(k,k)\cong k\) for every864\(q\geq 0\).865 866For the preceding action of \(C_2\) on \(\{a,b,c\}\) over867\(\mathbb F_2\), the swapped orbit is free and the fixed slot868has stabilizer \(C_2\). Proposition~\ref{prop:derived-orbits} gives869two degree-zero classes and one class in each positive degree.870Directly, on the fixed-slot summand the augmented group algebra871has square-zero augmentation ideal \((u)\). In the normalized872bar the endpoint actions and every internal multiplication873vanish, leaving one copy of \(k\) in each bar degree. Thus ordinary874orbit pooling accounts for the same number of coordinate classes875over both fields, and the chain-valued modular quotient carries876additional information about stabilizers. Over \(\mathbb R\),877Proposition~\ref{prop:derived-averaging} reduces the derived878quotient to ordinary balancing.879 880For an explicit invariant feature in this modular setting,881take the dimensions \(h_j\) of the first two homology groups882of the quotient complex. They depend only on the isomorphism883class of the finite \(G\)-set. A single free \(C_2\)-orbit and884a single fixed point both have \(h_0=1\) over \(\mathbb F_2\).885They have \(h_1=0\) and \(h_1=1\), respectively, so a real886linear readout of \((h_0,h_1)\) can distinguish these two887inputs after the homology computation. This example supplies888an exact representation statistic for finite symmetry data.889Its value as a feature in any larger learning problem depends890on whether the target depends on the stabilizers in the input.891\section{Compositional AI and related work}892\label{sec:ai-applications}893 894\subsection{Wiring and differentiable semantics}895 896An operad specifies which interfaces can be substituted into897one another. An algebra specifies what a substitution does to898data. Vagner, Spivak, and Lerman give this separation a concrete899form for open dynamical systems: their wiring diagrams compose900syntactically, and an algebra interprets the diagrams by wiring901together systems of differential equations902\cite[Sec.~3 and Prop.~4.5]{vagner2015}. In neural programs, the903analogous syntax can record port types, parameter occurrences,904and which subprograms consume each intermediate value. When a905wire feeds multiple consumers, the reverse pass sums all the906cotangents returned along that wire. Consequently, substitutions907can be compiled while keeping the equality of that local sum908with the pullback along the forward wiring map.909 910Gavranovi\'c et al.\ treat constraints and implementations of911deep learning architectures by monads and endofunctor algebras in912a \(2\)-category of parametric maps \cite{gavranovic2024}.913This accounts for recurrent architectures and gives an algebraic914language for equivariance. The chain-complex setting used here915specifies multilinear operations and modules over a fixed916algebra, so explicit passage to an algebra and its module data917is needed when applying it to a smooth parameterized program.918For example, a differentiated relation between two919substitutions compares their effects on inputs and parameter920directions. Such a relation is additional information about921which perturbations preserve the algebra.922 923Fong, Spivak, and Tuy\'eras give a symmetric monoidal functor924from parametrized differentiable functions to learners under925an invertibility hypothesis for the scalar error derivative926\cite[Thm.~III.2]{fong2019}. Cruttwell et al.\ encode reverse927derivatives as lenses and thereby treat models, losses, and928optimizers as composable data \cite[Secs.~2--3]{cruttwell2022}.929Spivak gives operadic dynamics of smooth adaptive arrangements,930with gradient descent as one of the semantics931\cite{spivak2026}. Those formulations describe learning dynamics.932Here the cotangent pullback is the dual of the linearized933substitution. Turning the resulting covector into a parameter934update also requires a choice of metric or optimizer.935 936\subsection{A module of probability perturbations}937\label{sec:stochastic-module}938 939For nonempty finite sets \(X_1,\dots,X_n,Y\), a stochastic operation \(K\)940assigns a probability measure \(K(-\mid\boldsymbol x)\) on \(Y\)941to each input tuple. To substitute a stochastic operation942\(H_i\) into slot \(i\), first sample their results independently943conditional on their respective input blocks \(\boldsymbol u_i\).944The substituted kernel is945\[946 (K(H_1,\ldots,H_n))(y\mid\boldsymbol u)947   =\sum_{\boldsymbol x}K(y\mid\boldsymbol x)948         \prod_i H_i(x_i\mid\boldsymbol u_i).949\]950Rearranging the finite sums verifies associativity. Permuting951inputs reindexes the sums, and identity kernels are units.952Taking these kernels as operations therefore defines a colored953operad in sets.954 955Let \(A_X=\mathbb R^{X}\) be the vector space of signed measures,956\(\epsilon_X(a)=\sum_{x\in X}a(x)\), and \(M_X=\ker\epsilon_X\).957The linearization of the kernel operad acts on \(A\) by958\[959 K_A(a_1,\ldots,a_n)(y)960   =\sum_{\boldsymbol x}K(y\mid\boldsymbol x)\prod_i a_i(x_i).