Independent scholarship · Open accessThe monthly digest

Indexed text attachment

operads-modules-machine-learning.tex

Source for Operads, Modules, and Differentiation in Machine Learning

View paper · Download original file

Lines 81–180 of 1174. Source text is displayed literally.

81 \longrightarrow \mathcal O\bigl(\textstyle\sum_i n_i\bigr).82\]83The associativity, unit, and equivariance axioms make substitution84independent of how a tree of operations is assembled. Interchanging85factors of degrees \(p\) and \(q\) produces the Koszul sign86\((-1)^{pq}\). An \(\mathcal O\)-algebra \(A\) has equivariant87chain maps \(\theta_n:\mathcal O(n)\otimes A^{\otimes n}\to A\)88compatible with substitution and the unit. These are Definitions~1.189and 2.1 of Part~I of the monograph~\cite{kriz-may}.90 91The free algebra monad on complexes is92\[93 T_{\mathcal O}(X)=94 \bigoplus_{n\geq 0}\mathcal O(n)\otimes_{k[\Sigma_n]}X^{\otimes n}.95\]96Its multiplication substitutes operations, and its algebra axiom97expresses the compatibility of \(\theta\) with that substitution.98Part~I, Definition~3.3 and Proposition~3.4, identify99\(\mathcal O\)-algebras with algebras over this monad. The tensor100powers make the structure maps multilinear in their input vectors.101A nonlinear smooth primitive therefore calls for a semantic102encoding before these structure maps can represent it.103 104\subsection{The marked slot and the enveloping algebra}105 106For a fixed \(\mathcal O\)-algebra \(A\), a module \(M\) in the107sense of K\v{r}\'{\i}\v{z} and May has chain maps108\begin{equation}109 \lambda_n:\mathcal O(n)\otimes A^{\otimes(n-1)}110                 \otimes M\longrightarrow M,\qquad n\geq 1 .111 \label{eq:operadic-module}112\end{equation}113Here the last slot is marked. Equivariance under \(\Sigma_{n-1}\)114permutes the unmarked slots. To put \(M\) in position \(i\), use the115permutation moving that position to the last slot, with the116corresponding action on \(\mathcal O(n)\) and the Koszul signs.117Write the resulting operation as \(\lambda_{n,i}\).118Equivariance under the subgroup fixing the mark makes this119description independent of the permutation used. When such120operations are nested, every subtree outside the path from121the mark to the root is evaluated in \(A\). The operations along122the marked path act on \(M\). The module axiom identifies this123evaluation with applying the composite operation once124(Part~I, Definition~4.1).125 126Define the complex of raw marked operations by127\[128 V_A=\bigoplus_{n\geq 1}129       \mathcal O(n)\otimes_{k[\Sigma_{n-1}]}A^{\otimes(n-1)} .130\]131The action maps assemble into a single map \(V_A\otimes M\to M\). The universal enveloping algebra \(U_{\mathcal O}(A)\)132is the quotient of the tensor algebra on \(V_A\) by the relations133for the unit and for operadic substitution. A product of two134marked operations represents insertion of the inner operation135at the mark of the outer one. Substitutions in unmarked slots136can be evaluated in \(A\) before the marked operation acts.137The maps in~\eqref{eq:operadic-module} give a representation of the138tensor algebra, and the module axioms make it factor through the139quotient. Conversely, restricting a \(U_{\mathcal O}(A)\)-action140to \(V_A\) recovers the maps and their axioms. This proves the141identification of categories in Part~I, Definition~4.9 and142Proposition~4.10. The associative and commutative examples143give \(A\otimes A^{\mathrm{op}}\) and \(A\), respectively144(Part~I, Examples~4.11).145 146\subsection{Square-zero extensions}147 148\begin{proposition}149\label{prop:operadic-square-zero}150An \(\mathcal O\)-module \(M\) over \(A\) determines a split151\(\mathcal O\)-algebra extension \(A\oplus M\to A\) on which152every operation with at least two inputs in \(M\) is zero.153Conversely, a split extension with this vanishing property154determines an \(A\)-module on its kernel.155\end{proposition}156 157\begin{proof}158Define the operations on \(A\oplus M\) by multilinearity.159With every argument in \(A\), use \(\theta_n\) and include160the result in the \(A\)-summand. With one argument in \(M\),161use \(\lambda_{n,i}\) and include the result in the \(M\)-summand.162With two or more arguments in \(M\), set the result to zero.163The individual actions are chain maps and are equivariant,164so the extended operations have both properties. To check165substitution, expand the inputs into their summands.166With zero marked inputs, the two evaluations agree by the167algebra axiom. With one marked input, they agree by the module168axiom. With two or more, either one inner subtree has at least169two marked inputs and vanishes, or at least two inner subtrees170supply \(M\)-inputs to the outer operation, which then vanishes.171The operadic unit acts as the identity on both summands, and172the projection and inclusion are algebra maps.173 174For an extension \(p:E\to A\) split by an algebra map, use175the splitting to write \(E=A\oplus\ker p\) as a complex.176An operation with one input in \(\ker p\) lands in \(\ker p\)177by multilinearity and compatibility with \(p\).178The operadic axioms on these operations are the module axioms.179The vanishing requirement determines the remaining operations,180so the constructions are inverse under the splitting.