Indexed text attachment
operads-modules-machine-learning.tex
Source for Operads, Modules, and Differentiation in Machine Learning
View paper · Download original file
Lines 141–240 of 1174. Source text is displayed literally.
141identification of categories in Part~I, Definition~4.9 and142Proposition~4.10. The associative and commutative examples143give \(A\otimes A^{\mathrm{op}}\) and \(A\), respectively144(Part~I, Examples~4.11).145 146\subsection{Square-zero extensions}147 148\begin{proposition}149\label{prop:operadic-square-zero}150An \(\mathcal O\)-module \(M\) over \(A\) determines a split151\(\mathcal O\)-algebra extension \(A\oplus M\to A\) on which152every operation with at least two inputs in \(M\) is zero.153Conversely, a split extension with this vanishing property154determines an \(A\)-module on its kernel.155\end{proposition}156 157\begin{proof}158Define the operations on \(A\oplus M\) by multilinearity.159With every argument in \(A\), use \(\theta_n\) and include160the result in the \(A\)-summand. With one argument in \(M\),161use \(\lambda_{n,i}\) and include the result in the \(M\)-summand.162With two or more arguments in \(M\), set the result to zero.163The individual actions are chain maps and are equivariant,164so the extended operations have both properties. To check165substitution, expand the inputs into their summands.166With zero marked inputs, the two evaluations agree by the167algebra axiom. With one marked input, they agree by the module168axiom. With two or more, either one inner subtree has at least169two marked inputs and vanishes, or at least two inner subtrees170supply \(M\)-inputs to the outer operation, which then vanishes.171The operadic unit acts as the identity on both summands, and172the projection and inclusion are algebra maps.173 174For an extension \(p:E\to A\) split by an algebra map, use175the splitting to write \(E=A\oplus\ker p\) as a complex.176An operation with one input in \(\ker p\) lands in \(\ker p\)177by multilinearity and compatibility with \(p\).178The operadic axioms on these operations are the module axioms.179The vanishing requirement determines the remaining operations,180so the constructions are inverse under the splitting.181\end{proof}182 183Suppose \(s(a)=(a,D(a))\) is a chain map section of184\(A\oplus M\to A\). For \(s\) to preserve the operations185is precisely to satisfy the following derivation law:186\begin{equation}187\begin{aligned}188 D\theta_n(c;a_1,\ldots,a_n)189 =\sum_{i=1}^{n}190 \lambda_{n,i}(c;a_1,\ldots,m_i,\ldots,a_n),\\[-2pt]191 m_i=D(a_i).192\end{aligned}193\label{eq:operadic-derivation}194\end{equation}195The empty sum for \(n=0\) makes \(D\) vanish on nullary196operations. Conceptually, the sum arises from expanding on197each leaf of an operation tree and retaining terms with one mark.198Induction over grafting gives the derivative of a composite as199the sum of the linear operators obtained by propagating each200mark along its path to the root. If several leaves use the same201parameter, all its occurrences contribute to the resulting202linear map of the shared perturbation.203 204\begin{example}[Two-sided sensitivity in a tied matrix block]205Take \(A=M_r(\mathbb R)\) with its usual multiplication and206use the regular bimodule \(M=A\). For fixed \(U,V\in A\),207let \(F(W)=UWV+W^2\). In the associative square-zero208extension, the second component of \(F(W,H)\) is209\[210 J_F(W)[H]=UHV+WH+HW .211\]212Each summand acts by a two-sided context \(H\mapsto aHb\),213represented by \(a\otimes b^{\mathrm{op}}\) in214\(A\otimes A^{\mathrm{op}}\). The first summand uses one215occurrence of \(W\) and the square uses two occurrences.216For the Frobenius pairing \(\langle G,H\rangle217=\operatorname{tr}(G^{\mathsf T}H)\), cyclicity of the trace gives218\[219 J_F(W)^*[G]=U^{\mathsf T}GV^{\mathsf T}220 +W^{\mathsf T}G+GW^{\mathsf T}.221\]222In a block followed by a scalar loss, \(G\) is the incoming223matrix cotangent. The adjoint sends it back to the shared224matrix parameter and adds the contributions of its occurrences.225\end{example}226 227\subsection{The scope of homological conversion}228 229The paired monad of Part~I, Definition~4.3 and Proposition~4.4,230assigns \((X,Y)\) the pair \((T_{\mathcal O}(X),V_X\otimes Y)\),231where \(V_X\) is the same marked-slot expression with \(X\)232in place of \(A\). Its algebras comprise an operadic algebra233and a module over it. In Part~II, Section~4, the authors234use the two-sided monadic bar construction on such data.235For a monad \(T\), a right \(T\)-functor \(F\), and a236\(T\)-algebra \(A\), the simplicial degree \(q\) is \(F T^q A\).237Its first and last faces use the right action and algebra238action, respectively. Interior faces use monad multiplication,239and degeneracies use the unit. The augmentation of240\(B_\bullet(T,T,A)\) has a simplicial contraction after forgetting