Indexed text attachment
operads-modules-machine-learning.tex
Source for Operads, Modules, and Differentiation in Machine Learning
View paper · Download original file
Lines 761–860 of 1174. Source text is displayed literally.
761acts orthogonally, uniqueness of the Riesz representative762gives \(g\nabla L(p)=\nabla L(p)\). Consequently the induced763gradient agrees with the ambient gradient after inclusion by764\(i\), and the product-metric gradient still includes the765orbit-size factors. This identifies the metric implicit in766restricting Euclidean gradient flow to the tied subspace.767 768Tying also has a direct architectural consequence. Suppose a769parameterized family \(F:P\times X\to Y\) satisfies diagonal770equivariance, \(F(gp,gx)=gF(p,x)\), for given \(G\)-actions on771\(X,Y\). Each tied parameter \(i\theta\) gives an equivariant772model, since \(g(i\theta)=i\theta\) and thus773\(F(i\theta,gx)=gF(i\theta,x)\). The hypothesis is a property of774the untied family, checkable at its generating operations.775 776\subsection{The two-sided bar and stabilizer homology}777 778The balancing construction is the degree-zero part of a derived779tensor product. K\v{r}\'i\v{z} and May develop derived tensor780products of differential graded modules by cell approximation in781Part~III, \S4, starting on p.~53 \cite{kriz-may}. For augmented782algebras the two-sided tensor bar, reviewed in Part~IV, \S6,783p.~71, supplies a concrete model. We use its simplicial formula784for the associative algebra \(A=k[G]\), with homological degrees785increasing with bar length. Their operadic comparison in786Part~II, \S4, p.~35 uses a monad bar \(B(F,C,A)\) with operadic787monad \(C\) and a \(C\)-algebra \(A\). The bar here uses tensor788powers of the group algebra and its ordinary right and left789modules.790 791Explicitly, use \(B_q(k,A,M)=k\otimes_k A^{\otimes q}\otimes_k M\)792with differential given by the alternating sum of faces. The first793face applies the augmentation to the first algebra factor,794interior faces multiply two adjacent factors, and the last face795acts on \(M\). Degeneracies insert the unit. Replacing \(M\) by796\(A\) gives a free right \(A\)-module in every degree. The797augmented underlying complex contracts by appending \(1\in A\)798as an extra last factor, with sign \((-1)^{q+1}\) in degree \(q\).799Indeed the last face of this extra degeneracy is the identity800and every earlier face commutes with it in the adjacent degree.801Thus \(B_\bullet(k,A,A)\to k\) is a free resolution, and tensoring802it over \(A\) with \(M\) gives803\[804 H_q(B_\bullet(k,A,M))\cong805 \operatorname{Tor}_q^A(k,M).806\]807The degree-one differential sends a generator808\([g]\otimes m\) to \(m-gm\), giving the ordinary quotient in809degree zero. One can also use normalized chains, since810the degenerate subcomplex of a simplicial \(k\)-module is811contractible.812 813For a permutation module \(M=k[S]\), this complex is the chain814complex of the nerve of the action groupoid \(G\ltimes S\).815For the given face convention, the generator816\((g_1,\ldots,g_q,s)\) corresponds to the string with terminal817object \(s\) and successive arrow labels818\(g_1^{-1},\ldots,g_q^{-1}\), beginning at \(g_1\cdots g_qs\).819Deleting the first object drops \(g_1\). Deleting the last820replaces \(s\) by \(g_qs\). Composing adjacent arrows takes821\(g_i,g_{i+1}\) to \(g_i g_{i+1}\) in the bar coordinates.822These are exactly the three types of faces above. The823groupoid describes how the identification is made, retaining824the automorphisms of a representative of each orbit.825 826\begin{proposition}[Orbit decomposition]827\label{prop:derived-orbits}828For a finite \(G\)-set \(S\), taking a representative \(s\) in829each orbit gives830\[831 \operatorname{Tor}^{k[G]}_q(k,k[S])832 \cong \bigoplus_{[s]\in S/G}833 \operatorname{Tor}^{k[G_s]}_q(k,k).834\]835\end{proposition}836 837\begin{proof}838The map \(g\otimes 1\mapsto gs\) identifies the induced module839\(k[G]\otimes_{k[G_s]} k\) with the span of the orbit of \(s\).840The translates of \(s\) are indexed by left cosets \(gG_s\).841If \(Q_\bullet\to k\) is a projective right \(k[G]\)-module842resolution, it remains a projective resolution over \(k[G_s]\)843because \(k[G]\) is free on the right over \(k[G_s]\). For each844orbit, tensor associativity identifies845\[846 \begin{aligned}847 &Q_\bullet\otimes_{k[G]}848 (k[G]\otimes_{k[G_s]} k)\849 &\qquad \cong Q_\bullet\otimes_{k[G_s]}k .850 \end{aligned}851\]852Taking homology and the sum over orbits proves the formula.853\end{proof}854 855Let \(\operatorname{char}(k)=p\) and \(G=C_p=\langle g\rangle\).856Writing \(u=g-1\), one has \(A=k[u]/(u^p)\), and the augmentation857sends \(u\) to zero. A free resolution of \(k\) alternates858multiplication by \(u\) in odd degrees and by \(u^{p-1}\) in859positive even degrees. The kernels of these two multiplications860are \((u^{p-1})\) and \((u)\), respectively, so the images are