Indexed text attachment
operads-modules-machine-learning.tex
Source for Operads, Modules, and Differentiation in Machine Learning
View paper · Download original file
Lines 741–840 of 1174. Source text is displayed literally.
741Indeed for \(s\in O\) every index \(t\in O\) occurs as \(g^{-1}s\)742for exactly \(|G_s|\) group elements, so \(R_G\) averages743uniformly over \(O\). This is also the orthogonal projection744onto \(P^G\): the operator is idempotent, and reindexing by745\(g^{-1}\) gives self-adjointness.746 747For example, let \(C_2\) swap \(a,b\) and fix \(c\), and take748\(V=\mathbb R\). The map \(i\) sends \((\theta,\eta)\) to749\((\theta,\theta,\eta)\). For \(y=(1,3,2)\) and750\(L(p)=\tfrac12\sum_{s\in S}(p_s-y_s)^2\), the product-metric751gradient in \(U\) is \((2\theta-4,\eta-2)\), while the induced752metric gives \((\theta-2,\eta-2)\). At the origin the first753shared coordinate receives two contributions to the derivative,754and the induced metric divides that sum by the orbit size.755This difference matters when a single learning rate is used756across orbits of unequal sizes.757 758When the ambient loss is invariant, its ambient gradient at a759tied point is itself fixed. Differentiating \(L\circ g=L\)760at \(p\in P^G\) gives \(DL(p)[g v]=DL(p)[v]\). Since \(g\)761acts orthogonally, uniqueness of the Riesz representative762gives \(g\nabla L(p)=\nabla L(p)\). Consequently the induced763gradient agrees with the ambient gradient after inclusion by764\(i\), and the product-metric gradient still includes the765orbit-size factors. This identifies the metric implicit in766restricting Euclidean gradient flow to the tied subspace.767 768Tying also has a direct architectural consequence. Suppose a769parameterized family \(F:P\times X\to Y\) satisfies diagonal770equivariance, \(F(gp,gx)=gF(p,x)\), for given \(G\)-actions on771\(X,Y\). Each tied parameter \(i\theta\) gives an equivariant772model, since \(g(i\theta)=i\theta\) and thus773\(F(i\theta,gx)=gF(i\theta,x)\). The hypothesis is a property of774the untied family, checkable at its generating operations.775 776\subsection{The two-sided bar and stabilizer homology}777 778The balancing construction is the degree-zero part of a derived779tensor product. K\v{r}\'i\v{z} and May develop derived tensor780products of differential graded modules by cell approximation in781Part~III, \S4, starting on p.~53 \cite{kriz-may}. For augmented782algebras the two-sided tensor bar, reviewed in Part~IV, \S6,783p.~71, supplies a concrete model. We use its simplicial formula784for the associative algebra \(A=k[G]\), with homological degrees785increasing with bar length. Their operadic comparison in786Part~II, \S4, p.~35 uses a monad bar \(B(F,C,A)\) with operadic787monad \(C\) and a \(C\)-algebra \(A\). The bar here uses tensor788powers of the group algebra and its ordinary right and left789modules.790 791Explicitly, use \(B_q(k,A,M)=k\otimes_k A^{\otimes q}\otimes_k M\)792with differential given by the alternating sum of faces. The first793face applies the augmentation to the first algebra factor,794interior faces multiply two adjacent factors, and the last face795acts on \(M\). Degeneracies insert the unit. Replacing \(M\) by796\(A\) gives a free right \(A\)-module in every degree. The797augmented underlying complex contracts by appending \(1\in A\)798as an extra last factor, with sign \((-1)^{q+1}\) in degree \(q\).799Indeed the last face of this extra degeneracy is the identity800and every earlier face commutes with it in the adjacent degree.801Thus \(B_\bullet(k,A,A)\to k\) is a free resolution, and tensoring802it over \(A\) with \(M\) gives803\[804 H_q(B_\bullet(k,A,M))\cong805 \operatorname{Tor}_q^A(k,M).806\]807The degree-one differential sends a generator808\([g]\otimes m\) to \(m-gm\), giving the ordinary quotient in809degree zero. One can also use normalized chains, since810the degenerate subcomplex of a simplicial \(k\)-module is811contractible.812 813For a permutation module \(M=k[S]\), this complex is the chain814complex of the nerve of the action groupoid \(G\ltimes S\).815For the given face convention, the generator816\((g_1,\ldots,g_q,s)\) corresponds to the string with terminal817object \(s\) and successive arrow labels818\(g_1^{-1},\ldots,g_q^{-1}\), beginning at \(g_1\cdots g_qs\).819Deleting the first object drops \(g_1\). Deleting the last820replaces \(s\) by \(g_qs\). Composing adjacent arrows takes821\(g_i,g_{i+1}\) to \(g_i g_{i+1}\) in the bar coordinates.822These are exactly the three types of faces above. The823groupoid describes how the identification is made, retaining824the automorphisms of a representative of each orbit.825 826\begin{proposition}[Orbit decomposition]827\label{prop:derived-orbits}828For a finite \(G\)-set \(S\), taking a representative \(s\) in829each orbit gives830\[831 \operatorname{Tor}^{k[G]}_q(k,k[S])832 \cong \bigoplus_{[s]\in S/G}833 \operatorname{Tor}^{k[G_s]}_q(k,k).834\]835\end{proposition}836 837\begin{proof}838The map \(g\otimes 1\mapsto gs\) identifies the induced module839\(k[G]\otimes_{k[G_s]} k\) with the span of the orbit of \(s\).840The translates of \(s\) are indexed by left cosets \(gG_s\).