Independent scholarship · Open accessThe monthly digest

Indexed text attachment

operads-modules-machine-learning.tex

Source for Operads, Modules, and Differentiation in Machine Learning

View paper · Download original file

Lines 961–1060 of 1174. Source text is displayed literally.

961\]962Since \(\sum_y K(y\mid\boldsymbol x)=1\), its output mass is963\(\prod_i\epsilon_{X_i}(a_i)\). Hence the operation restricts964to \(M_Y\) whenever a chosen input lies in \(M_{X_i}\). These965one-marked-input restrictions satisfy the module axioms by966restricting the algebra substitution identities. In particular967the first-order change in the result under varying one968probability law is the marked-input map on its signed969zero-mass perturbation. Baseline laws are nonnegative and970normalized. At a boundary point of the probability simplex,971a zero-mass direction can be feasible in only one sign.972 973If \(M_X\) is equipped with \(\|m\|_{\rm TV}=\tfrac12\sum_x|m(x)|\),974the marked-input map at probability laws in the remaining slots975has operator norm at most the diameter976\(\sup_{\boldsymbol x,\boldsymbol x'}\|K(-\mid\boldsymbol x)-977K(-\mid\boldsymbol x')\|_{\rm TV}\). Indeed a nonzero \(m\)978is \(\|m\|_{\rm TV}\) times the difference of two probability979laws obtained by normalizing its positive and negative parts.980The two outputs after replacing the marked input are convex981mixtures of rows of \(K\). Their distance is bounded by that982diameter, by comparing all pairs of rows with the product983mixture weights. This is the contraction factor used in the984following consistency test.985 986\subsection{Stochastic substitution and answer consistency}987 988Bottman and Richardson model question decomposition by a989question operad. Their operations are question templates, and990composition plugs the output of one question into a blank991of another. They propose992comparing answers across partial evaluations of a decomposition993tree \cite{bottman2026}. Bottman, Liu, and Richardson report a994companion empirical study of this consistency signal on995multi-hop question-answering datasets \cite{bottmanliu2026}.996One mathematical use of an operadic specification is to say997which evaluation orders should agree under a particular998answer semantics. The following bound quantifies the propagation999of local discrepancies for finite stochastic evaluations.1000 1001Fix a rooted tree with a nonempty finite answer set \(X_v\) at each vertex.1002For an internal vertex \(v\), let \(I_v\) be the product of the1003answer sets of its children. Two evaluators have local Markov1004kernels \(K_v,\widetilde K_v\) from \(I_v\) to distributions on1005\(X_v\). At a leaf \(I_v\) is a singleton and the kernels specify1006its two answer laws. Assume the child evaluations are sampled1007independently for each evaluator, with original query and1008external random context fixed. At each vertex the evaluator1009applies its kernel to the product of those laws. Write1010\(\mu_v,\widetilde\mu_v\) for the resulting output laws. For1011probability laws on a finite set let \(\|\cdot\|_{\rm TV}\)1012denote total variation distance, defined as the supremum of the1013absolute difference in probability over all events. Put1014\begin{align*}1015\delta_v1016 &= \sup_{i\in I_v}1017     \|K_v(i)-\widetilde K_v(i)\|_{\rm TV},\\1018\eta_v1019 &= \sup_{i,j\in I_v}1020     \|K_v(i)-K_v(j)\|_{\rm TV}.1021\end{align*}1022Thus \(0\leq\eta_v\leq 1\).1023 1024\begin{proposition}[Propagation of discrepancies]1025For the root \(r\), the propagated discrepancy satisfies1026\begin{equation}1027\|\mu_r-\widetilde\mu_r\|_{\rm TV}1028 \leq1029 \sum_v \delta_v1030    \prod_{w\in\operatorname{Anc}(v)} \eta_w ,1031\label{eq:ai-tv-tree}1032\end{equation}1033where \(\operatorname{Anc}(v)\) consists of strict ancestors1034and an empty product is one.1035\end{proposition}1036 1037\begin{proof}1038Let \(e_v=\|\mu_v-\widetilde\mu_v\|_{\rm TV}\) and let1039\(\alpha_v,\widetilde\alpha_v\) be the two laws on \(I_v\).1040Independent maximal couplings of the child laws give1041\(\|\alpha_v-\widetilde\alpha_v\|_{\rm TV}1042 \leq \sum_{u}e_u\), summed over children \(u\).1043Apply \(K_v\) to a maximal coupling of the two input laws.1044When the two inputs agree, couple the output samples to be1045equal. When they differ, the two conditional output laws have a1046coupling with mismatch probability at most \(\eta_v\).1047This bounds the propagated term by1048\(\eta_v\|\alpha_v-\widetilde\alpha_v\|_{\rm TV}\).1049Replacing \(K_v\) by \(\widetilde K_v\) at fixed input law adds1050at most \(\delta_v\) by the triangle inequality under mixing.1051Hence \(e_v\leq\delta_v+\eta_v\sum_u e_u\).1052Iterating this recurrence gives \eqref{eq:ai-tv-tree}.1053\end{proof}1054 1055For example, if two leaves perturb their distributions by at1056most \(0.05\) each and a common parent uses a kernel with1057\(\eta_v=0.2\), its output moves by at most \(0.02\).1058If each row of the parent kernel also changes by at most1059\(0.01\), the bound increases to \(0.03\). The estimate holds for pairwise1060comparisons after specifying a common evaluation tree and the