Indexed text attachment
operads-modules-machine-learning.tex
Source for Operads, Modules, and Differentiation in Machine Learning
View paper · Download original file
Lines 1081–1174 of 1174. Source text is displayed literally.
1081 1082The application to smooth architecture evaluation concerns the degree-zero part of the monograph's setting. The passage to \(\mathbb R[P]\) ensures that the module operators are defined on real chain complexes concentrated in degree zero. It gives a model for organizing derivatives and a check on compositional implementations. The original feature operations may be nonlinear. For a piecewise smooth layer the calculation applies on regions of differentiability. A kink, such as that of ReLU at zero, requires an additional derivative convention.1083 1084There are several different meanings of tangent structure. In the smooth calculation, a tangent space \(T_x X\) consists of directions in a parameter or state space. In the Quillen-theoretic setting, the tangent \(\infty\)-category of a presentable \(\infty\)-category at an object is the stabilization of the slice. For operadic algebras under stable hypotheses, its comparison to operadic modules is a theorem about that stabilization~\cite[Corollary~1.0.5]{harpaz2019}. This supports the role of operadic modules in homotopical linearization. Applying that theorem to learning requires a model of the architecture or data in the specified category, as well as a definition of the perturbations to be measured. The present paper directly establishes the smooth derivative interpretation and computes a separate derived symmetry example.1085 1086The clearest uses of the monograph in machine learning are the interpretation of typed architecture substitutions, the module law for one-input sensitivities, and the derived analysis of symmetry relations when ordinary balancing loses homology. For AI systems whose components return stochastic outputs, agreement between two routes is an empirical question about suitably typed kernels. The consistency calculation in Section~\ref{sec:ai-applications} supplies a diagnostic with clear sampling assumptions. The bridge in both deterministic and stochastic cases comes from the marked-slot module and the precise rule for composing it.1087 1088\paragraph{Reproducibility.} The source bundle contains the full LaTeX and a standard-library Python program for the worked numerical derivative and symmetry checks in Section~\ref{sec:examples}. These checks supplement the proofs and are limited to the worked examples.1089 1090\paragraph{Authorship.} The public byline is Anonymous. A language-model assistant helped draft, typeset, check, and revise the manuscript under the submitter's direction.1091 1092\begin{thebibliography}{12}1093\small1094\setlength{\itemsep}{2pt}1095 1096\bibitem{kriz-may}1097I.~K\v{r}\'{\i}\v{z} and J.~P. May.1098\newblock \emph{Operads, algebras, modules and motives}.1099\newblock Ast\'erisque 233, Soci\'et\'e Math\'ematique de France, 1995.1100\newblock \url{https://www.numdam.org/item/AST_1995__233__1_0/}.1101Page locators in this paper follow the authors' 109-page version,1102\url{https://math.uchicago.edu/~may/BOOKS/KM.pdf}.1103 1104\bibitem{fong2019}1105B. Fong, D.~I. Spivak, and R. Tuy\'eras.1106\newblock Backprop as functor: A compositional perspective on supervised learning.1107\newblock In \emph{Proc. 34th Annual ACM/IEEE Symposium on Logic in Computer Science}, 2019.1108\newblock \url{https://arxiv.org/abs/1711.10455}.1109 1110\bibitem{cruttwell2022}1111G.~S.~H. Cruttwell, B. Gavranovi\'c, N. Ghani, P. Wilson, and F. Zanasi.1112\newblock Categorical foundations of gradient-based learning.1113\newblock In \emph{Programming Languages and Systems}, LNCS 13240, pp. 1--28, 2022.1114\newblock \url{https://arxiv.org/abs/2103.01931}.1115 1116\bibitem{gavranovic2024}1117B. Gavranovi\'c, P. Lessard, A. Dudzik, T. von Glehn,1118J. G. M. Ara\'ujo, and P. Veli\v{c}kovi\'c.1119\newblock Position: Categorical deep learning is an algebraic theory of all architectures.1120\newblock In \emph{Proc. 41st International Conference on Machine Learning},1121PMLR 235, pp. 15209--15241, 2024.1122\newblock \url{https://proceedings.mlr.press/v235/gavranovic24a.html}.1123 1124\bibitem{vagner2015}1125D. Vagner, D.~I. Spivak, and E. Lerman.1126\newblock Algebras of open dynamical systems on the operad of wiring diagrams.1127\newblock \emph{Theory and Applications of Categories}, 30(51):1793--1822, 2015.1128\newblock \url{https://arxiv.org/abs/1408.1598}.1129 1130\bibitem{harpaz2019}1131Y. Harpaz, J. Nuiten, and M. Prasma.1132\newblock Tangent categories of algebras over operads.1133\newblock \emph{Israel Journal of Mathematics}, 234:691--742, 2019.1134\newblock \url{https://arxiv.org/abs/1612.02607}.1135 1136\bibitem{zaheer2017}1137M. Zaheer, S. Kottur, S. Ravanbakhsh, B. P\'oczos,1138R. Salakhutdinov, and A.~J. Smola.1139\newblock Deep Sets.1140\newblock \emph{Advances in Neural Information Processing Systems} 30, 2017.1141\newblock \url{https://arxiv.org/abs/1703.06114}.1142 1143\bibitem{spivak2026}1144D.~I. Spivak.1145\newblock Compositional dynamics in learning and mechanics.1146\newblock arXiv preprint, 2026.1147\newblock \url{https://arxiv.org/abs/2606.28984}.1148 1149\bibitem{bottman2026}1150N. Bottman and K. Richardson.1151\newblock Operads for compositional reasoning in LLMs.1152\newblock arXiv preprint, 2026.1153\newblock \url{https://arxiv.org/abs/2606.13634}.1154 1155\bibitem{bottmanliu2026}1156N. Bottman, Y. Liu, and K. Richardson.1157\newblock Operadic consistency: A label-free signal for compositional reasoning failures in LLMs.1158\newblock arXiv preprint, 2026.1159\newblock \url{https://arxiv.org/abs/2606.13649}.1160 1161\bibitem{baydin2018}1162A.~G. Baydin, B.~A. Pearlmutter, A.~A. Radul, and J.~M. Siskind.1163\newblock Automatic differentiation in machine learning: A survey.1164\newblock \emph{Journal of Machine Learning Research}, 18(153):1--43, 2018.1165\newblock \url{https://jmlr.org/papers/v18/17-468.html}.1166 1167\bibitem{fike2011}1168J.~A. Fike and J.~J. Alonso.1169\newblock The development of hyper-dual numbers for exact second-derivative calculations.1170\newblock \emph{49th AIAA Aerospace Sciences Meeting}, paper AIAA 2011-886, 2011.1171\newblock \url{https://doi.org/10.2514/6.2011-886}.1172\end{thebibliography}1173\end{document}1174