Indexed text attachment
representation-theory-of-parametrized-neural-models-on-spaces.tex
Source for Representation Theory of Parametrized Neural Models on Spaces
View paper · Download original file
Lines 281–380 of 1343. Source text is displayed literally.
281 f(gp,gx)=g f(p,x).282\end{equation}283It is a condition on the entire family. At a fixed $p$, it284forces equivariance under the stabilizer $G_p$. In a smooth285site one can instead use the classifying stack of a Lie group.286Sections of that stack are torsors with smooth local287trivializations, and \eqref{eq:equiv-eval} then holds for288smooth test families. The homotopy type of a smooth base289retains path transport and may identify smoothly distinct data.290 291\subsection{Realization on a quotient}292 293Suppose a group object $G$ acts on $P$ and a realization294$r:P\to M$ is coherently invariant with trivial action on295$M$. Here $M$ can itself be a space of functions or probability296laws. The action groupoid has simplicial levels $P\times G^n$.297Its colimit is $P//G$, and the colimit universal property gives298\[299 \Map(P//G,M)\simeq \Tot\,\Map(P\times G^\bullet,M).300\]301Thus $r$ factors through $P//G$ exactly when equipped with302descent, whose first comparison identifies $r(p)$ and $r(gp)$.303Equivalences between local parameter presentations are part304of a family in the quotient. Over $\Spc$, the fiber of305$P//G\to BG$ over the base point is the parameter space $P$,306with transport the original action. For smooth $P$ and Lie307$G$, the same construction in sheaves gives the smooth quotient308stack. The example in Section~\ref{gauge:circle} distinguishes309the quotient presentations from their common observable laws.310 311\subsection{Stochastic evaluation}312\label{sec:stochastic}313 314In a cartesian closed category, a strong monad $T$ has a315natural strength $\mathrm{st}_{A,Y}:A\times T Y\to T(A\times Y)$316compatible with products, unit $\eta$, and multiplication $\mu$.317When $T$ is a probability monad, the layer $P\times X\to T Y$318assigns a probability-valued prediction. Currying represents it319by $P\to[X,T Y]$. For320$f:P\times X\to T Y$ and $g:Q\times Y\to T Z$, their321composite with parameter $P\times Q$ is322\[323 \begin{aligned}324 P\times Q\times X &\longrightarrow Q\times TY\\325 &\xrightarrow{\ \mathrm{st}\ } T(Q\times Y)326 \xrightarrow{\ T g\ } T^2 Z327 \xrightarrow{\ \mu_Z\ } TZ ,328 \end{aligned}329\]330with first map $(p,q,x)\mapsto(q,f(p,x))$.331For a formal check, fix a context $A$ and form the category332whose arrows $X\to Y$ are maps $A\times X\to Y$. It is the333co-Kleisli category of the comonad $A\times(-)$ with diagonal334and projection. Strength is a mixed distributive law335$A\times T(-)\to T(A\times(-))$ for this comonad and the monad336$T$. Its compatibility with the diagonal and projection is337product coherence and naturality in $A$. Monad unit and338multiplication compatibility are the strength axioms.339Thus $T$ acts as a monad on the category with context $A$.340Kleisli composition for that monad extends a map341$h:A\times Y\to T Z$ using strength followed by $T h$ and342$\mu_Z$. Taking $A=P\times Q$ and restricting the layers along343its two projections gives the displayed formula. Context maps344preserve the construction by naturality of strength.345A compatible system of monads on slices gives the same346construction over $B$ when their strengths and monad347operations commute with base change.348 349For instance, the finitely supported distribution monad on sets350gives finite channels by restricting the feature and output sets351to be finite. If $p,q$ are fixed, the composite has law352\[353 k_{p,q}(z\mid x)=\sum_y g_q(z\mid y) f_p(y\mid x).354\]355Dirac laws embed deterministic evaluations. Repeated sampling can356share an unknown parameter by pulling a product family back along357its diagonal. For a coin of bias $1/5$ or $4/5$ with equal prior358probability, shared-parameter sampling gives probability $17/50$359of two successes. Independently drawing the parameter for each360observation gives $1/4$. The single-observation law therefore361needs to be supplemented by a parameter sharing convention.362 363For smooth conditional models, the probability-valued evaluator364can be defined on a represented family of probability laws.365The next section uses finite outcome laws with positive366probabilities, where the open simplex is a manifold and the367Fisher tensor can be computed in ordinary coordinates.368\par369\section{Statistical and manifold models}370\label{stat:section}371 372A parametrized smooth map \(F:P\times X\to M\) is a manifold-valued373prediction, with a family of maps \(F_p:X\to M\). A deterministic network374may also retain intermediate feature bundles by factoring \(F\) through375smooth maps of total spaces. With an ordinary parameter manifold and no376prescribed equivalences of parameters, these are ordinary smooth families.377For stochastic predictions on a finite set \(Y\), a smooth readout is a map378\[379 q:P\times X\longrightarrow\Delta^\circ(Y),380 \qquad (p,x)\longmapsto(q_i(p,x))_{i\in Y},