Independent scholarship · Open accessThe monthly digest

Indexed text attachment

evolutionary-game-theory-in-machine-learning-and-ai.tex

Source for Evolutionary Game Theory in Machine Learning and AI: Naturality, Regret, and Recurrence

View paper · Download original file

Lines 545–644 of 725. Source text is displayed literally.

545\begin{remark}[Populations of language-model agents]\label{rem:llm}546Recent work studies populations of agents built on large language models with exactly these tools: replicator547dynamics over agent types in open-source games \cite{OSG25}, evolutionarily stable strategies as a design548target for trust mechanisms \cite{EvT25}, group selection and replicator--mutator models for the evolution of549cooperative prompts \cite{GSP26}, evolutionary game theory as a probe of strategic behaviour \cite{SIL25}, and a550survey of the area \cite{XLLS26}. The titles and identifiers of these five items were checked on 17 November5512026; we have not evaluated their results. Our contribution to that programme is a set of constraints.552Two prompts that induce the same behaviour are clones, so an analysis of a prompt population should be natural553in the sense of \Cref{sec:setting}; stability should be asked of sets of prompts and not of single ones,554because stability of a point is destroyed by cloning and stability of a set is not \cite[Thm.~2.8]{EGHC}; and555conclusions drawn from the replicator equation about invasion by a single new agent are subject to the556correction of \Cref{thm:third}.557\end{remark}558 559\section{Inference}\label{sec:bayes}560 561\begin{proposition}[Bayes is the replicator]\label{prop:bayes}562\Scl[\cite{Sha09,Har09b}] Let $S$ be a set of hypotheses with likelihoods $\ell_h$. Bayesian updating of a prior563$x$ on an observation $o$ is one step of Hedge with $\eta=1$ and payoff $\pi_h=\log\ell_h(o)$. \chk{B18}564\end{proposition}565 566\noindent Bayesian inference is the replicator learner fed log-likelihoods, and the dictionary is exact.567Naturality for clone maps is the statement that splitting a hypothesis into two indistinguishable ones, with the568prior mass divided, changes no posterior probability of any event. The regret identity of \Cref{thm:regc} is569the statement that the log-loss of the Bayesian mixture exceeds that of the best hypothesis by at most570$\log(1/x_h(0))$. And the elimination of dominated strategies with its rate \cite{EGHC} is posterior571concentration: if observations come from $h^*$, the log-odds of any other $h$ against $h^*$ fall at the572expected rate $\KL{\ell_{h^*}}{\ell_h}$ per observation.573 574\section{Summary of results}\label{sec:summary}575 576\begin{table*}[t]577\centering\small578\begin{tabular}{@{}>{\raggedright}p{0.16\textwidth}>{\raggedright}p{0.3\textwidth}>{\raggedright}p{0.34\textwidth}l@{}}579\toprule580Topic & Qualitative statement & Quantitative statement & Checks\\581\midrule582Hedge and replicator & one is the other, sampled & exact at every step size & B1\\583Policy gradient & replicator with an extra factor $x_i$ & escape time $\sim1/(2D\epsilon)$ against $\log(1/\epsilon)/D$; ratios $3.0$, $11.8$, $73.3$ & B2, B3\\584Duplicated actions & (R), (H) natural; (PG), (Q) not & slowdown $2k/(k+1)$; defect $0.028$ & B19--B21\\585Regret & no regret & $\le\ln n$ continuous; $\le\sqrt{T\ln n/2}$ discrete; observed ratio $\le0.50$ & B4, B5\\586Zero-sum, flow & relative entropy conserved; recurrence & numerical drift $1.3\times10^{-15}$ over time $40$ & R4 of \cite{EGHC}\\587Zero-sum, Hedge & iterate leaves, average converges & $+\tfrac12\eta^2\Var$ per step; boundary after $1451$ steps; exploitability $0.0015$ & B6--B8\\588Optimism & last iterate converges & $0.070\to1.5\times10^{-10}$ in $6000$ steps & B9\\589Entropy regularization & converges; not natural; biased & rate at least $\alpha$, by an identity; type masses $(0.40,0.36,0.24)$ at $\tau=0.5$ & B10, B11, B21\\590Evaluation & ratings plus cycles plus harmonic & index $0.885$; curl $1.5$ & B12, B15\\591Redundant agents & averaging and name-mutation chain not natural; replicator average natural & $(0,0,0.25,-0.25)$; $(0.4,0.4,0.2)$; $(\tfrac13,\tfrac13,\tfrac13)$ & B13, B14\\592New conventions & can invade by drift & iff barrier $<\tfrac13$; $N\rho=1.021,\dots,0.958$ & B16\\593Copying topology & amplifiers and suppressors & $0.167<0.251<0.287$ & B17\\594Inference & Bayes is Hedge at $\eta=1$ & exact & B18\\595\bottomrule596\end{tabular}597\caption{Qualitative statements and their quantitative forms. All numbers are produced by the companion598script.}\label{tab:summary}599\end{table*}600 601\Cref{tab:summary} collects the results. Read down the middle column it is a list of facts most of which are602individually known; read across, each has a constant attached; and the recurring entry is naturality, which603sorts the methods into those that depend on how actions and agents are named and those that do not.604 605\section{Limits}\label{sec:limits}606 607\emph{Scope.} Everything here is tabular: a finite set of actions and exact payoffs. Function approximation,608sampling noise, and partial observation are outside it; they are where methods such as \cite{HMO20} do their609work. The numbers come from games with at most six actions and populations of at most a few hundred, chosen610to be exactly computable, and are illustrations of the statements and not benchmarks. No experiment on a611trained model was run.612 613\emph{Novelty.} The identifications of \Cref{prop:mwu,prop:pg,prop:bayes}, the regret bounds, recurrence, the614Hodge decomposition, the one-third law and the isothermal theorem are from the literature. The observations615are \Cref{thm:natural}, the identity of \Cref{thm:q}, \Cref{prop:evalnat}, and the placing of the three616repairs in \Cref{rem:repairs}. They are short, and parts of them are likely known in other forms; the617invariance of Nash averaging under redundancy, for one, is a stated motivation of \cite{BTPG18}.618 619\emph{How categorical.} Modestly. The category $\Game$, the functor to dynamical systems, and the operad of620wiring diagrams are ordinary category theory; the only cohomology is in degree one; the homotopy theory is in621the Conley index behind \Cref{rem:mcc}, which we invoke and do not compute. The language earns its place by one622question, whether a construction commutes with the maps, which turned out to separate methods that the623literature usually compares on other grounds.624 625\emph{References.} The bibliography was assembled from memory, except for \cite{OPP19,HMO20} and the five items626of \Cref{rem:llm}, whose bibliographic details were checked online on 17 November 2026. It should be verified627before it is relied on.628 629\emph{Authorship.} This paper was drafted with the assistance of an AI system (Anthropic's Claude), which also630wrote and ran the companion script.631 632\paragraph{Open problems.} \Spg (1) Characterize the learners that are natural for clone maps among those633derived from a regularizer; we expect entropy on types and not on names to be forced. (2) Give the634rate in \Cref{thm:q} for a non-uniform equilibrium, where the limit moves with $\tau$. (3) Extend635\Cref{prop:evalnat}(c) from time averages to chain components, so that a ranking by recurrent classes is636natural by construction. (4) Find the analogue of the isothermal theorem for populations of learners whose637fitness depends on the population, where the graph and the game interact.638 639\appendix640\section{The companion script}\label{app:checks}641 642\texttt{verify-evolutionary-learning.py} uses only the Python standard library, fixes its random seed, runs643in under half a minute, and exits with status zero if and only if its 21 checks pass. Flows are integrated by644a fourth-order Runge--Kutta scheme; fixation probabilities are exact. The checks test instances and recompute