Independent scholarship · Open accessThe monthly digest

Indexed text attachment

evolutionary-game-theory-in-machine-learning-and-ai.tex

Source for Evolutionary Game Theory in Machine Learning and AI: Naturality, Regret, and Recurrence

View paper · Download original file

Lines 521–620 of 725. Source text is displayed literally.

521\Sli[\cite{NSTF04}] Let two variants play a coordination game: each does best against its own kind, and522the newcomer must exceed a fraction $x^*$ of the population before it is favoured. Under weak selection in a523large population a single newcomer takes over with probability greater than the neutral value $1/N$ if and524only if $x^*<\tfrac13$.525\end{theorem}526 527\noindent With $N=400$ and selection intensity $10^{-3}$, $N\rho$ was $1.021$, $1.009$, $0.991$ and $0.958$ at528$x^*=0.25$, $0.30$, $0.36$ and $0.45$. \chk{B16} The deterministic dynamics says that a new convention, protocol529or shared representation cannot invade from rarity at all when it needs partners; the finite population says530it can, by drift, and when it is more likely than chance to do so.531 532\begin{theorem}[The copying topology matters]\label{thm:graph}533\Sli[\cite{LHN05}] Let offspring of agent $i$ replace agent $j$ with probability $w_{ij}$. The fixation534probability of a variant of relative fitness $r$ equals the well-mixed value $(1-r^{-1})/(1-r^{-N})$ for all $r$535if and only if $W$ is doubly stochastic. A star amplifies selection. On a directed line, in which each536agent copies only its predecessor, fixation has probability $1/N$ whatever the fitness.537\end{theorem}538 539\noindent For $N=6$ and $r=1.2$ the exact values are $0.167$ on the line, $0.251$ well-mixed and $0.287$ on the540star. \chk{B17} The design reading is direct. Symmetric exchange between agents, of any shape, neither helps541nor hurts selection. A hub that many agents copy from and that copies from them sharpens it. A strictly542feed-forward pipeline of copying removes selection altogether: which variant ends up everywhere is decided by543where it started.544 545\begin{remark}[Populations of language-model agents]\label{rem:llm}546Recent work studies populations of agents built on large language models with exactly these tools: replicator547dynamics over agent types in open-source games \cite{OSG25}, evolutionarily stable strategies as a design548target for trust mechanisms \cite{EvT25}, group selection and replicator--mutator models for the evolution of549cooperative prompts \cite{GSP26}, evolutionary game theory as a probe of strategic behaviour \cite{SIL25}, and a550survey of the area \cite{XLLS26}. The titles and identifiers of these five items were checked on 17 November5512026; we have not evaluated their results. Our contribution to that programme is a set of constraints.552Two prompts that induce the same behaviour are clones, so an analysis of a prompt population should be natural553in the sense of \Cref{sec:setting}; stability should be asked of sets of prompts and not of single ones,554because stability of a point is destroyed by cloning and stability of a set is not \cite[Thm.~2.8]{EGHC}; and555conclusions drawn from the replicator equation about invasion by a single new agent are subject to the556correction of \Cref{thm:third}.557\end{remark}558 559\section{Inference}\label{sec:bayes}560 561\begin{proposition}[Bayes is the replicator]\label{prop:bayes}562\Scl[\cite{Sha09,Har09b}] Let $S$ be a set of hypotheses with likelihoods $\ell_h$. Bayesian updating of a prior563$x$ on an observation $o$ is one step of Hedge with $\eta=1$ and payoff $\pi_h=\log\ell_h(o)$. \chk{B18}564\end{proposition}565 566\noindent Bayesian inference is the replicator learner fed log-likelihoods, and the dictionary is exact.567Naturality for clone maps is the statement that splitting a hypothesis into two indistinguishable ones, with the568prior mass divided, changes no posterior probability of any event. The regret identity of \Cref{thm:regc} is569the statement that the log-loss of the Bayesian mixture exceeds that of the best hypothesis by at most570$\log(1/x_h(0))$. And the elimination of dominated strategies with its rate \cite{EGHC} is posterior571concentration: if observations come from $h^*$, the log-odds of any other $h$ against $h^*$ fall at the572expected rate $\KL{\ell_{h^*}}{\ell_h}$ per observation.573 574\section{Summary of results}\label{sec:summary}575 576\begin{table*}[t]577\centering\small578\begin{tabular}{@{}>{\raggedright}p{0.16\textwidth}>{\raggedright}p{0.3\textwidth}>{\raggedright}p{0.34\textwidth}l@{}}579\toprule580Topic & Qualitative statement & Quantitative statement & Checks\\581\midrule582Hedge and replicator & one is the other, sampled & exact at every step size & B1\\583Policy gradient & replicator with an extra factor $x_i$ & escape time $\sim1/(2D\epsilon)$ against $\log(1/\epsilon)/D$; ratios $3.0$, $11.8$, $73.3$ & B2, B3\\584Duplicated actions & (R), (H) natural; (PG), (Q) not & slowdown $2k/(k+1)$; defect $0.028$ & B19--B21\\585Regret & no regret & $\le\ln n$ continuous; $\le\sqrt{T\ln n/2}$ discrete; observed ratio $\le0.50$ & B4, B5\\586Zero-sum, flow & relative entropy conserved; recurrence & numerical drift $1.3\times10^{-15}$ over time $40$ & R4 of \cite{EGHC}\\587Zero-sum, Hedge & iterate leaves, average converges & $+\tfrac12\eta^2\Var$ per step; boundary after $1451$ steps; exploitability $0.0015$ & B6--B8\\588Optimism & last iterate converges & $0.070\to1.5\times10^{-10}$ in $6000$ steps & B9\\589Entropy regularization & converges; not natural; biased & rate at least $\alpha$, by an identity; type masses $(0.40,0.36,0.24)$ at $\tau=0.5$ & B10, B11, B21\\590Evaluation & ratings plus cycles plus harmonic & index $0.885$; curl $1.5$ & B12, B15\\591Redundant agents & averaging and name-mutation chain not natural; replicator average natural & $(0,0,0.25,-0.25)$; $(0.4,0.4,0.2)$; $(\tfrac13,\tfrac13,\tfrac13)$ & B13, B14\\592New conventions & can invade by drift & iff barrier $<\tfrac13$; $N\rho=1.021,\dots,0.958$ & B16\\593Copying topology & amplifiers and suppressors & $0.167<0.251<0.287$ & B17\\594Inference & Bayes is Hedge at $\eta=1$ & exact & B18\\595\bottomrule596\end{tabular}597\caption{Qualitative statements and their quantitative forms. All numbers are produced by the companion598script.}\label{tab:summary}599\end{table*}600 601\Cref{tab:summary} collects the results. Read down the middle column it is a list of facts most of which are602individually known; read across, each has a constant attached; and the recurring entry is naturality, which603sorts the methods into those that depend on how actions and agents are named and those that do not.604 605\section{Limits}\label{sec:limits}606 607\emph{Scope.} Everything here is tabular: a finite set of actions and exact payoffs. Function approximation,608sampling noise, and partial observation are outside it; they are where methods such as \cite{HMO20} do their609work. The numbers come from games with at most six actions and populations of at most a few hundred, chosen610to be exactly computable, and are illustrations of the statements and not benchmarks. No experiment on a611trained model was run.612 613\emph{Novelty.} The identifications of \Cref{prop:mwu,prop:pg,prop:bayes}, the regret bounds, recurrence, the614Hodge decomposition, the one-third law and the isothermal theorem are from the literature. The observations615are \Cref{thm:natural}, the identity of \Cref{thm:q}, \Cref{prop:evalnat}, and the placing of the three616repairs in \Cref{rem:repairs}. They are short, and parts of them are likely known in other forms; the617invariance of Nash averaging under redundancy, for one, is a stated motivation of \cite{BTPG18}.618 619\emph{How categorical.} Modestly. The category $\Game$, the functor to dynamical systems, and the operad of620wiring diagrams are ordinary category theory; the only cohomology is in degree one; the homotopy theory is in