%% Evolutionary Game Theory in Machine Learning and AI: Naturality, Regret, and Recurrence
%% Higher Categorical Semantics. Byline: Anonymous.
%% Single-file LaTeX source (UTF-8), version 1.0, 17 November 2026. Two-column article.
%% Build: pdflatex (three passes). Packages: mathpazo, helvet, geometry, amsmath, amsthm, mathtools, microtype, tikz, booktabs, array, enumitem, caption, hyperref, cleveref.
%% Companion script: verify-evolutionary-learning.py (21 computational checks, Python standard library only).
%% Drafted and typeset with AI assistance (Anthropic's Claude) under the direction of the submitting author.
\documentclass[10pt,twocolumn]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[sc]{mathpazo}
\usepackage[scaled=0.9]{helvet}
\usepackage[letterpaper,margin=0.78in,columnsep=0.3in]{geometry}
\usepackage{amsmath,amssymb,amsthm,mathtools}
\usepackage{microtype}
\usepackage{xcolor}
\definecolor{accent}{HTML}{7A1F2B}
\definecolor{ink2}{HTML}{55595F}
\definecolor{fwd}{HTML}{1F4E79}
\definecolor{bwd}{HTML}{5B3A7A}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,positioning}
\usepackage{booktabs,array}
\usepackage{enumitem}
\usepackage[font=small,labelfont={bf,sf}]{caption}
\usepackage[colorlinks=true,linkcolor=accent,citecolor=accent,urlcolor=accent,
  pdftitle={Evolutionary Game Theory in Machine Learning and AI: Naturality, Regret, and Recurrence},
  pdfauthor={Anonymous}]{hyperref}
\usepackage[nameinlink,capitalise,noabbrev]{cleveref}
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
\newcommand{\stag}[2]{{\footnotesize\sffamily\upshape\color{#1}[#2]}\enspace\ignorespaces}
\newcommand{\Scl}[1][]{\stag{ink2}{classical\if\relax\detokenize{#1}\relax\else; #1\fi}}
\newcommand{\Sli}[1][]{\stag{fwd}{literature\if\relax\detokenize{#1}\relax\else; #1\fi}}
\newcommand{\Srf}[1][]{\stag{accent}{reformulation\if\relax\detokenize{#1}\relax\else; cf.\ #1\fi}}
\newcommand{\Sob}{\stag{accent}{observation}}
\newcommand{\Spg}{\stag{bwd}{programme}}
\newcommand{\chk}[1]{{\footnotesize\sffamily\upshape\color{accent}[#1]}}
\newcommand{\cat}[1]{\mathsf{#1}}
\renewcommand{\Game}{\cat{Game}}
\newcommand{\Dyn}{\cat{Dyn}}
\newcommand{\Dist}{\Delta}
\newcommand{\R}{\mathbb{R}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\one}{\mathbf{1}}
\newcommand{\T}{\mathsf{T}}
\newcommand{\KL}[2]{D(#1\,\|\,#2)}
\newcommand{\sm}{\operatorname{softmax}}
\newcommand{\Var}{\operatorname{Var}}
\newcommand{\term}[1]{\textbf{\itshape #1}}
\setlength{\parskip}{0pt}
\frenchspacing
\raggedbottom
\title{\textbf{Evolutionary Game Theory in Machine Learning and AI:\\ Naturality, Regret, and Recurrence}}
\author{Anonymous}
\date{Higher Categorical Semantics \quad$\cdot$\quad Version 1.0, 17 November 2026}
\begin{document}
\twocolumn[
\maketitle
\begin{center}\begin{minipage}{0.86\textwidth}\small
\textbf{Abstract.} The replicator equation of evolutionary game theory and the standard learning rules of
machine learning are the same objects under different names: multiplicative weights is the replicator flow
sampled in time, natural policy gradient is the replicator field, and Bayesian updating is the replicator map
with log-likelihood as payoff. We organize these identifications around one structure. A learner is an open
dynamical system with payoffs as input and a mixed strategy as output; a game is a wiring of learners; and a
learning rule is \emph{natural} if its closed-loop dynamics commutes with payoff-preserving maps of games, in
particular with the duplication of actions. Each qualitative statement is paired with a quantitative one.
Naturality: the replicator learner is natural, while softmax policy gradient, entropy regularization, score
averaging and a mutation-chain ranking are not, with computed defects; duplicating an action $k$ times multiplies
the policy-gradient learning time by exactly $2k/(k+1)$. Regret: at most $\ln n$ in continuous time for every
horizon, against $\sqrt{T\ln n/2}$ in discrete time. Recurrence: in zero-sum games the relative entropy to
equilibrium is conserved by the flow, increases by exactly $\log\E e^{\eta f}\approx\eta^2\Var(f)/2$ per step
of multiplicative weights, and decays at rate at least $\alpha$ under entropy-regularized dynamics. Finite
populations of learners obey the one-third law and the amplifier and suppressor phenomena of evolutionary graph
theory. All numerical statements are recomputed by a companion script with 21 checks. No new theorem of
learning theory is claimed; the contribution is the organization, the exact constants, and the naturality
criterion.
\end{minipage}\end{center}
\vspace{1.2em}
]

\section{Introduction}

Evolutionary game theory studies populations in which strategies that earn more become more common. Its
central object, the replicator equation
\begin{equation}\label{eq:rep}
\dot x_i=x_i\bigl((Ax)_i-x^{\T}Ax\bigr),
\end{equation}
was written down for animal conflicts \cite{TaJo78,MSP73}. Machine learning arrived at the same equation
several times: as the continuous-time limit of reinforcement learning \cite{BoSa97} and of multiplicative
weights \cite{FrSc97,Sor09}, as natural policy gradient on a softmax policy \cite{Kak01}, and implicitly in
population-based training, where agents copy their betters \cite{Jad17}. A compact account of the evolutionary side, written
for philosophers and candid about the fragility of the models, is Alexander's Element \cite{Alex23}; a
companion volume \cite{EGHC} retells it in categorical language. This paper carries the results of that volume
to machine learning.

The organizing idea is to separate what the evolutionary literature fuses. In \eqref{eq:rep} a learning rule
and a game are multiplied together. We take them apart (\Cref{sec:setting}): a \emph{learner} is an open
dynamical system, a box with payoffs flowing in and a mixed strategy flowing out, and a \emph{game} is a wiring
of such boxes. Self-play, two-player training, leagues and populations on graphs are different diagrams
filled with the same boxes. The question that the categorical setting makes precise is which learners are
\emph{natural}: whose behaviour does not depend on how the actions happen to be named, and in particular on
whether one action has been listed twice.

\paragraph{Results.} Each is stated with its status. Tags follow the conventions of the series:
\Scl a classical result; \Sli[source] a result of the cited literature; \Srf[source] a known result restated;
\Sob an observation of this paper, proved here. Tags such as \chk{B6} name checks in the companion script
\texttt{verify-evolutionary-learning.py}.
\begin{itemize}[leftmargin=1.2em,itemsep=1pt,topsep=2pt]
\item \emph{Naturality} (\Cref{sec:setting,sec:eval}). The replicator learner is natural for all
payoff-preserving maps. Softmax policy gradient is not natural for duplication, and the slowdown is the
exact factor $2k/(k+1)$ (\Cref{thm:natural}). Entropy regularization and two common ways of ranking agents fail
the same test (\Cref{prop:evalnat}).
\item \emph{Regret} (\Cref{sec:regret}). An identity gives regret at most $\ln n$ in continuous time; the
term that discretization adds is the one that reappears as divergence in games.
\item \emph{Recurrence} (\Cref{sec:zerosum}). In zero-sum games continuous-time learning cycles, discrete-time
learning spirals out at a computable rate, and three repairs act at three different places of the wiring
diagram.
\item \emph{Populations} (\Cref{sec:pop}). Fixation probabilities, the one-third law, and the effect of the
copying topology.
\end{itemize}
\Cref{tab:summary} lists the pairs of qualitative and quantitative statements. \Cref{sec:limits} states what
is not claimed.

\paragraph{Related work.} The link between reinforcement learning and replicator dynamics goes back to
\cite{BoSa97} and is surveyed in \cite{BTHK15}; the behaviour of regularized learning in zero-sum games is the
subject of \cite{PiSh14,MPP18,BaPi18}; evolutionary methods for evaluating agents are developed in
\cite{BTPG18,OPP19}. On the categorical side, learners as open systems composed along wiring diagrams follow
\cite{SSV20,Mye21}, and compositional accounts of gradient-based learning are given in \cite{FST19,CGG22}. What we
add to both is the question of naturality with respect to maps of games, which neither literature asks in this
form.

\section{Learners, games, and naturality}\label{sec:setting}

\subsection{Games and their maps}

A \term{symmetric game} $(S,A)$ is a finite set $S$ of actions and a matrix $A\in\R^{S\times S}$; $A_{ij}$ is
the payoff of $i$ against $j$. Mixed strategies form the simplex $\Dist(S)$. A \term{payoff-preserving map}
$f\colon(S,A)\to(T,B)$ is a function with $B_{f(i)f(j)}=A_{ij}$. These are the morphisms of a category $\Game$
\cite{EGHC}. They are generated by three kinds: \emph{inclusions} of a subset of actions, \emph{clone maps},
which are surjections identifying actions with equal rows and columns, and \emph{symmetries}. A map $f$
induces the pushforward $f_*\colon\Dist(S)\to\Dist(T)$, which adds the weights of identified actions. The basic
fact is that $f_*$ preserves payoffs: $(Bf_*x)_{f(i)}=(Ax)_i$ and $(f_*x)^{\T}B(f_*x)=x^{\T}Ax$.

\subsection{Learners}

\begin{definition}
A \term{learner} on $S$ is an open dynamical system with input a payoff vector $\pi\in\R^S$, a state $z$, a
readout $x=r(z)\in\Dist(S)$, and an update: $\dot z=V(z,\pi)$ in continuous time or $z^+=U(z,\pi)$ in discrete
time.
\end{definition}

\noindent Five learners recur below. All read out by $\sm$, applied to $Q/\tau$ in the last case.
\begin{center}\small
\begin{tabular}{@{}lll@{}}
\toprule
learner & state & update\\
\midrule
replicator (R) & scores $z$ & $\dot z=\pi$\\
Hedge, MWU (H) & scores $z$ & $z^+=z+\eta\pi$\\
optimistic (O) & scores $z$ & $z^+=z+\eta(2\pi_t-\pi_{t-1})$\\
policy grad.\ (PG) & logits $\theta$ & $\dot\theta_i=x_i(\pi_i-x\!\cdot\!\pi)$\\
Boltzmann $Q$ (Q) & values $Q$ & $\dot Q=\alpha(\pi-Q)$\\
\bottomrule
\end{tabular}
\end{center}
\noindent (PG) is gradient ascent on expected payoff $x\cdot\pi$ in the logits; (Q) is the expected,
all-actions form of $Q$-learning with Boltzmann exploration \cite{TVL03,SaCr03}.

\begin{proposition}[Hedge is the replicator, sampled]\label{prop:mwu}
\Scl The replicator learner fed a payoff stream $\pi(\cdot)$ has
\[ x(t)=\sm\Bigl(\log x(0)+\textstyle\int_0^t\pi\Bigr), \]
and in the variable $x$ it obeys $\dot x_i=x_i(\pi_i-x\cdot\pi)$. One step of Hedge with step size $\eta$ is the
time-$\eta$ map of this flow under a payoff held constant over the step. \chk{B1}
\end{proposition}

\begin{proposition}[Policy gradient and its natural form]\label{prop:pg}
\Scl[\cite{Kak01,Ama98}] For $x=\sm(\theta)$, $\partial(x\cdot\pi)/\partial\theta_i=x_i(\pi_i-x\cdot\pi)$.
Multiplying this gradient by the inverse of the Fisher matrix $\operatorname{diag}(x)-xx^{\T}$, on the
tangent space, gives $\pi-(x\cdot\pi)\one$: natural policy gradient on a softmax policy is the replicator
learner. \chk{B2}
\end{proposition}

\noindent So (PG) differs from (R) by one factor of $x_i$ in the update of the logits. Neural Replicator
Dynamics makes exactly this change in a deep policy-gradient method, bypassing the gradient through the softmax,
and reduces to Hedge in the single-state tabular case \cite{HMO20}. In the variable $x$,
\begin{equation}\label{eq:pgx}
\text{(PG)}\qquad\dot x_i=x_i\Bigl(x_i(\pi_i-\bar\pi)-\sum_jx_j^2(\pi_j-\bar\pi)\Bigr),
\end{equation}
with $\bar\pi=x\cdot\pi$. The extra factor has a cost that is easy to state.

\begin{proposition}[Escape from a bad start]\label{prop:escape}
\Scl[cf.\ \cite{MXSS20}] Two actions, payoff gap $D>0$, the better action at initial probability $\epsilon$. The
time for it to reach $\tfrac12$ is
\begin{align*} t_{\mathrm R}&=\frac1D\log\frac{1-\epsilon}{\epsilon},\\
t_{\mathrm{PG}}&=\frac1{2D}\Bigl(\frac1\epsilon-\frac1{1-\epsilon}+2\log\frac{1-\epsilon}{\epsilon}\Bigr). \end{align*}
\end{proposition}
\begin{proof}
With $x$ the probability of the better action, (R) is $\dot x=Dx(1-x)$ and, by \eqref{eq:pgx}, (PG) is
$\dot x=2Dx^2(1-x)^2$. Integrate.
\end{proof}

\noindent For $D=1$ and $\epsilon=0.1$, $0.01$, $0.001$ the two times are $2.20$ and $6.64$, $4.60$ and
$54.1$, $6.91$ and $506$: ratios $3.0$, $11.8$ and $73.3$. \chk{B3} The replicator learner recovers from a bad
initialization in logarithmic time and policy gradient in time of order $1/\epsilon$. In evolutionary terms (PG)
treats a rare good action as the drift game treats a neutral mutant \cite{EGHC}.

\subsection{Closing the loop}

A learner becomes a dynamical system when its input is supplied. \term{Self-play} in a symmetric game feeds
back $\pi=A\,r(z)$; for (R) this is \eqref{eq:rep}. Two learners in a bimatrix game $(A,B)$ are wired
crosswise, $\pi^1=A\,r(z^2)$ and $\pi^2=B^{\T}r(z^1)$. A league or a population on a graph is a larger diagram.
Formally, open dynamical systems are algebras for an operad of wiring diagrams \cite{SSV20,Mye21}, and a
multi-agent learning problem is a diagram together with a choice of box for each node (\Cref{fig:wiring}).

\begin{figure}[t]
\centering
\begin{tikzpicture}[>=Stealth,every node/.style={font=\footnotesize},scale=0.82]
\node[draw,thick,minimum width=1.25cm,minimum height=0.62cm] (p) at (0,0) {learner};
\node[draw,minimum width=0.7cm,minimum height=0.5cm] (a) at (0,-1.05) {$A$};
\draw[->] (p.east) -- ++(0.45,0) |- node[pos=0.25,right]{$x$} (a.east);
\draw[->] (a.west) -- ++(-0.75,0) |- node[pos=0.25,left]{$\pi$} (p.west);
\node at (0,-1.75) {self-play};
\begin{scope}[xshift=4.6cm]
\node[draw,thick,minimum width=0.95cm,minimum height=0.62cm] (r) at (-1.25,0) {$L_1$};
\node[draw,thick,minimum width=0.95cm,minimum height=0.62cm] (c) at (1.25,0) {$L_2$};
\node[draw,minimum width=0.7cm,minimum height=0.5cm] (a2) at (0,0.62) {$B^{\T}$};
\node[draw,minimum width=0.7cm,minimum height=0.5cm] (b2) at (0,-0.62) {$A$};
\draw[->] (r.north) |- (a2.west);
\draw[->] (a2.east) -| (c.north);
\draw[->] (c.south) |- (b2.east);
\draw[->] (b2.west) -| (r.south);
\node at (0,-1.75) {two players};
\end{scope}
\end{tikzpicture}
\caption{A game is a wiring of learners. The boxes can be changed without changing the diagram, and the
diagram without changing the boxes.}\label{fig:wiring}
\end{figure}

\subsection{Naturality}

\begin{definition}
A family of learners, one for each finite set $S$, is \term{natural} for a class of payoff-preserving maps if
for every $f\colon(S,A)\to(T,B)$ in the class, $f_*$ carries self-play trajectories in $\Dist(S)$ to self-play
trajectories in $\Dist(T)$.
\end{definition}

\noindent Naturality for inclusions says that an action never tried is never missed, and the rest behaves as
if it did not exist. Naturality for clone maps says that listing an action twice, or splitting it into
indistinguishable variants, changes nothing about the total weight of the variants. Naturality for symmetries
is equivariance.

\begin{theorem}[Which learners are natural]\label{thm:natural}
\Sob
\begin{enumerate}[label=(\alph*),leftmargin=1.5em,itemsep=1pt,topsep=2pt]
\item \emph{(R)} and \emph{(H)} are natural for all payoff-preserving maps.
\item \emph{(PG)} is natural for inclusions and symmetries and not for clone maps. If an action of total
probability $y$ is split into copies with probabilities $x_1,\dots,x_k$, the leading term of its drift is
$\sum_cx_c^2$ where the unsplit learner has $y^2$.
\item In the two-action problem of \Cref{prop:escape}, with either action split into $k$ equal copies, the
probability $y$ of the better class obeys $\dot y=(1+\tfrac1k)Dy^2(1-y)^2$. Every learning time is multiplied
by exactly $2k/(k+1)$.
\item \emph{(Q)} is natural for symmetries and not for clone maps. Its rest points are the logit equilibria
$x=\sm(Ax/\tau)$, which depend on the number of copies of each action, and every face of the simplex is
repelling: no action stays unused.
\end{enumerate}
\end{theorem}
\begin{proof}
(a) For (R), with $y=f_*x$, the pushforward identity gives
$\dot y_t=\sum_{f(i)=t}x_i((Ax)_i-x^{\T}Ax)=y_t((By)_t-y^{\T}By)$ for $t$ in the image and $y_t\equiv0$ otherwise.
(H) is a time-$\eta$ map of the same flow with frozen payoffs, and frozen payoffs are also preserved.
(b) In \eqref{eq:pgx} the factor $x_i$ gives invariance of faces. Summing \eqref{eq:pgx} over the copies of
one action gives $(\sum_cx_c^2)(\pi-\bar\pi)-y\sum_jx_j^2(\pi_j-\bar\pi)$, and $\sum_cx_c^2<y^2$ unless all but one
copy is empty. (c) By symmetry the copies stay equal. Substituting $x_c=y/k$ for the split action in
\eqref{eq:pgx} and summing gives the stated equation in both cases; compare with $2Dy^2(1-y)^2$. (d) In the
variable $x$, (Q) in self-play is
\begin{equation}\label{eq:q}
\dot x_i=x_i\Bigl(\tfrac\alpha\tau(\pi_i-\bar\pi)-\alpha\bigl(\log x_i-\textstyle\sum_jx_j\log x_j\bigr)\Bigr),
\end{equation}
whose rest points satisfy $\log x_i=\pi_i/\tau+\mathrm{const}$. The entropy term makes $\dot x_i/x_i\to+\infty$ as
$x_i\to0$, and it distinguishes one action of weight $y$ from two of weight $y/2$, since
$2\cdot\tfrac y2\log\tfrac y2\ne y\log y$.
\end{proof}

\noindent \emph{Computed defects.} For Rock--Paper--Scissors with Rock duplicated, the largest discrepancy over
fifty random states between the pushed-forward field and the field of the three-action game was $4\times10^{-17}$
for (R) and $0.028$ for (PG). \chk{B19} From $\epsilon=0.01$ the policy-gradient time to $\tfrac12$ was $54.1$,
$72.1$, $86.5$ and $96.2$ for $k=1,2,4,8$, and the replicator time $4.60$ for every $k$. \chk{B20} For (Q) the
logit equilibrium of Rock--Paper--Scissors is uniform at every temperature; with Rock duplicated its mass on the
three types is $(0.49,0.29,0.23)$ at $\tau=2$, $(0.40,0.36,0.24)$ at $\tau=0.5$ and $(0.34,0.35,0.31)$ at
$\tau=0.1$. \chk{B21}

\begin{remark}
Large action spaces in practice contain near-duplicates: synonymous tokens, equivalent moves, redundant
members of a league. \Cref{thm:natural} says which update rules are indifferent to them. It is the
finite-dimensional shadow of the invariance of natural gradient under reparametrization \cite{Ama98}, stated for
the maps that matter in games. Entropy bonuses are the clearest case of a method that counts names: at high
temperature the regularized equilibrium gives duplicated Rock half the mass.
\end{remark}

\section{Regret}\label{sec:regret}

A learner has \term{no regret} if, against any stream of payoffs, its average payoff is asymptotically at
least that of the best fixed action. For the replicator learner this is an identity.

\begin{theorem}[Continuous time]\label{thm:regc}
\Scl[\cite{Sor09,KwMe17}] For the replicator learner and any bounded measurable payoff stream,
\[ \int_0^T\bigl(\pi_i-x\cdot\pi\bigr)\,dt=\log\frac{x_i(T)}{x_i(0)}\le\log\frac1{x_i(0)} \]
for every action $i$ and every horizon $T$. From the uniform initial state the regret is at most $\ln n$.
\end{theorem}
\begin{proof}
$\frac d{dt}\log x_i=\pi_i-x\cdot\pi$ by \Cref{prop:mwu}, and $x_i(T)\le1$.
\end{proof}

\noindent The bound does not grow with $T$. Against a stream of six payoffs that alternates between random
vectors and vectors chosen to punish the current favourite, the regret at $T=80$ was $0.24$, against
$\ln6=1.79$, and the identity held to $5\times10^{-8}$. \chk{B4}

\begin{theorem}[Discrete time]\label{thm:regd}
\Scl[\cite{FrSc97,CBL06}] For Hedge with payoffs in $[0,1]$ and $\eta=\sqrt{8\ln n/T}$, the regret after $T$ steps
is at most $\sqrt{T\ln n/2}$.
\end{theorem}
\begin{proof}[Sketch]
One step multiplies the normalizing constant by $\E_xe^{\eta\pi}$, and Hoeffding's lemma gives
$\log\E_xe^{\eta(\pi-x\cdot\pi)}\le\eta^2/8$. Summing, the discrete learner satisfies the identity of
\Cref{thm:regc} up to an excess of at most $\eta/8$ per step: the regret is at most $\ln n/\eta+\eta T/8$.
\end{proof}

\noindent Over forty runs with $n\in\{2,5,10\}$, $T\in\{200,1000\}$, half against random payoffs and half
against an adversary, the largest ratio of regret to bound was $0.50$. \chk{B5} The whole difference between
$\ln n$ and $\sqrt T$ is the term
\begin{equation}\label{eq:excess}
\Lambda_\eta(x,\pi)=\log\E_x\,e^{\eta(\pi-x\cdot\pi)}=\tfrac12\eta^2\Var_x(\pi)+O(\eta^3),
\end{equation}
the price of holding the payoff fixed for a step. The same term decides the fate of discrete learning in
games.

\begin{corollary}\label{cor:avg}
\Scl If both players of a zero-sum game with payoffs in an interval of length $1$ use Hedge with the step of
\Cref{thm:regd}, their time-averaged strategies form a $2\sqrt{\ln n/(2T)}$-equilibrium. In a general game the
time-averaged joint play approaches the set of coarse correlated equilibria.
\end{corollary}

\noindent In self-play of Rock--Paper--Scissors, whose payoffs have range $2$, the time average over
$T=20000$ steps could be exploited for $0.0015$, against the corresponding bound $0.0105$. \chk{B8}

\section{Zero-sum games: recurrence, divergence, repair}\label{sec:zerosum}

Regret bounds speak of averages. What the strategies themselves do is another matter, and it is the matter
that concerns adversarial training \cite{Goo14}, where the iterate and not the average is the model.

\subsection{The flow cycles}

\begin{theorem}[Conservation and recurrence]\label{thm:rec}
\Sli[\cite{HoSi98,PiSh14,MPP18}] Let $A$ be antisymmetric with an interior equilibrium $x^*$. Along replicator
self-play $\KL{x^*}{x(t)}$ is constant. For two replicator learners in a zero-sum game with interior equilibrium
$(x^*,y^*)$ the sum $\KL{x^*}{x}+\KL{y^*}{y}$ is constant, and the flow is Poincar\'e recurrent: almost every
trajectory returns arbitrarily close to its starting point infinitely often.
\end{theorem}
\begin{proof}[Proof of conservation]
$\frac d{dt}\KL{x^*}{x}=-\sum_ix^*_i(\pi_i-\bar\pi)=-(x^*)^{\T}Ax+x^{\T}Ax$. Antisymmetry gives $x^{\T}Ax=0$, and
$(x^*)^{\T}Ax=-x^{\T}Ax^*=0$ because $Ax^*$ is constant on the support of an interior equilibrium and that
constant is the value $0$.
\end{proof}

\noindent Conservation says that such a game does no work on the learner. An antisymmetric matrix has no
symmetric part, so the gradient component of the replicator field vanishes \cite{EGHC}; and when the equilibrium
is uniform the game is purely cyclic in the sense of the Hodge decomposition of \Cref{sec:eval}.

\subsection{The discretization spirals out}

\begin{proposition}[Exact drift of Hedge]\label{prop:drift}
\Srf[\cite{BaPi18}] In Hedge self-play of an antisymmetric game with interior equilibrium $x^*$, each step
changes the relative entropy by exactly
\[ \KL{x^*}{x^+}-\KL{x^*}{x}=\Lambda_\eta(x,Ax)\ \ge0, \]
with equality only if all actions in use earn the same. Hence $\KL{x^*}{x_t}$ is non-decreasing, and it increases
by about $\tfrac12\eta^2\Var_x(Ax)$ per step.
\end{proposition}
\begin{proof}
$x^+_i=x_ie^{\eta\pi_i}/Z$ with $Z=\E_xe^{\eta\pi}$, so the difference is $-\eta\,x^*\!\cdot\pi+\log Z$. As in
\Cref{thm:rec}, $x^*\!\cdot\pi=0=x\cdot\pi$, so the difference is $\log Z=\Lambda_\eta$, and Jensen's inequality
gives the sign.
\end{proof}

\noindent At the state $(0.5,0.3,0.2)$ of Rock--Paper--Scissors the increase per step was $5.01\times10^{-5}$,
$2.01\times10^{-4}$ and $8.08\times10^{-4}$ for $\eta=0.05$, $0.1$, $0.2$, against
$\tfrac12\eta^2\Var=5.00\times10^{-5}$, $2.00\times10^{-4}$, $8.00\times10^{-4}$. \chk{B6} With $\eta=0.1$ the
relative entropy increased at every step and some action fell below probability $0.01$ after $1451$ steps.
\chk{B7} The quantity \eqref{eq:excess} that costs a learner $\sqrt T$ in regret is, in a game, energy pumped
into the cycle at every step. The time average still converges (\Cref{cor:avg}); the iterate leaves.

\subsection{Three repairs, in three places}

\begin{proposition}[Optimism]\label{prop:opt}
\Sli[\cite{RaSr13,DISZ18,DaPa19}] The optimistic learner \emph{(O)}, which uses $2\pi_t-\pi_{t-1}$ as its
prediction of the next payoff, converges in the last iterate in zero-sum games with a unique equilibrium, for
small enough $\eta$.
\end{proposition}

\noindent From the same initial state and $\eta=0.1$ the relative entropy to equilibrium fell from $0.070$ to
$1.5\times10^{-10}$ in $6000$ steps. \chk{B9}

\begin{theorem}[Entropy regularization, with its rate]\label{thm:q}
\Sob Let $A$ be antisymmetric with the uniform state $u$ as equilibrium. Under the dynamics \eqref{eq:q},
for every $\tau>0$,
\[ \frac d{dt}\KL ux=-\alpha\bigl(\KL ux+\KL xu\bigr), \]
and therefore $\KL u{x(t)}\le e^{-\alpha t}\KL u{x(0)}$.
\end{theorem}
\begin{proof}
$\frac d{dt}\KL ux=-\sum_iu_i\dot x_i/x_i$. The payoff terms vanish as in \Cref{thm:rec}. The entropy terms
give $\alpha\bigl(\sum_iu_i\log x_i-\sum_jx_j\log x_j\bigr)$, and
$\KL ux+\KL xu=\sum_jx_j\log x_j-\sum_iu_i\log x_i$.
\end{proof}

\noindent Convergence of $Q$-learning dynamics in competitive games is known in much greater generality
\cite{LPS21}; the point of \Cref{thm:q} is the exact identity, in which the rate is the learning rate $\alpha$
and does not depend on the game or the temperature. With $\alpha=0.7$ the relative entropy fell from $0.070$ to
$6.1\times10^{-8}$ at $t=10$, against the bound $6.4\times10^{-5}$. \chk{B10} In general the limit is a logit
equilibrium and not a Nash equilibrium \chk{B11}, and by \Cref{thm:natural}(d) it is not natural.

\begin{remark}[Where each repair acts]\label{rem:repairs}
Averaging changes the \emph{readout}: the box and the wiring are untouched and one reports $\frac1T\sum x_t$.
Optimism changes the \emph{box}: the learner carries one more step of memory. Regularization changes the
\emph{wiring}: \eqref{eq:q} is the replicator learner in the game with the additional self-payoff
$-\tau\log x_i$, a loop from each learner's output to its own input, up to a constant change of time scale. In
evolutionary terms these are
time-averaging of frequencies, anticipation, and mutation \cite{TVL03}. The trade is visible in the table of
\Cref{sec:summary}: averaging keeps naturality and loses the iterate; regularization gains a rate and loses
both naturality and the equilibrium.
\end{remark}

\section{Evaluating agents}\label{sec:eval}

Training by self-play, leagues and population-based methods all need to say which of a set of agents is
better \cite{Vin19,LZG17}. The data is an \term{evaluation matrix}: an antisymmetric $K$ with $K_{ij}$ the
advantage of agent $i$ over agent $j$, for instance the log-odds of winning. This is a symmetric zero-sum game
on the set of agents, and an \term{evaluation method} assigns to it a score vector or a distribution on agents.

\begin{theorem}[Ratings and cycles]\label{thm:hodge}
\Sli[\cite{JLYY11,BTPG18}] Every antisymmetric $K$ decomposes uniquely and orthogonally as
\[ K_{ij}=(r_i-r_j)+C_{ij},\quad r_i=\tfrac1n\textstyle\sum_jK_{ij},\quad\textstyle\sum_jC_{ij}=0. \]
The first part satisfies $G_{ij}+G_{jk}+G_{ki}=0$ on every triple, and every matrix satisfying this is of that
form. The second part carries the same curl $K_{ij}+K_{jk}+K_{ki}$ as $K$.
\end{theorem}

\noindent The ratings $r$ are the least-squares fit of an Elo-like scale; the \term{transitivity index}
$\|G\|^2/\|K\|^2$ measures how much of the data a scale can explain. For five agents with planted ratings and one
planted cycle of strength $0.5$ among three of them, the decomposition returned the ratings
$(0.7,0.3,0,-0.3,-0.7)$, index $0.885$ and curl $1.5$ on the planted triple. \chk{B12} That closed implies
exact is the vanishing of the first cohomology of a simplex.

\begin{proposition}[Incomplete comparisons]\label{prop:harmonic}
\Sli[\cite{JLYY11}] If only some pairs of agents have been compared, the decomposition acquires a third,
harmonic component: circulations around loops of the comparison graph that bound no triple of mutually
compared agents. It is locally consistent and globally not a rating, and its dimension is the first Betti
number of the comparison complex.
\end{proposition}

\noindent Four agents compared only around a square, each beating the next by one unit, give the smallest
example: zero divergence, no triangles, nothing to fit. \chk{B15} Sparse tournament schedules create such
classes by design.

The naturality criterion of \Cref{sec:setting} applies to evaluation unchanged. A league that contains two
copies of an agent, or ten near-copies obtained from one checkpoint, should not be ranked differently for
that reason.

\begin{proposition}[Naturality of evaluation methods]\label{prop:evalnat}
\Sob Let $f\colon(S,K)\to(T,K')$ be a clone map between evaluation games.
\begin{enumerate}[label=(\alph*),leftmargin=1.5em,itemsep=1pt,topsep=2pt]
\item Mean score against the field, $s_i=\frac1n\sum_jK_{ij}$, is not natural: duplicating an agent raises the
score of everything that beats it.
\item The set of Nash equilibria is natural: $x$ is an equilibrium of $(S,K)$ if and only if $f_*x$ is one of
$(T,K')$. When the equilibrium of $(T,K')$ is unique, every rule that selects an equilibrium is therefore
natural on types. Invariance to redundant agents is the stated aim of Nash averaging \cite{BTPG18}.
\item The time average of replicator self-play is natural, and when $(T,K')$ has a unique interior equilibrium
it assigns to each type its equilibrium mass.
\item The stationary distribution of the small-mutation chain in which a monomorphic population is invaded by
a mutant drawn uniformly from the other \emph{names}, with transition probability given by the fixation
probability, is not natural.
\end{enumerate}
\end{proposition}
\begin{proof}
(a) and (d) by example, below. (b) Payoffs against $f_*x$ at the types are the payoffs against $x$ at the
agents, and a clone map omits no type \cite[Prop.~1.10]{EGHC}. (c) By \Cref{thm:natural}(a), $f_*$ carries
trajectories to trajectories and hence time averages to time averages; the time-average theorem for the
replicator equation \cite{HoSi98} identifies the limit.
\end{proof}

\noindent \emph{The example.} Rock, a second copy of Rock, Paper, Scissors. Mean scores are $(0,0,0.25,-0.25)$:
Paper looks best and Scissors worst, although the game is symmetric among the three types. The
time-averaged replicator mass on the types is $(0.3334,0.3333,0.3333)$. \chk{B13} The small-mutation chain,
uniform on plain Rock--Paper--Scissors, gives $(0.2,0.2,0.4,0.2)$ on the four names: Rock's type has mass $0.4$,
Paper $0.4$ and Scissors $0.2$. \chk{B14} The reason is visible in the chain: a population of Scissors is now
invaded twice as often.

\begin{remark}[Chain components]\label{rem:mcc}
The chain in (d) is in the manner of $\alpha$-Rank \cite{OPP19}, which ranks agents by the stationary
distribution of a Markov chain on strategy profiles built from fixation probabilities, and which grounds the
construction in Conley's theory: the long-run behaviour of a dynamical system is carried by its chain
recurrent set, and Markov--Conley chains are a finite model of the sink components. This is the right
generalization of equilibrium for games that cycle \cite{PaPi19}, and the dynamical content is natural: chain
components of a natural flow map into chain components. Statement (d) concerns only the mutation kernel of
our small implementation, which counts names; we make no claim about $\alpha$-Rank as published or as used.
A kernel that draws mutants by type would restore invariance, at the price of knowing the types.
\end{remark}

\section{Finite populations of learners}\label{sec:pop}

Population-based training keeps $N$ agents, lets better ones overwrite worse ones, and perturbs the copies
\cite{Jad17}. With copying in proportion to fitness and one replacement at a time this is the Moran process
of population genetics, and its classical theory applies \cite{Now06,EGHC}.

\begin{theorem}[Fixation and the one-third law]\label{thm:third}
\Sli[\cite{NSTF04}] Let two variants play a coordination game: each does best against its own kind, and
the newcomer must exceed a fraction $x^*$ of the population before it is favoured. Under weak selection in a
large population a single newcomer takes over with probability greater than the neutral value $1/N$ if and
only if $x^*<\tfrac13$.
\end{theorem}

\noindent With $N=400$ and selection intensity $10^{-3}$, $N\rho$ was $1.021$, $1.009$, $0.991$ and $0.958$ at
$x^*=0.25$, $0.30$, $0.36$ and $0.45$. \chk{B16} The deterministic dynamics says that a new convention, protocol
or shared representation cannot invade from rarity at all when it needs partners; the finite population says
it can, by drift, and when it is more likely than chance to do so.

\begin{theorem}[The copying topology matters]\label{thm:graph}
\Sli[\cite{LHN05}] Let offspring of agent $i$ replace agent $j$ with probability $w_{ij}$. The fixation
probability of a variant of relative fitness $r$ equals the well-mixed value $(1-r^{-1})/(1-r^{-N})$ for all $r$
if and only if $W$ is doubly stochastic. A star amplifies selection. On a directed line, in which each
agent copies only its predecessor, fixation has probability $1/N$ whatever the fitness.
\end{theorem}

\noindent For $N=6$ and $r=1.2$ the exact values are $0.167$ on the line, $0.251$ well-mixed and $0.287$ on the
star. \chk{B17} The design reading is direct. Symmetric exchange between agents, of any shape, neither helps
nor hurts selection. A hub that many agents copy from and that copies from them sharpens it. A strictly
feed-forward pipeline of copying removes selection altogether: which variant ends up everywhere is decided by
where it started.

\begin{remark}[Populations of language-model agents]\label{rem:llm}
Recent work studies populations of agents built on large language models with exactly these tools: replicator
dynamics over agent types in open-source games \cite{OSG25}, evolutionarily stable strategies as a design
target for trust mechanisms \cite{EvT25}, group selection and replicator--mutator models for the evolution of
cooperative prompts \cite{GSP26}, evolutionary game theory as a probe of strategic behaviour \cite{SIL25}, and a
survey of the area \cite{XLLS26}. The titles and identifiers of these five items were checked on 17 November
2026; we have not evaluated their results. Our contribution to that programme is a set of constraints.
Two prompts that induce the same behaviour are clones, so an analysis of a prompt population should be natural
in the sense of \Cref{sec:setting}; stability should be asked of sets of prompts and not of single ones,
because stability of a point is destroyed by cloning and stability of a set is not \cite[Thm.~2.8]{EGHC}; and
conclusions drawn from the replicator equation about invasion by a single new agent are subject to the
correction of \Cref{thm:third}.
\end{remark}

\section{Inference}\label{sec:bayes}

\begin{proposition}[Bayes is the replicator]\label{prop:bayes}
\Scl[\cite{Sha09,Har09b}] Let $S$ be a set of hypotheses with likelihoods $\ell_h$. Bayesian updating of a prior
$x$ on an observation $o$ is one step of Hedge with $\eta=1$ and payoff $\pi_h=\log\ell_h(o)$. \chk{B18}
\end{proposition}

\noindent Bayesian inference is the replicator learner fed log-likelihoods, and the dictionary is exact.
Naturality for clone maps is the statement that splitting a hypothesis into two indistinguishable ones, with the
prior mass divided, changes no posterior probability of any event. The regret identity of \Cref{thm:regc} is
the statement that the log-loss of the Bayesian mixture exceeds that of the best hypothesis by at most
$\log(1/x_h(0))$. And the elimination of dominated strategies with its rate \cite{EGHC} is posterior
concentration: if observations come from $h^*$, the log-odds of any other $h$ against $h^*$ fall at the
expected rate $\KL{\ell_{h^*}}{\ell_h}$ per observation.

\section{Summary of results}\label{sec:summary}

\begin{table*}[t]
\centering\small
\begin{tabular}{@{}>{\raggedright}p{0.16\textwidth}>{\raggedright}p{0.3\textwidth}>{\raggedright}p{0.34\textwidth}l@{}}
\toprule
Topic & Qualitative statement & Quantitative statement & Checks\\
\midrule
Hedge and replicator & one is the other, sampled & exact at every step size & B1\\
Policy gradient & replicator with an extra factor $x_i$ & escape time $\sim1/(2D\epsilon)$ against $\log(1/\epsilon)/D$; ratios $3.0$, $11.8$, $73.3$ & B2, B3\\
Duplicated actions & (R), (H) natural; (PG), (Q) not & slowdown $2k/(k+1)$; defect $0.028$ & B19--B21\\
Regret & no regret & $\le\ln n$ continuous; $\le\sqrt{T\ln n/2}$ discrete; observed ratio $\le0.50$ & B4, B5\\
Zero-sum, flow & relative entropy conserved; recurrence & numerical drift $1.3\times10^{-15}$ over time $40$ & R4 of \cite{EGHC}\\
Zero-sum, Hedge & iterate leaves, average converges & $+\tfrac12\eta^2\Var$ per step; boundary after $1451$ steps; exploitability $0.0015$ & B6--B8\\
Optimism & last iterate converges & $0.070\to1.5\times10^{-10}$ in $6000$ steps & B9\\
Entropy regularization & converges; not natural; biased & rate at least $\alpha$, by an identity; type masses $(0.40,0.36,0.24)$ at $\tau=0.5$ & B10, B11, B21\\
Evaluation & ratings plus cycles plus harmonic & index $0.885$; curl $1.5$ & B12, B15\\
Redundant agents & averaging and name-mutation chain not natural; replicator average natural & $(0,0,0.25,-0.25)$; $(0.4,0.4,0.2)$; $(\tfrac13,\tfrac13,\tfrac13)$ & B13, B14\\
New conventions & can invade by drift & iff barrier $<\tfrac13$; $N\rho=1.021,\dots,0.958$ & B16\\
Copying topology & amplifiers and suppressors & $0.167<0.251<0.287$ & B17\\
Inference & Bayes is Hedge at $\eta=1$ & exact & B18\\
\bottomrule
\end{tabular}
\caption{Qualitative statements and their quantitative forms. All numbers are produced by the companion
script.}\label{tab:summary}
\end{table*}

\Cref{tab:summary} collects the results. Read down the middle column it is a list of facts most of which are
individually known; read across, each has a constant attached; and the recurring entry is naturality, which
sorts the methods into those that depend on how actions and agents are named and those that do not.

\section{Limits}\label{sec:limits}

\emph{Scope.} Everything here is tabular: a finite set of actions and exact payoffs. Function approximation,
sampling noise, and partial observation are outside it; they are where methods such as \cite{HMO20} do their
work. The numbers come from games with at most six actions and populations of at most a few hundred, chosen
to be exactly computable, and are illustrations of the statements and not benchmarks. No experiment on a
trained model was run.

\emph{Novelty.} The identifications of \Cref{prop:mwu,prop:pg,prop:bayes}, the regret bounds, recurrence, the
Hodge decomposition, the one-third law and the isothermal theorem are from the literature. The observations
are \Cref{thm:natural}, the identity of \Cref{thm:q}, \Cref{prop:evalnat}, and the placing of the three
repairs in \Cref{rem:repairs}. They are short, and parts of them are likely known in other forms; the
invariance of Nash averaging under redundancy, for one, is a stated motivation of \cite{BTPG18}.

\emph{How categorical.} Modestly. The category $\Game$, the functor to dynamical systems, and the operad of
wiring diagrams are ordinary category theory; the only cohomology is in degree one; the homotopy theory is in
the Conley index behind \Cref{rem:mcc}, which we invoke and do not compute. The language earns its place by one
question, whether a construction commutes with the maps, which turned out to separate methods that the
literature usually compares on other grounds.

\emph{References.} The bibliography was assembled from memory, except for \cite{OPP19,HMO20} and the five items
of \Cref{rem:llm}, whose bibliographic details were checked online on 17 November 2026. It should be verified
before it is relied on.

\emph{Authorship.} This paper was drafted with the assistance of an AI system (Anthropic's Claude), which also
wrote and ran the companion script.

\paragraph{Open problems.} \Spg (1) Characterize the learners that are natural for clone maps among those
derived from a regularizer; we expect entropy on types and not on names to be forced. (2) Give the
rate in \Cref{thm:q} for a non-uniform equilibrium, where the limit moves with $\tau$. (3) Extend
\Cref{prop:evalnat}(c) from time averages to chain components, so that a ranking by recurrent classes is
natural by construction. (4) Find the analogue of the isothermal theorem for populations of learners whose
fitness depends on the population, where the graph and the game interact.

\appendix
\section{The companion script}\label{app:checks}

\texttt{verify-evolutionary-learning.py} uses only the Python standard library, fixes its random seed, runs
in under half a minute, and exits with status zero if and only if its 21 checks pass. Flows are integrated by
a fourth-order Runge--Kutta scheme; fixation probabilities are exact. The checks test instances and recompute
the numbers quoted in the text. They do not prove the theorems.

\begin{center}\footnotesize
\begin{tabular}{@{}lp{0.82\columnwidth}@{}}
\toprule
B1 & Hedge equals the time-$\eta$ replicator flow, 30 random cases\\
B2 & policy gradient in logits; Fisher preconditioning\\
B3 & escape times, \Cref{prop:escape}, against numerical integration\\
B4 & continuous-time regret identity and bound\\
B5 & discrete regret bound, 40 runs, random and adversarial\\
B6 & exact drift of Hedge, \Cref{prop:drift}, three step sizes\\
B7 & monotone divergence to the boundary\\
B8 & exploitability of the time average\\
B9 & optimistic Hedge, last iterate\\
B10 & identity and rate of \Cref{thm:q}\\
B11 & rest points of \eqref{eq:q} are logit equilibria\\
B12 & Hodge decomposition of a planted example\\
B13 & mean score and replicator average under cloning\\
B14 & small-mutation chain: uniform, dominated, cloned\\
B15 & a harmonic class on a square of comparisons\\
B16 & one-third law, exact fixation, $N=400$\\
B17 & line, well-mixed, star: exact fixation, $N=6$\\
B18 & Bayes equals Hedge with $\eta=1$\\
B19 & naturality defect of (R) and (PG) under cloning\\
B20 & slowdown $2k/(k+1)$ for $k=1,2,4,8$\\
B21 & logit equilibrium with a cloned action, three temperatures\\
\bottomrule
\end{tabular}
\end{center}

\begin{thebibliography}{XLLS26}\footnotesize\setlength{\itemsep}{0.5pt}
\newcommand{\bi}[1]{\bibitem[#1]{#1}}
\bi{Alex23} J.~M.~Alexander, \emph{Evolutionary Game Theory}, Elements in Decision Theory and Philosophy, Cambridge University Press, 2023.
\bi{Ama98} S.~Amari, Natural gradient works efficiently in learning, \emph{Neural Comput.} 10 (1998), 251--276.
\bi{BaPi18} J.~P.~Bailey, G.~Piliouras, Multiplicative weights update in zero-sum games, in \emph{Proc.\ ACM Conf.\ Economics and Computation (EC)}, 2018, 321--338.
\bi{BoSa97} T.~B\"orgers, R.~Sarin, Learning through reinforcement and replicator dynamics, \emph{J.~Econom.\ Theory} 77 (1997), 1--14.
\bi{BTPG18} D.~Balduzzi, K.~Tuyls, J.~P\'erolat, T.~Graepel, Re-evaluating evaluation, in \emph{Advances in Neural Information Processing Systems} 31, 2018.
\bi{BTHK15} D.~Bloembergen, K.~Tuyls, D.~Hennes, M.~Kaisers, Evolutionary dynamics of multi-agent learning: a survey, \emph{J.~Artificial Intelligence Res.} 53 (2015), 659--697.
\bi{CBL06} N.~Cesa-Bianchi, G.~Lugosi, \emph{Prediction, Learning, and Games}, Cambridge University Press, 2006.
\bi{CGG22} G.~S.~H.~Cruttwell, B.~Gavranovi\'c, N.~Ghani, P.~Wilson, F.~Zanasi, Categorical foundations of gradient-based learning, in \emph{European Symposium on Programming (ESOP)}, 2022.
\bi{DaPa19} C.~Daskalakis, I.~Panageas, Last-iterate convergence: zero-sum games and constrained min-max optimization, in \emph{Innovations in Theoretical Computer Science (ITCS)}, 2019.
\bi{DISZ18} C.~Daskalakis, A.~Ilyas, V.~Syrgkanis, H.~Zeng, Training GANs with optimism, in \emph{Int.\ Conf.\ Learning Representations (ICLR)}, 2018.
\bi{EGHC} Anonymous, \emph{Evolutionary Game Theory in Higher Categories}, Higher Categorical Semantics, 2026. Companion volume.
\bi{EvT25} Wang et al., Ev-Trust, a trust mechanism for multi-agent systems of language models built on evolutionary stability, preprint, arXiv:2512.16167, 2025. Full title and author list not recorded when the listing was checked.
\bi{FrSc97} Y.~Freund, R.~E.~Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, \emph{J.~Comput.\ System Sci.} 55 (1997), 119--139.
\bi{FST19} B.~Fong, D.~I.~Spivak, R.~Tuy\'eras, Backprop as functor: a compositional perspective on supervised learning, in \emph{LICS 2019}, IEEE, 2019.
\bi{Goo14} I.~Goodfellow, J.~Pouget-Abadie, M.~Mirza, B.~Xu, D.~Warde-Farley, S.~Ozair, A.~Courville, Y.~Bengio, Generative adversarial nets, in \emph{Advances in Neural Information Processing Systems} 27, 2014.
\bi{GSP26} Group selection to evolve cooperative prompts in populations of LLM agents, listed for NeurIPS 2026; authors not recorded when the listing was checked.
\bi{Har09b} M.~Harper, The replicator equation as an inference dynamic, preprint, arXiv:0911.1763.
\bi{HMO20} D.~Hennes, D.~Morrill, S.~Omidshafiei, R.~Munos, J.~Perolat, M.~Lanctot, A.~Gruslys, J.-B.~Lespiau, P.~Parmas, E.~Du\'e\~nez-Guzm\'an, K.~Tuyls, Neural replicator dynamics: multiagent learning via hedging policy gradients, in \emph{Proc.\ AAMAS}, 2020; arXiv:1906.00190.
\bi{HoSi98} J.~Hofbauer, K.~Sigmund, \emph{Evolutionary Games and Population Dynamics}, Cambridge University Press, 1998.
\bi{Jad17} M.~Jaderberg, V.~Dalibard, S.~Osindero, W.~M.~Czarnecki, J.~Donahue, A.~Razavi, O.~Vinyals, T.~Green, I.~Dunning, K.~Simonyan, C.~Fernando, K.~Kavukcuoglu, Population based training of neural networks, preprint, arXiv:1711.09846.
\bi{JLYY11} X.~Jiang, L.-H.~Lim, Y.~Yao, Y.~Ye, Statistical ranking and combinatorial Hodge theory, \emph{Math.\ Program.} 127 (2011), 203--244.
\bi{Kak01} S.~Kakade, A natural policy gradient, in \emph{Advances in Neural Information Processing Systems} 14, 2001.
\bi{KwMe17} J.~Kwon, P.~Mertikopoulos, A continuous-time approach to online optimization, \emph{J.~Dyn.\ Games} 4 (2017), 125--148.
\bi{LHN05} E.~Lieberman, C.~Hauert, M.~A.~Nowak, Evolutionary dynamics on graphs, \emph{Nature} 433 (2005), 312--316.
\bi{LPS21} S.~Leonardos, G.~Piliouras, K.~Spendlove, Exploration-exploitation in multi-agent competition: convergence with bounded rationality, in \emph{Advances in Neural Information Processing Systems} 34, 2021.
\bi{LZG17} M.~Lanctot, V.~Zambaldi, A.~Gruslys, A.~Lazaridou, K.~Tuyls, J.~P\'erolat, D.~Silver, T.~Graepel, A unified game-theoretic approach to multiagent reinforcement learning, in \emph{Advances in Neural Information Processing Systems} 30, 2017.
\bi{MPP18} P.~Mertikopoulos, C.~Papadimitriou, G.~Piliouras, Cycles in adversarial regularized learning, in \emph{Proc.\ ACM-SIAM Symp.\ Discrete Algorithms (SODA)}, 2018, 2703--2717.
\bi{MSP73} J.~Maynard Smith, G.~R.~Price, The logic of animal conflict, \emph{Nature} 246 (1973), 15--18.
\bi{MXSS20} J.~Mei, C.~Xiao, C.~Szepesv\'ari, D.~Schuurmans, On the global convergence rates of softmax policy gradient methods, in \emph{Proc.\ Int.\ Conf.\ Machine Learning (ICML)}, 2020.
\bi{Mye21} D.~J.~Myers, Double categories of open dynamical systems (extended abstract), in \emph{Applied Category Theory 2020}, EPTCS 333, 2021, 154--167.
\bi{Now06} M.~A.~Nowak, \emph{Evolutionary Dynamics: Exploring the Equations of Life}, Harvard University Press, 2006.
\bi{NSTF04} M.~A.~Nowak, A.~Sasaki, C.~Taylor, D.~Fudenberg, Emergence of cooperation and evolutionary stability in finite populations, \emph{Nature} 428 (2004), 646--650.
\bi{OPP19} S.~Omidshafiei, C.~Papadimitriou, G.~Piliouras, K.~Tuyls, M.~Rowland, J.-B.~Lespiau, W.~M.~Czarnecki, M.~Lanctot, J.~Perolat, R.~Munos, $\alpha$-Rank: multi-agent evaluation by evolution, \emph{Sci.\ Rep.} 9 (2019); arXiv:1903.01373.
\bi{OSG25} Evaluating LLMs in open-source games, preprint, arXiv:2512.00371, 2025; authors not recorded when the listing was checked.
\bi{PaPi19} C.~Papadimitriou, G.~Piliouras, Game dynamics as the meaning of a game, \emph{ACM SIGecom Exchanges} 16 (2019), no.~2, 53--63.
\bi{PiSh14} G.~Piliouras, J.~S.~Shamma, Optimization despite chaos: convex relaxations to complex limit sets via Poincar\'e recurrence, in \emph{Proc.\ ACM-SIAM Symp.\ Discrete Algorithms (SODA)}, 2014, 861--873.
\bi{RaSr13} A.~Rakhlin, K.~Sridharan, Optimization, learning, and games with predictable sequences, in \emph{Advances in Neural Information Processing Systems} 26, 2013.
\bi{SaCr03} Y.~Sato, J.~P.~Crutchfield, Coupled replicator equations for the dynamics of learning in multiagent systems, \emph{Phys.\ Rev.\ E} 67 (2003), 015206.
\bi{Sha09} C.~R.~Shalizi, Dynamics of Bayesian updating with dependent data and misspecified models, \emph{Electron.\ J.~Stat.} 3 (2009), 1039--1074.
\bi{SIL25} Strategic intelligence in large language models: evidence from evolutionary game theory, preprint, arXiv:2507.02618, 2025; authors not recorded when the listing was checked.
\bi{Sor09} S.~Sorin, Exponential weight algorithm in continuous time, \emph{Math.\ Program.} 116 (2009), 513--528.
\bi{SSV20} P.~Schultz, D.~I.~Spivak, C.~Vasilakopoulou, Dynamical systems and sheaves, \emph{Appl.\ Categ.\ Structures} 28 (2020), 1--57.
\bi{TaJo78} P.~D.~Taylor, L.~B.~Jonker, Evolutionarily stable strategies and game dynamics, \emph{Math.\ Biosci.} 40 (1978), 145--156.
\bi{TVL03} K.~Tuyls, K.~Verbeeck, T.~Lenaerts, A selection-mutation model for Q-learning in multi-agent systems, in \emph{Proc.\ AAMAS}, 2003, 693--700.
\bi{Vin19} O.~Vinyals et al., Grandmaster level in StarCraft II using multi-agent reinforcement learning, \emph{Nature} 575 (2019), 350--354.
\bi{XLLS26} Xie, Liu, Lan, Szolnoki, Evolutionary games in LLM-agent societies: early milestones and perspectives, \emph{Chaos Solitons Fractals} 213 (2026), 119158. Initials not recorded when the listing was checked.
\end{thebibliography}
\end{document}
