Continuous, single-column reading. Equations, tables, proofs, references and appendices stay in document order.
Contents · 106 sections
Evolutionary Game Theory
in Higher Categories
Stability, stochastic selection,
and compositional models
Anonymous
November 2026
Population states and payoff maps
Evolutionary game theory uses a payoff comparison to describe how the composition of a population changes the prospects of its members. The state of a population is a distribution of strategic types. The same set of types can support many different models, since changing the matching process or the rule for converting payoffs to growth can change the resulting dynamics. This chapter isolates the distribution and payoff operations used in the pairwise, randomly matched model [1, Chs. 1–2].
Distributions and relabeling
For a nonempty finite set I of types, the population state space is the simplex Δ(I)={x∈RI:xi≥0,i∈I∑xi=1}. Write δi for the state concentrated on i. The support of x is the set of its positive coordinates. For a nonempty subset S⊆I, the inclusion Δ(S)→Δ(I) inserts zero in the omitted coordinates. Its image is a face, and its relative interior consists precisely of the states of support S. The affine hull of the full simplex is the hyperplane ∑ixi=1, whose translation space is TI={v:∑ivi=0}. Feasible directions at a boundary state are those v∈TI with vi≥0 whenever xi=0. Indeed, this is necessary for x+tv to have nonnegative coordinates at positive t. It is sufficient for small t because only finitely many of the positive coordinates can decrease.
A map of type sets f:I→J pushes a distribution forward by (f∗x)j=f(i)=j∑xi. Its linear extension uses the same formula on all vectors. If g:J→K, collecting the terms in the sums indexed by each fiber of gf gives (gf)∗=g∗f∗. The identity map gives the identity on distributions. Thus simplex formation is a functor from finite sets to convex sets and affine maps. It extends on all sets to DI, the set of finitely supported probability distributions. A map on Dirac states extends uniquely to an affine map out of DI, since every element is a finite combination ∑ixiδi and its coefficient function is uniquely determined by x. Accordingly a stochastic assignment of an output distribution to each type is determined by its values on types and then acts affinely on populations.
There is also a canonical way to combine mixtures of populations. For w∈DDI, its barycenter has coordinates μI(w)i=∑xwxxi. Taking the barycenter of a Dirac distribution on a population returns that population. Mixing Dirac distributions on individual types returns their original coefficient distribution. The two orders of flattening three layers of mixtures agree because in either order the coefficient of i is obtained by adding the same products of three weights. This gives the unit and multiplication of the finite-distribution monad. The construction uses only finite sums even when I is infinite. It differs from fixing a population size, since the weights may be any nonnegative reals adding to one.
Random matching and expected fitness
A symmetric two-player game on I assigns the row player’s payoff Aij when the row type is i and its opponent is j. The opponent receives Aji in that encounter. Symmetry is the exchange of player roles in this description. No symmetry of the entries of A is required. If the players draw types independently with distributions p,q, the expected payoff is uA(p,q)=i,j∑piAijqj=pTAq. Its two separate affine laws follow by collecting the coefficients of the product distribution. In particular, a randomly sampled opponent from a population x gives the fitness vector Ax, while the population average is uA(x)=xTAx. A correlated matching prescription instead supplies a conditional distribution of opponents for each row type. If these distributions are q(i,x), the corresponding fitness is Fi(x)=∑jAijq(i,x)j. General matching rules therefore need not produce a linear fitness map, even when the encounter payoffs are fixed.
The pairing in uA is functorial with respect to probability aggregation. It uses the transpose of the aggregation map to transport a score. For a score b∈RJ, define (f∗b)i=bf(i). Then i∑xi(f∗b)i=j∑(f∗x)jbj. This follows by collecting the left-hand terms over each fiber of f. The formula distinguishes the direction in which population weights move from the direction in which a fitness score is read back.
Exact maps between games
Consider the category whose objects are pairs (I,A) and whose maps (I,A)→(J,B) are type maps f:I→J satisfying Aij=Bf(i),f(j) for every i,j. Identity maps satisfy the same equation. If f and g are payoff-preserving, the comparison for gf follows by applying the two equalities successively. This category describes exact games with strategy maps. There are useful coarser notions of comparison, including maps that preserve only payoff differences, developed in Chapter [ch:payoff-equivalence].
Proposition 1.1 (Fitness under exact aggregation). If f:(I,A)→(J,B) is payoff-preserving, then Ax=f∗(Bf∗x),uA(p,q)=uB(f∗p,f∗q). Conversely, either the fitness identity at all population states or the payoff identity at all pairs of distributions implies payoff preservation.
Proof. Regroup the terms of (Ax)i=∑kBf(i),f(k)xk over the fibers of f. This gives the claimed fitness identity. The identity for payoffs then follows from the pairing between aggregation and score pullback. Conversely, evaluating the first identity at δj gives equality in each component i. Evaluating the second identity at (δi,δj) also gives equality of entries. ◻
An exact type map commutes with expected payoff.
A type bijection satisfying the comparison is a change of labels. A noninjective type map can identify distinct population states, while a nonsurjective map may leave types of the target unrepresented. If all target types have preimages, every target state can be represented by lifting its mass to selected types above its support. Such a lift need not be unique. Expected payoff agrees for all lifts by Proposition 1.1. A selection rule using the labels of the lifts can still see the difference. Passing to aggregated population states must therefore be justified for the particular rules one wants to transport.
Parameterized populations
Suppose payoffs vary continuously with an environmental parameter s in a space S, giving an array A(s). Evaluation (s,p,q)↦uA(s)(p,q) is continuous by its finite-sum formula. Substituting a map r:S′→S pulls back the family to A(r(s′)), and evaluations agree after the same substitution. If local descriptions use different names for the types, a transition is a bijection of the corresponding type sets. An exact transition then gives the comparison in Figure 1.1, and these comparisons compose because pushforward does. The simplex may consequently vary as a bundle with transition maps permuting its coordinates. Expected payoff is then a single continuous function on pairs of states over the same parameter, as long as the local payoff matrices obey the transition comparisons. The groupoid of labels keeps track of both the state and its change of presentation.
This setup will support two further comparisons. Static stability asks which distributions do well against populations with a small mutant component. A dynamics asks where trajectories go after an actual change in the population. The payoff operation is common to the two questions, and additional assumptions on the trajectory rule will enter later.
The exact payoff condition is closed under pullback of the type labels. Given B on J and any f:I→J, there is a unique matrix on I for which f is exact. Its entries are Bf(i),f(j). Pulling back twice gives the same matrix as pulling back along the composite. Thus the payoff spaces form a contravariant system over type sets, whereas the population spaces form a covariant system. Their comparison in the payoff square explains why treating both as covariant under a quotient of labels can change the model.
Evolutionary stability and invasion
Fix a population simplex Δ(I) and an expected payoff uA as in Chapter [ch:population]. We use the symmetric Nash condition uA(p,p)≥uA(q,p)(q∈Δ(I)). It says that no alternative earns more against p than a resident earns against p. An evolutionary stability test also asks what happens when the alternative changes the population against which these payoffs are evaluated. Maynard Smith and Price introduced the stability idea in the study of animal conflict [3]. Alexander develops its different formulations in [1, Ch. 2].
Best replies and the resident population
Let Mp=maxi(Ap)i. The value uA(p,p) is the p-weighted average of the numbers (Ap)i. A pure maximizer earns Mp against p, while the average cannot exceed Mp. Hence p satisfies the symmetric Nash condition if and only if uA(p,p)=Mp. Equivalently, every type in the support of p attains Mp. If any type with positive weight earned less, the weighted average would be strictly smaller than Mp.
This calculation also identifies the complete best-reply correspondence at p. It is the face spanned by those pure types at which Ap reaches its maximum. Strict symmetric Nash means all other population states q=p earn less against p. In the mixed strategy simplex this forces the best-reply face to consist of a single point. In particular, a resident supported on several types can satisfy the Nash condition without satisfying its strict version.
The invasion test and the two inequalities
For a resident p, a competitor q=p, and a mutant share 0<ε<1, put xε=(1−ε)p+εq. The pair p,q may consist of mixed strategies. The parameter ε specifies the share of the competitor in the population. It does not describe the probabilities within either of those two strategies. Define the payoff advantage in the resulting population by Dp,q(ε)=uA(p,xε)−uA(q,xε). The resident p is evolutionarily stable if for each q=p there exists a threshold b(q)>0 such that Dp,q(ε)>0 whenever 0<ε<min{1,b(q)}. The threshold in this definition can depend on q. A threshold valid for all competitors is a different quantifier placement, addressed in Chapter [ch:superiority].
Theorem 1.1 (Invasion criterion for pairwise games). A resident p is evolutionarily stable if and only if it is symmetric Nash and, for each q=p with uA(p,p)=uA(q,p), one has uA(p,q)>uA(q,q).
Proof. For each fixed competitor write d0=uA(p,p)−uA(q,p) and d1=uA(p,q)−uA(q,q). The affine law in the opponent argument gives Dp,q(ε)=(1−ε)d0+εd1. If this is positive on a sufficiently small interval of positive ε, continuity gives d0≥0. When d0=0, the same interval condition requires d1>0 by the displayed identity. Conversely, if d0>0, the advantage remains positive for all sufficiently small ε by continuity. If d0=0 and d1>0, the advantage is εd1>0 throughout the range of positive mutant shares. Applying the argument to every q=p gives the equivalence. ◻
The two terms are evaluated in different environments. The first evaluates both strategies against a purely resident population. The second evaluates both against the competing strategy. In the event of a first-order tie, the strictly positive second comparison explains why the resident gains a strict advantage when a positive mutant share is present. Every strict symmetric Nash equilibrium is evolutionarily stable, since none of its competitors ties in the first comparison.
For a general continuous fitness map the first comparison still gives a necessary condition for positive advantage near zero mutant share. The exact tie test in Theorem 1.1 uses affine dependence on the population. With nonlinear dependence, the advantage along the segment can have a nontrivial shape. An endpoint comparison then requires additional hypotheses before it determines the sign near the resident.
Neutral stability and sets of solutions
Neutral stability uses the weak inequality Dp,q(ε)≥0 in the invasion test. This standard variant allows a competing strategy to match the resident’s score when its share is small [2, Ch. 2]. The same proof, with a weak inequality on the tie condition, shows that p is neutrally stable exactly when it is symmetric Nash and d1≥0 for each distinct q with d0=0. Thus the solution predicates satisfy ESS(A)⊆NS(A)⊆Nash(A). These are subsets of the state space. An equality between local payoffs in the invasion test need not select one of the tied states. Persistence or attraction of the resident depends on the dynamics chosen to resolve or retain such ties.
The symmetric Nash condition defines a closed relation between payoff matrices and population states. Indeed, for each fixed pure type i it is the nonnegativity of the continuous difference uA(p,p)−(Ap)i, and the pure inequalities imply the conditions for all mixtures by averaging. An evolutionary stability relation adds a conditional strict inequality. For instance, along the parameter ray A(t)=tA for t>0, all comparisons in Theorem 1.1 retain their sign. At t=0 all comparisons vanish. If the simplex has more than one point, every resident is then neutral and no resident is strictly evolutionarily stable. Whenever the positive ray has an ESS this supplies a limiting failure of strict stability. Thus a family of games with a continuous payoff map may have different solution fibers at limiting parameter values.
Transporting the comparison
A label change transports all these predicates. To see this, apply a payoff-preserving bijection f:(I,A)→(J,B). The map f∗ respects the mixture xε by affinity and preserves its payoff by Proposition [prop:exact-fitness]. It also gives a bijection between the competitors of p and of f∗p. The inequalities and their quantifiers therefore agree. For a family with label transitions the stable locus can be described on each chart and glued as a subset of the simplex family by this argument. It retains the topology inherited from that family when the payoffs vary in a topological base.
For a payoff-preserving surjection f, the Nash predicate obeys the same comparison, since every competitor of f∗p can be lifted. Strict stability has an additional obstruction. If q=p has f∗q=f∗p, the payoff identity makes Dp,q(ε)=0 for every share. The lifted matrix can thus preserve all coarse payoff information and have additional tied alternatives upstairs. This is one reason to record the class of game maps before using it to compare solution sets.
A population formulation of best replies
The Nash relation has a useful description by the subset of types with maximal score. Let B⊆I be nonempty, and consider the part of the simplex where Bp=B. On this part, belonging to the best-reply face is equivalent to the coordinate condition pi=0 for all i∈/B. The region Bp=B is specified by equality of the scores on B and strict inequalities between the common score and each score outside B. For a fixed matrix these are linear equality and inequality constraints on p. Thus the support condition can be checked on these regions without a search over all mixed competitors. Evolutionary stability requires a further comparison on the face of tied competitors. That is a quadratic condition and need not reduce to checking only its vertices.
If p is strictly Nash, its payoff gap on the region of competitors at distance at least r>0 from p has a positive minimum whenever that region is nonempty. This follows from continuity on a closed subset of the compact simplex. There is generally no positive minimum over every distinct competitor, since a sequence of competitors can approach p. The existence of a uniform invasion barrier is instead a uniform statement about the relative contributions of the first and second payoff differences. The next chapter establishes it by examining the directions in which the first difference vanishes.
Local superiority and mixed strategies
The invasion test fixes a competitor and then varies its share. Local superiority compares the resident with every nearby population against that population itself. In finite-dimensional matrix games, critical directions at the resident relate these two descriptions. They also isolate the part of the payoff matrix that controls stability of a mixed resident. The equivalence with a uniform invasion barrier is discussed in [1, Ch. 2] and [2, Ch. 2].
The critical cone
Write TI={v:∑ivi=0}. For p∈Δ(I) let Cp={v∈TI:vi≥0 if pi=0}. Every displacement q−p belongs to Cp, and every v∈Cp is an actual displacement up to multiplication by a small positive scalar. Introduce the linear and quadratic forms Lp(v)=−vTAp,QA(v)=vTAv and the critical cone Kp=Cp∩kerLp. The Nash condition is exactly Lp≥0 on Cp. In one direction one scales each feasible vector to an actual displacement and applies Nash. In the other direction one applies the inequality to q−p for arbitrary q.
If q=p+v, the two terms in the invasion criterion are d0=Lp(v) and d1=Lp(v)−QA(v). It follows from Theorem [thm:invasion-criterion] and scaling of the quadratic form that p is an ESS⟺Lp∣Cp≥0,QA(v)<0(0=v∈Kp). The critical cone consists of directions with no initial loss against the resident. The quadratic condition applies only on that cone. At a pure resident with a unique best response, for example, there is no nonzero critical direction. At a boundary resident with additional tied best responses, the cone can include directions that add previously absent types. Restricting the calculation to the resident’s support can omit those directions.
Uniform superiority
Call p locally superior if uA(p,x)>uA(x,x) for every x=p sufficiently near p in the simplex. A uniform invasion barrier is one positive threshold that works for all distinct competitors in the definition of evolutionary stability.
Theorem 1.1. For a matrix game on a finite type set, evolutionary stability, local superiority, and the existence of a uniform invasion barrier are equivalent.
Proof. Suppose p is an ESS. For v=x−p the superiority difference is uA(p,x)−uA(x,x)=Lp(v)−QA(v). If there is no nonzero feasible direction, the local condition is vacuous. Otherwise put S=Cp∩{v:∥v∥=1}, a compact set. On its critical subset S∩Kp, the quadratic form is negative. The relatively open subset of S on which QA<0 contains this critical set. On its complement, if nonempty, Lp has a strictly positive minimum m by compactness and the fact that its zeros are critical. Also ∣QA∣ has a finite maximum M on S. For w∈S and r>0 we have Lp(rw)−QA(rw)=r(Lp(w)−rQA(w)). It is positive wherever QA(w)<0 because Lp≥0. On the complementary subset it is positive for 0<r<m/(1+M). This proves local superiority, including the case in which the complementary subset is empty.
Now assume local superiority on a neighborhood of radius δ>0. The simplex has finite diameter D. For each mutant q=p, the invader mixture xε=(1−ε)p+εq satisfies 0<∥xε−p∥≤εD. The identity uA(p,xε)−uA(xε,xε)=ε(uA(p,xε)−uA(q,xε)) follows from affinity in the first argument. Hence every share below min{1,δ/(1+D)} gives positive invasion advantage independently of q. This yields a uniform invasion barrier. A uniform threshold gives the pointwise thresholds of evolutionary stability immediately. ◻
Two features of the proof have distinct roles. Compactness of the unit directions controls approach to the critical cone. Boundedness of the simplex controls the passage from nearby populations to a common mutant share for every q. General strategy spaces can lose either property. The finite-dimensional proof cannot then be transferred without a replacement estimate.
Interior residents and the quadratic test
In the relative interior of Δ(I), the cone Cp is the whole tangent space TI. A linear form nonnegative on a vector space is zero. The Nash condition therefore says that Ap is constant in the type index. For an interior Nash resident, Kp=TI, and evolutionary stability is equivalent to the restriction of the symmetric part (A+AT)/2 being negative definite on TI. Indeed vTAv only sees that symmetric part. When A=AT, this is the Hessian test for the quadratic function x↦xTAx/2 restricted to the affine hull of the simplex. Its differential at a Nash state vanishes along TI, and its exact expansion has quadratic term QA/2. When A is not symmetric, the argument establishes the sign of the critical quadratic form. The derivative of this same quadratic function contains the symmetric part and in general differs from the fitness vector Ax.
Negative definiteness on TI also implies uniqueness of a symmetric Nash state whenever one exists. If p,q both satisfy Nash, adding uA(p,p)≥uA(q,p) and uA(q,q)≥uA(p,q) gives (p−q)TA(p−q)≥0. Since p−q∈TI, this forces equality of the states under negative definiteness.
Persistence in payoff families
The interior test has a direct parameter consequence. Let Π be the Euclidean projection onto TI. If QA<0 on TI∖{0}, the operator ΠA∣TI is invertible. Indeed, a vector v in its kernel satisfies vTAv=0. Injectivity then gives invertibility on the finite-dimensional space TI. For any fixed b in the affine hull of the simplex, the unique solution of ΠAp=0 in that affine hull is pA=b−(ΠA∣TI)−1ΠAb. The expression varies continuously with A near a matrix for which the operator is invertible. Strict positivity of each coordinate is open. Negative definiteness is also open, by taking the maximum of QA over the compact unit sphere in TI when TI=0. Thus an interior ESS satisfying this test persists through sufficiently small payoff perturbations, and in that neighborhood the displayed formula gives the unique Nash state. Relabeling commutes with the zero-mean projection, and uniqueness makes these locally computed states agree wherever the labels change. This is an explicit instance of transport of a solution in a parameter family.
A local margin and continuation on a face
The compact-direction argument yields a useful quantitative version. If p is an ESS and the feasible unit set is nonempty, there are constants c,δ>0 such that uA(p,x)−uA(x,x)≥c∥x−p∥2(∥x−p∥<δ). Indeed, if the critical unit set is nonempty, choose a>0 with QA≤−a there. On the directions where QA<−a/2, the difference at v=rw is at least (a/2)r2. On the closed complement the minimum of Lp is some m>0. For r<min{1,m/(2(1+M))}, the difference is at least (m/2)r≥(m/2)r2. If either region is empty only the other bound is needed. When the critical unit set is empty the second estimate applies on all directions. The lower bound is an estimate on payoffs. Its use as an estimate on motion requires a dynamics whose drift is related to those payoffs.
At a Nash point, let Bp={i:(Ap)i=Mp}, where Mp=maxi(Ap)i. Since ∑ivi=0, one has Lp(v)=i∈/Bp∑(Mp−(Ap)i)vi. All terms on the right are nonnegative for feasible directions because every omitted index has zero weight in p. Consequently v∈Kp precisely when v∈Cp and vi=0 outside Bp. If every absent type has a strictly lower payoff, then Bp=supp(p)=S. The critical cone is simply the tangent space to Δ(S). An ESS of this kind with a strict quadratic test on that tangent space persists on the same face under small matrix perturbations: solve the interior equation for the restricted matrix, extend by zero, and use continuity to preserve the payoff gaps to all absent types. Any Nash state sufficiently near p for a sufficiently small perturbation still has positive coordinates on S, and the same payoff gaps prevent extra positive coordinates outside S. Its uniqueness therefore follows from the restricted affine equation. Here the strict gaps and the curvature on the face control different aspects of the same continuation.
Payoff equivalence and spaces of games
The payoff array contains information about the absolute score of each encounter. The static comparisons in Chapter [ch:ess] and the usual replicator field use only relative scores against the same population. This difference suggests an equivalence relation on games with fixed type labels. Once labels are also allowed to vary, the resulting space of games has a groupoid presentation. Its arrows identify numerical presentations of the same relative-fitness model.
The quotient by common fitness increments
Fix a nonempty finite type set I, put n=∣I∣, and write 1 for its constant vector. Let AI=End(RI),WI={1bT:b∈RI}. If B=A+1bT, then Bx=Ax+(bTx)1. The increment is common to every row type, though its size can depend on the opponent distribution x. For p,q,x in the simplex we have (p−q)T(B−A)x=0, since p,q have the same total mass. Thus comparing p and q against x gives identical advantage in the two games. This includes both comparisons in the ESS test.
Define the zero-mean projection ΠIv=v−1(∑ivi)/n. Its kernel is the span of 1 and its image is TI. The map A↦ΠIA has kernel WI: each column is killed precisely when that column is constant across the rows. It maps onto the linear maps RI→TI, since the projection acts as the identity on their image. It follows that QI=AI/WI≃Hom(RI,TI). Each class has a canonical representative with zero column sums. Alternatively, knowing the relative scores against every population state determines this class. Indeed the relative scores give all row differences in (A−B)x. If these vanish for all x, evaluation at each pure state shows A−B∈WI.
Proposition 1.1 (The quotient determined by replicator motion). Write XA for the polynomial vector field on the simplex given by (XA(x))i=xi((Ax)i−xTAx). Then XA=XB on the whole simplex if and only if A−B∈WI. If B=λA+1bT for constant λ>0, then XB=λXA and the static stability predicates agree.
Proof. The common fitness increment cancels in the expression for XA because the weights of x add to one. Scaling every fitness by λ scales both terms in the difference and every payoff comparison by the same positive number. This proves the forward formulas.
Conversely, take x in the relative interior of the simplex. Each xi>0, so equality of the fields implies ((A−B)x)i=xT(A−B)x for all i. Thus ΠI(A−B)x=0 on the relative interior. Continuity gives zero on its closure, and in particular on every pure state. The column argument then places A−B in WI. ◻
If a solution curve for XA is parameterized by t, changing its parameter to λt gives a solution for XB on the corresponding time interval. Positive scaling therefore gives a comparison of time units. Keeping time fixed gives the sharper quotient QI. Neither equivalence identifies the unadjusted absolute payoffs. Those retain a role if another process, such as a baseline contribution to fitness, is sensitive to their absolute levels.
Relabeling as a groupoid action
Let SI be the permutation group of the type set. A permutation σ acts by A↦PσAPσ−1, where Pσδi=δσ(i). It preserves WI, since it fixes 1. It also commutes with ΠI, since it preserves the sum of the coordinates. The action therefore passes to QI. Its action groupoid has objects q∈QI and arrows q→q′ given by permutations σ for which q′=σq. Two matrices in the same orbit determine isomorphic objects. The permutations fixing a class are the automorphisms of that object. Passing to the set of orbits forgets this group. Accordingly, the orbit set classifies a single game up to relabeling, while the groupoid can also record changes of presentation along a parameter space.
One can equivalently keep the raw matrices and allow translation by WI in the arrows. The group WI⋊SI acts on AI by (w,σ)A=σA+w. Its multiplication uses the evident permutation action on WI. Consider the functor of action groupoids AI//(WI⋊SI)⟶QI//SI sending A to its class and an arrow (w,σ) to σ. It is fully faithful: if the classes of A,B satisfy [B]=σ[A], then w=B−σA belongs to WI and is the unique translation giving an arrow above σ. It is essentially surjective by the canonical normalized representative ΠIA in each class. A quasi-inverse uses this representative, sending each permutation to an arrow with translation zero. The comparison back to a raw matrix is translation by its difference from its normalized form. The formulas are continuous and even smooth as functions of the matrices. Consequently they also give equivalent presentations for families with continuous or smooth transition data. Here the removal of redundant payoff coordinates gives an equivalence of groupoids because the arrows over each permutation are recovered uniquely.
Canonical normalization respects every permutation of labels.
Aggregation and local labels
For a parameter base S, a family with local labels can be given on an open cover by maps to QI along with permutations on overlaps satisfying the cocycle law. The comparison on overlaps is the one in Figure 1.1, and population states transform by the same permutations. Thus the payoff family and population bundle can carry nontrivial label transport. For example, a loop can act by a permutation fixing the payoff class and moving some points of the simplex. A solution family that is required to be a section must respect that transport. A numerical solution set at a single parameter point does not specify how to continue the individual solutions around such a loop.
A payoff-preserving surjection of type sets is a different operation. It can remove duplicate labels by summing their population weights. Its fibers over interior mass assignments can have positive dimension. It therefore fails to identify the underlying state spaces even when the expected payoff descends. In particular, if a state puts positive mass above a target type that has at least two preimages, a nonzero transfer of mass within that fiber leaves its image unchanged. Chapter [ch:ess] shows that such a transfer prevents the specified lift from being an ESS. The groupoid equivalence for normalization uses actual comparisons with inverses. A coarse aggregation needs a separate argument for each invariant transported along it.
Games with selected residents
The total solution space is more informative for families than a list of stable residents for each matrix. For the ESS predicate, form the subspace SI={(q,p)∈QI×Δ(I):p is evolutionarily stable at q}. The definition is independent of the representative of q. Permuting the game and the resident together preserves this subspace, by the payoff computation in Chapter [ch:ess]. Projection therefore gives a functor SI//SI→QI//SI. In a parameter family, a selected resident is a lift along this projection. In local coordinates it consists of a choice on each chart and compatibility of the choices on every overlap.
Over a chosen game class with a specified identification of its labels, the stable residents form the ordinary set ESS(q). If one instead restricts to the full subgroupoid of games isomorphic to q, then the category of marked residents is equivalent to the action groupoid of Stab(q) on ESS(q). Indeed, bringing a game in that orbit back to q takes its resident to one at q, and different choices of a permutation differ by a stabilizer element. The groupoid records how residents can be identified by changes of labels. This construction only uses the stated stability predicate and its invariance under permutations. It applies equally to the Nash and neutral predicates.
Replicator flows and mutation
From abundance to proportion
Let S be a nonempty finite strategy set and write Δ(S) for its probability simplex. A fitness assignment is a map F:Δ(S)→RS. Its i-th entry is the rate associated with strategy i, evaluated in the population x. For a matrix A, the assignment F(x)=Ax evaluates the expected payoff of each strategy against the population. The derivation below applies to nonaffine assignments as well. Taylor and Jonker give an early dynamical treatment, and Alexander explains the abundance and revision interpretations of the equation [4, 1].
Suppose ni is the abundance of strategy i and that n˙i=niFi(x), where xi=ni/∑jnj. Writing M=∑jnj and ϕ(x)=∑jxjFj(x), one obtains M˙=Mϕ(x). The quotient rule gives x˙i=xi(Fi(x)−ϕ(x)). The subtraction of mean fitness is thus the correction due to normalization. Conversely, if x solves (1), then M(t)=M(0)exp(∫0tϕ(x(s))ds) recovers a positive total abundance and ni=Mxi solves the abundance equation. The abundance formulation uses independence of the fitness assignment from the total. If the rates depend on total abundance, the proportions generally retain that dependence and need an additional state variable.
There are two simple symmetries of (1). Adding c(x) to every Fi(x) adds c(x) to ϕ(x) and leaves the field fixed. Relabeling S transports x and F together. For an affine assignment, adding the same row vector to each row of the payoff matrix is one instance of the first symmetry. Equality of vector fields records a specific observational equivalence of fitness data.
The simplex and its faces
Assume the fitness assignment extends to a locally Lipschitz map on a neighborhood of Δ(S). On that neighborhood form ϕ(x)=∑jxjFj(x) without requiring ∑jxj=1. For a local solution x set m(t)=∑jxj(t). Then m˙=(1−m)ϕ(x). If m(0)=1, the integrating factor for this scalar equation implies m(t)=1 throughout its interval. Independently, each coordinate satisfies xi(t)=xi(0)exp(∫0t(Fi(x(s))−ϕ(x(s)))ds). This identity includes coordinates starting at zero, because the scalar linear equation along a given solution has the zero solution for such an initial value.
Proposition 1.1 (Existence and restriction to faces). For the locally Lipschitz assignment just specified, (1) has a unique solution for every initial state of Δ(S), defined for all real times. Throughout any finite time interval its support equals the initial support. On the face indexed by a nonempty subset T⊆S, the solution is the replicator solution for the restricted assignment on Δ(T).
Proof. Local existence and uniqueness follow from the locally Lipschitz field. Conservation of mass and (2) keep a solution in the simplex throughout its interval of existence, in either direction of time. A solution confined to this compact subset of the domain can be continued at each finite endpoint. Support preservation follows from the positivity of the exponential. On the face the terms xjFj(x) with j∈/T vanish from the mean, and each missing coordinate has zero velocity. Uniqueness identifies the restricted trajectory with the solution calculated directly in the face. ◻
In particular, if jT:Δ(T)→Δ(S) extends a state by zero, and FT restricts F∘jT to the coordinates in T, the solution operators obey the commutative square Here both vertical maps are jT. The group law Φs+t=Φs∘Φt follows by uniqueness applied after a change of time origin. A relabeling yields the same square with the vertical maps replaced by permutations. These comparisons fix the sense in which the construction respects the data of a game. A map between strategy sets that identifies distinct types needs separate hypotheses to carry the differential equation through the induced map on states.
A rest state has equal fitness ϕ(x) on its support. This condition is sufficient because the velocity of every absent type is already zero. A Nash state also bounds the payoffs of absent types above by ϕ(x). The distinction is caused by support preservation. In the interior the two conditions coincide. For any pair of present types, dtdlogxjxi=Fi(x)−Fj(x). Thus a constant lower bound γ>0 for the payoff difference along an interval [0,t] gives xi(t)/xj(t)≥eγtxi(0)/xj(0). The inference concerns the relative growth of two present types. It yields no positive mass for a type initially absent.
Two mutation conventions
A revision generator Q has nonnegative off-diagonal entries and zero row sums, with Qji denoting the rate from j to i for j=i. Independent mutation during the evolution of the proportions gives x˙i=xi(Fi(x)−ϕ(x))+j∑xjQji. The generator contributes zero to the mass derivative. On the simplex the contribution to a vanishing coordinate is the sum of the incoming rates weighted by the masses. This property yields forward invariance even when the initial state lies on the boundary. To prove it, in the ambient equation replace xj by max(xj,0) in the incoming terms with j=i. The modified problem is locally Lipschitz. In the equation for coordinate i the terms involving xi are linear in xi along a given solution, and the remaining forcing is nonnegative. Variation of constants proves that all coordinates stay nonnegative for t≥0. The modification consequently agrees with the original equation on this solution. The mass identity then retains m=1, and compactness permits global forward continuation.
For a constant Q, a positive coordinate remains positive by the same variation formula. If Qji>0 and coordinate j is positive for t>0, its forcing makes i positive for t>0 as well. Induction along directed edges shows that irreducibility, meaning directed reachability between all strategies, produces full support at every strictly positive time. Mutations change the invariant face structure present in (1). They also destroy its backward invariance on the closed simplex in general.
Another convention incorporates mutation into reproduction. Let R be a row stochastic matrix of offspring probabilities and let bj(x)≥0 be an absolute birth rate. With fitness used in these units the normalized equation is x˙i=j∑xjbj(x)Rji−xij∑xjbj(x). This follows by normalizing the abundance equation with incoming births n˙i=∑jnjbj(x)Rji. Setting R equal to the identity recovers selection, and the sum of the velocities on the simplex is zero. Under the same local Lipschitz hypothesis, forward invariance follows by the incoming-rate argument. For that argument the rates can be extended off the simplex by their positive parts so that incoming forcing remains nonnegative. For the diagonal term the coefficient is biRii−∑jxjbj.
A common baseline added to bj changes (4) by c(RTx−x). This describes mutation occurring during additional births. In (3), the corresponding baseline change to F has no effect. The two conventions therefore require different information about which transformations of fitness preserve the model. The formulation of an evolution operator should specify where mutation enters before passing to equivalence classes of payoff data [6].
Time changes and the strength of mutation
Multiplying a fitness assignment by a positive constant a changes the replicator solution from x(t) to x(at). The geometric trajectories are the same and their parametrizations differ. The same rule holds for a positive continuous scalar a(x) on the simplex. Along a trajectory of the rescaled field, the clock for the original field has rate a(x(t)). On the compact simplex, the positive minimum of a ensures that this clock advances without bound as physical time grows. Thus this kind of rescaling preserves forward limit sets as well as rest points.
In the additive mutation model, the same clock rescales both selection and the generator. Holding Q fixed while rescaling F can change the stationary states. More precisely, if x is stationary for both fitness F and fitness aF with the same Q and a constant a=1, subtraction of the two stationary equations gives RF(x)=0. The original equation then gives QTx=0. Conversely, a common zero of these two fields is stationary for every relative scaling. When Q is irreducible this test picks out at most its unique stationary distribution [6]. Consequently, fitness units and mutation rates carry joint information when the model includes both mechanisms.
Invasion and Lyapunov geometry
Relative entropy near a resident state
A state p∈Δ(S) is locally superior for F if (p−x)⋅F(x)>0 for every distinct x sufficiently close to p in the simplex. This formulation compares the resident with a nearby population in the environment of that population. For symmetric matrix games it is equivalent to the usual evolutionarily stable strategy condition discussed in Chapter [ch:ess]. The implication to dynamical stability is the classical result studied by Hofbauer, Schuster, and Sigmund [5, 1].
Set Up={x∈Δ(S):xi>0 whenever pi>0}. The quantity D(p∥x)=i:pi>0∑pilogxipi is finite on Up, including when p lies in a proper face. The bound −logz≥1−z at z=xi/pi yields D(p∥x)≥1−i:pi>0∑xi≥0. Equality demands xi=pi on the support of p, which also forces the remaining entries of x to vanish. The derivative along (1) is dtdD(p∥x(t))=x(t)⋅F(x(t))−p⋅F(x(t)). The cancellation uses ∑ipi=1. Support preservation makes the logarithms legitimate for all finite times whenever they are defined initially.
Theorem 1.1 (Local stability from superiority). For the replicator equation of Chapter [ch:replicator], a locally superior state p is a locally asymptotically stable rest point relative to Δ(S).
Proof. For any q∈Δ(S), approach p along the segment x=p+ϵ(q−p). Dividing the superiority inequality by ϵ>0 and taking its limit shows (q−p)⋅F(p)≤0. Thus p is Nash and is a rest point of the replicator field. Take a closed relative ball around p contained in Up and in the neighborhood where superiority holds. Taking the ball small enough is possible even when p lies on the boundary because only its positive coordinates need to stay positive. If the simplex is a singleton, stability is immediate. Otherwise, for a sufficiently small radius the relative sphere is nonempty, and the minimum there of D(p∥−) is positive by compactness. Any trajectory starting inside with entropy below this minimum stays inside. By taking the radius arbitrarily small we obtain Lyapunov stability.
For such a trajectory, the entropy is nonincreasing and bounded below. If a point z=p were a limit point, its entropy derivative would be strictly negative. The trajectory has bounded speed on the closed ball. It would therefore spend intervals of a fixed positive duration where the derivative is bounded above by a negative constant, on arbitrarily late visits near z. Taking disjoint visits contradicts convergence of the entropy values. The limit points are consequently all p. The trajectory lies in a compact set and hence converges to p. ◻
The same argument supplies a global conclusion on Up if p is a Nash state and (x−p)⋅(F(x)−F(p))<0(x=p) on the whole simplex. The Nash inequality implies (x−p)⋅F(x)<0. A finite sublevel set of D(p∥−) stays away from zeros of the required coordinates, so the compact-set argument applies there. For completeness, each term with pi>0 is bounded below by pilogpi, while the term for xi tends to infinity at xi=0. This gives positive lower bounds for the required coordinates on any one sublevel set. This strict inequality is a special sufficient condition. Hofbauer and Sandholm establish broader geometric criteria for convergence in stable population games [8].
Selection as a metric gradient
At an interior state x, tangent vectors are vectors u with ∑iui=0. The Shahshahani metric is gx(u,v)=i∑xiuivi. The same definition applies to the relative interior of each face by restricting the coordinates to its support [7, 6]. Define the fitness one-form on a face by αx(u)=∑iFi(x)ui. If RF denotes the replicator vector field, its metric pairing with u is gx(RF(x),u)=i∑(Fi(x)−ϕ(x))ui=αx(u). The field is the metric dual of the payoff one-form. Adding a state-dependent baseline to all payoffs leaves the one-form unchanged, because all tangent vectors have zero sum.
Pointwise, every tangent covector has a representative in the ambient strategy coordinates, and two such representatives differ by a multiple of 1. Writing X for the relative interior of the full simplex, the fitness, one-form, and field correspond by Here Rx denotes the linear map that projects a fitness vector by its resident mean and multiplies by the current frequencies. At a proper face the corresponding metric duality uses only supported coordinates. All payoff coordinates outside the support then disappear from the tangent covector, even though they are needed to evaluate invasion by absent types.
If α=dV for a differentiable potential V, this duality says that RF is the Riemannian gradient of V. The rate of increase of the potential is dtdV(x(t))=gx(RF,RF)=i∑xi(Fi(x)−ϕ(x))2. Only supported coordinates enter this identity. For a potential continuously differentiable on the closed simplex in its affine hull, the right side extends continuously to every face. Compactness bounds the potential and the speed of the trajectory. As in the entropy argument, a nonrest limit point would give repeated fixed-duration intervals of strictly positive increase, which is impossible. This shows that each limit point is a rest point. It leaves convergence to a particular rest point open when the rest set has several possible limit points.
For matrix payoffs, write A=S+K with S symmetric and K skew-symmetric. When K=0, one can take V(x)=xTSx/2. More generally A defines a potential on the interior precisely when the bilinear form uTAv is symmetric on tangent vectors. Indeed, the exterior derivative of the payoff one-form on constant tangent vectors is vTAu−uTAv. This must vanish for a potential, and it suffices because the open simplex is convex. Thus a skew part matters for integrability only through its restriction to tangent directions.
Dissipation and a conservative obstruction
At an interior rest point p of a matrix game, Ap=c1. Consequently (x−p)⋅Ax=(x−p)TS(x−p). The right side depends only on the symmetric part because uTKu=0. Strict negative definiteness of S on the tangent hyperplane gives strict entropy dissipation. The skew part can nevertheless change the direction of the velocity. For a purely skew field A=K with Kp=0, one instead has ϕ(x)=0 and pTKx=0 everywhere. Equation (5) makes D(p∥x) a first integral on the interior. Every trajectory from a distinct interior state has positive constant entropy, so it cannot converge to p. This obstructs asymptotic stability regardless of whether individual level sets carry periodic motion.
Suppose that p is interior and that there is a strict uniform bound of the form (x−p)⋅F(x)≤−c∥x−p∥2 near p, with c>0. The entropy Hessian in the ambient positive orthant is diag(pi/xi2). This Hessian is bounded above and below by positive definite diagonal matrices on a small closed neighborhood of p. The first derivative of entropy at p vanishes on the tangent space. Taylor’s theorem therefore gives positive constants a,b such that a∥x−p∥2≤D(p∥x)≤b∥x−p∥2. An initial state close enough stays in this neighborhood by Lyapunov stability. Then (5) implies exponential decay at rate at least c/b for the entropy. This argument uses the full support of the resident. At a boundary resident, redistributing mass to an absent strategy can give a linear term in the expansion. For a fixed q with mass m outside the support of p, direct differentiation yields D(p∥p+ϵ(q−p))=ϵm+O(ϵ2)(ϵ↓0). The general stability implication accommodates this boundary term through the entropy sign and compactness.
These statements concern the vector field associated with a given payoff rule. Equation (6) uses the exact derivative of a potential and (5) uses the comparison with a resident. They give separate ways to turn a strategic inequality into a dynamical conclusion, with distinct hypotheses at the boundary and on the rest set.
Revision protocols and response dynamics
Response as a differential inclusion
A payoff assignment gives current incentives. A revision protocol determines which of those incentives are used when a strategy changes. This extra datum affects boundary motion, the identification of rest points, and convergence. Alexander and Sandholm develop the passage from revision protocols to population dynamics [1, 10].
Write B(x)=y∈Δ(S)argmaxy⋅F(x). Because the objective is linear in y, this is the nonempty convex face spanned by the maximizers of Fi(x). Best-response revision uses the differential inclusion x˙(t)∈B(x(t))−x(t). The solution convention is an absolutely continuous path satisfying the inclusion almost everywhere. If F is continuous, the graph of B is closed. In fact, taking a limit in the maximizing inequalities for yn∈B(xn) shows that any limit y maximizes against the limit x. Compactness of the simplex then gives upper semicontinuity. One may extend the correspondence to the ambient Euclidean space by precomposing B with the nearest-point projection onto the simplex. The standard existence theorem for locally bounded upper semicontinuous differential inclusions with nonempty compact convex values supplies a local solution [11].
For a solution, b(t)=x˙(t)+x(t) belongs to the simplex almost everywhere. The integrating factor gives x(t)=e−tx(0)+∫0te−(t−s)b(s)ds. The coefficients form a probability weighting. This keeps the path in the simplex and hence within the region where the original correspondence is defined. The velocity is bounded there, so the path extends to every forward time by continuation. Existence requires no choice of best response at a tie. There may be several paths from the same initial state. A point admits a constant path exactly when x∈B(x), the Nash condition. Convergence results for this inclusion require substantial further structure. For instance, Hofbauer and Sorin study continuous concave–convex zero-sum games and identify convergence to the saddle-point set [12].
Pairwise comparison
Suppose a reviser currently using j switches towards i at rate ρ(Fi(x)−Fj(x)), absorbing any common uniform sampling frequency into time. Let ρ be nonnegative and locally Lipschitz, strictly positive for positive arguments and zero for nonpositive arguments. The drift of xi is the difference between incoming and outgoing flux: Wi(x)=j∑xjρ(Fi(x)−Fj(x))−xij∑ρ(Fj(x)−Fi(x)). Each individual transfer contributes with opposite signs to the two relevant coordinates. Summing therefore gives zero, and the rate at an absent coordinate is nonnegative. If F is locally Lipschitz, the incoming-term variation argument of Chapter [ch:replicator] shows that the resulting equation has a unique global forward trajectory from each simplex state.
Proposition 1.1 (Rest states under comparison). A state is a rest point of (7) if and only if it is Nash.
Proof. Reindexing the sum of the outgoing terms gives F(x)⋅W(x)=i,j∑xjρ(Fi(x)−Fj(x))(Fi(x)−Fj(x))≥0. Every summand is nonnegative, and it is positive exactly when xj>0 and Fi(x)>Fj(x). At a rest point no such pair exists. Thus every used strategy maximizes payoff, so x∈B(x). Conversely, if x∈B(x), every transition away from a used strategy has rate zero. Both flux terms vanish. ◻
This is the positive-correlation argument for pairwise comparison dynamics [9]. It also yields a strict increase of the potential along nonstationary paths when the payoff one-form is exact. In a related protocol, sample the candidate from the current population and use [Fi−Fj]+=max(Fi−Fj,0) as the switch rate conditional on the candidate. The flux from j to i is then xjxi[Fi−Fj]+. Combining it with its reverse flux and using [a]+−[−a]+=a recovers (1). This latter sampling convention cannot create an absent type. The state-independent availability of candidates is the hypothesis that makes the preceding characterization include the boundary Nash inequalities.
Logit revision and regularization
Fix a noise parameter η>0. The logit response is Biη(x)=∑jexp(Fj(x)/η)exp(Fi(x)/η). The smooth response map from fitness vectors to probabilities preserves common payoff baselines and is equivariant under strategy relabeling. For a bijection S→S′ with permutation map P the latter statement is The two vertical maps apply P. The result follows because the normalization is a sum over the strategy set. A many-to-one map need not commute with the same response, since several copied entries contribute several terms to that sum.
In detail, let q:S′↠S be a surjection and pull a payoff vector f back to f∘q. The mass assigned to the fiber of i is proportional to ∣q−1(i)∣exp(fi/η). The effective coarse payoff includes a shift of ηlog∣q−1(i)∣, which cancels from the response when the fiber sizes agree. More generally, using positive reference weights wj on S′ makes the mass of a fiber proportional to exp(fi/η)∑j:q(j)=iwj. The weighted response therefore commutes with lumping for pulled-back fitness vectors when one pushes the reference weights forward. To extend this calculation to a semiconjugacy of the revision equations, the fine fitness must be constant within each fiber and must be computed from the coarse state by the same rule at every instant.
One can derive the logit rule by regularizing the response optimization. Let H(y)=−∑iyilogyi, using the continuous extension 0log0=0. For fixed x, write Z=∑jexp(Fj(x)/η). Substituting the definition of Bη gives y⋅F(x)+ηH(y)=ηlogZ−ηD(y∥Bη(x)). The identity is valid also for y on a proper face because the second argument of entropy has full support. By the nonnegativity established in Chapter [ch:lyapunov], the unique regularized maximizer is Bη(x). Compare its objective value with that of a pure maximizing strategy, which has entropy zero. Since H(y)≤log∣S∣, 0≤imaxFi(x)−Bη(x)⋅F(x)≤ηlog∣S∣. The entropy bound follows by computing relative entropy against the uniform distribution. In particular, each logit rest point is an ηlog∣S∣-approximate Nash state, in the sense that this is a bound on the gain from any unilateral population-level deviation.
The dynamics x˙=Bη(x)−x have a unique forward solution for locally Lipschitz F. The same integrating factor used for best response shows forward invariance. For a fixed η, continuity on the compact simplex gives a positive lower bound for every response coordinate. Thus the solution is interior for all t>0, with each coordinate bounded uniformly away from zero after any positive time. Brouwer’s fixed point theorem ensures a rest point for continuous F. Every rest point is interior and satisfies x=Bη(x).
If ηk→0 through positive parameters and x(k)=Bηk(x(k)) converges, its limit is Nash. For a strategy strictly suboptimal at the limit, continuity gives a payoff gap of at least some δ>0 compared with a maximizing strategy there, for all large k. Its probability in x(k) is at most exp(−δ/ηk), hence is zero in the limit. Only maximizers can retain mass. This argument does not choose which Nash states arise along branches of fixed points.
The potential case gives a convergence result at a fixed noise level. Let V be a potential in the sense of Chapter [ch:lyapunov], continuously differentiable on the simplex in its affine hull. On the interior the logit dynamics satisfy dtd(V+ηH)(x)=ηi∑(Biη(x)−xi)(logBiη(x)−logxi)=η[D(Bη(x)∥x)+D(x∥Bη(x))]. The first equality uses Fi(x)=ηlogBiη(x)+ηlog∑jeFj(x)/η and the fact that the velocity sums to zero. Thus V+ηH strictly increases except at rest points. The interior bounds after positive time make the compactness argument of Chapter [ch:lyapunov] applicable. If V is concave, then V+ηH is strictly concave on the simplex. Each interior logit rest point satisfies its constrained first-order condition, so there is only one. All trajectories converge to it. The existence, limit, and convergence statements use different hypotheses on fitness and on the response rule. More general perturbation and revision theories are developed by Sandholm [10].
Interacting populations and spatial structure
The comparison neighborhood in Chapter [ch:revision] consisted of individuals drawn from the same population. An individual can instead interact frequently with nearby populations while drawing its behavioral replacement from its own population. An individual who moves changes the composition of both the source and destination. These mechanisms give different equations and lead to different forms of reduction. We study exact synchronized subspaces for a deterministic interaction model and then derive a quantitative synchronization criterion for migration.
Product simplices and assortative encounters
Let I be a finite set of populations and Sr a finite set of types for each r∈I. A state in DI=∏r∈IΔ(Sr) records conditional frequencies within the populations. The payoff to type i∈Sr is Fir(x), potentially a function of the full profile. Local normalization gives ϕr(x)=i∈Sr∑xirFir(x),x˙ir=xir(Fir(x)−ϕr(x)). Each population has its own conserved mass. Assuming a locally Lipschitz extension near the product, the proof of Chapter [ch:replicator] applies simultaneously. The equation for each population’s mass receives its own continuous coefficient ϕr(x), and each coordinate still has an integrating factor. Compactness of the product yields the global flow. A choice of one face per population gives an invariant product face. If every Fr depends solely on xr, the flow is the product of the separate population flows. If payoffs couple populations, a product face can remain invariant while the flow within it is coupled.
Take a common type set S, a matrix A of row-player payoffs on S, and a row stochastic matrix W=(Wrs) on I. An encounter by a resident of r draws its partner’s population from row r and then draws the type from that population. The vector of expected payoffs is FWr(x)=As∈I∑Wrsxs. Diagonal weights quantify within-population encounters. Directed weights allow asymmetric access to partners. The state equations apply the selection step within each destination of the payoff map, retaining separate local averages even when all populations see the same distribution of partners. At a stationary profile each resident type in each population has the same payoff. Comparison with absent types is required to promote this equality to a population Nash condition.
Here W is part of a deterministic model posed directly at the population level. For an individual-based graph process, replacing, for example, the conditional distribution of neighbors of type i by the unconditional population mixture is an additional modeling step. Ohtsuki and Nowak derive update-dependent replicator approximations for regular graphs using weak selection and pair approximation [13]. That derivation treats local correlations explicitly. The present equation follows directly from the specified partner sampling.
Exact reduction along equitable partitions
Let π:I↠J partition I into blocks, and write L:DJ⟶DI for replication of block values, (Lz)r=zπ(r). The partition is equitable for W if the total weight from r to block b depends only on the block a=π(r). Define Wˉab to be that total. This gives a row stochastic matrix Wˉ and, if L also denotes the incidence matrix Lra=1π(r)=a, the identity WL=LWˉ.
Proposition 1.1. Write RW and RWˉ for the replicator fields built with the same payoff matrix A. For an equitable partition, RW(Lz)=LRWˉ(z). In particular the image of L is invariant, and the restriction of the flow to that image is conjugate to the reduced flow.
Proof. For r in block a, the expanded profile has payoff FWr(Lz)=A∑bWˉabzb=FWˉa(z). Its resident frequencies are za, so its local mean payoff is the same as the block mean payoff. Substituting those two equalities makes all derivatives agree with the expanded block derivatives. A reduced solution lifts to a full solution, which is unique from its initial condition. ◻
For the flows Φ, this reduction is an intertwining relation. If WˉL′=L′W for a second equitable partition, then W(LL′)=(LL′)W. Thus the embeddings and their intertwining equations compose. The one-block partition is equitable for any stochastic W. Its synchronous dynamics is the ordinary replicator equation for A.
For a general profile, a weighted block average retains less information than is needed to compute its derivative. For probability weights μ on a specified block, put xˉi=∑rμrxir over that block and hir=Fir(x)−ϕr(x). Then xˉ˙i=xˉir∑μrhir+r∑μr(xir−xˉi)(hir−s∑μshis). The second term is a covariance. Its appearance precedes any further attempt to express the averages of relative payoffs through the coarse state. Equitability controls the aggregate encounter weights, while the embedding supplies the synchronized state on which that control implies an exact closed system.
A potential from detailed balance
Suppose W admits a positive probability vector μ satisfying μrWrs=μsWsr. If A=AT, define P(x)=21r,s∑μrWrs(xr)TAxs. Differentiation of the first factor gives 21∑rμr⟨ur,FWr(x)⟩. In the term from the second factor exchange r and s, then apply detailed balance and symmetry of A. This gives the other half of dPx(u)=∑rμr⟨ur,FWr(x)⟩.
On the relative interior of each product face, give the tangent space the weighted product of the Shahshahani metrics from Chapter [ch:lyapunov]. Its inner product is ∑rμr∑i:xir>0uirvir/xir. For v=RW(x) the local means cancel against ∑iuir=0. Therefore RW is the gradient of P in this metric. The rate of increase of P equals the weighted sum of the within-population payoff variances, and vanishes at stationary profiles. This argument uses reciprocity of the encounter-weighted payoff couplings. A directed row stochastic matrix by itself supplies the reduction theorem but does not supply the symmetry used in the derivative of P.
Migration and exponential synchronization
Let V=RF be a common local replicator field with a Lipschitz constant C on Δ(S) in Euclidean norm. Consider the additional coupling x˙r=V(xr)+κ(s∑Wrsxs−xr),κ≥0. This term diffuses relative frequencies between equal-normalization populations. It mixes a particular type across populations even when its local frequency is initially zero. If the replicator has the ambient form above, the total mass vector m satisfies m˙r=(1−mr)ϕr+κ((Wm)r−mr) along any solution. The constant unit vector solves this linear inhomogeneous equation, with the coefficients taken along that solution. Uniqueness preserves each mass. The incoming linear terms have nonnegative off-diagonal coefficients for every type. Variation of constants, or local Picard iteration of the resulting positive integral equation for each type, preserves nonnegativity. Thus the model has a global flow on DI.
Proposition 1.2. Let ∣I∣≥2 and suppose W is irreducible and reversible with a positive stationary distribution μ. Let λ>0 be the smallest positive eigenvalue of I−W in the μ-weighted inner product. If κλ>C, the squared distance to the synchronous subspace, in the product norm with weights μ, decays at least at rate 2(κλ−C).
Proof. For the mean xˉ=∑rμrxr set δr=xr−xˉ and E=21∑rμr∥δr∥2. Since the deviations have weighted mean zero, the term from xˉ˙ in E˙ vanishes. The local contribution is ∑rμr⟨δr,V(xr)−V(xˉ)⟩≤2CE. The migration contribution equals −κ times the Dirichlet form ⟨δ,(I−W)δ⟩μ=21r,s∑μrWrs∥δr−δs∥2. Reversibility makes I−W self-adjoint. Irreducibility forces every zero of the displayed form to be constant across populations. Applying the spectral gap on the mean-zero subspace bounds the form below by 2λE. The claim follows by integrating E˙≤−2(κλ−C)E. ◻
Migration cancels from the mean equation by stationarity of μ. The mean therefore satisfies ∥xˉ˙−V(xˉ)∥≤C2E. The estimate controls the deviation from the local law along the mean path, and the stability of any particular synchronous orbit is a further dynamical question. The partition theorem gives an exact invariant subspace. The migration estimate gives an attraction criterion for one such subspace. Both properties specify what a spatial reduction preserves and under which hypotheses it does so.
Finite populations and Markov dynamics
Alexander starts the finite-population analysis from individual replacement and then asks what changes when interaction is local [1, Sec. 4]. To make this distinction precise, separate the population state, the law of an update, and the statistic that describes the next state. A stochastic update assigns a probability to each possible successor. Its support determines what can happen, and its probability weights determine the relative likelihoods. Both are needed to describe extinction, fixation, and stationary behavior.
Count states and fitness
Let S be a nonempty finite strategy set. Fix a population size N≥2. A labeled configuration is a function z:{1,…,N}→S. Its count vector is n(z), with ni(z) the number of individuals using strategy i. The set of count states and the associated frequencies are CN(S)={n∈NS:i∈S∑ni=N},xi=Nni. Count states are orbits of the action of label permutations on configurations. The induced action on an update law can be used to test whether the orbit statistic itself evolves by a Markov chain. Under exchangeable sampling, the law of the updated counts depends only on the current counts. On a graph, two configurations in the same orbit can place the same strategy at vertices with different neighborhoods. The Markov property for counts then requires a separate argument.
Suppose Aij is the payoff to strategy i against j, and that an individual’s opponent is sampled uniformly from the other N−1 individuals. The expected payoff of a present type i is uiN(n)=j∈S∑N−1nj−1{i=j}Aij. If ni=0, there is no individual of type i whose payoff is needed. Give each currently present type a strictly positive reproductive weight fi(n). For instance a positive response fi(n)=ϕ(uiN(n)) implements a prescribed mapping from payoffs to reproductive probabilities. The choice is part of the model. Adding the same constant to all payoffs preserves many static comparisons. Adding the same constant to positive reproductive weights changes ratios fi/fj in general and hence can change the stochastic dynamics. A common positive multiplication of all weights at a state leaves their normalized ratios unchanged. The distinction parallels the several meanings of payoff equivalence in Chapter [ch:payoff-equivalence].
Birth, mutation, and replacement
The birth–death process updates one position at a time. It belongs to the overlapping-generation tradition of Moran’s model [14]. Here the parent is chosen with probability proportional to its reproductive weight, the offspring type is sampled from a mutation kernel, and a uniformly chosen member of the current population is replaced. The parent is eligible for replacement in this convention. Let Mik be the probability that a type i parent produces type k, with each row of M summing to one. Define bi(n)=∑jnjfj(n)nifi(n),ck(n)=i∑bi(n)Mik,dj(n)=Nnj. For an absent type we set nifi(n)=0 by convention. The offspring distribution is c(n). Let ei denote the count vector with a unit at i and zero elsewhere. If k=j and nj>0, the transition from n to n+ek−ej has probability ck(n)dj(n). The probability of remaining at n is ∑jcj(n)dj(n). These probabilities sum to one because all ordered pairs of offspring and removed types have been accounted for. A death rule depending on the parent or on its location replaces the product ckdj by a joint kernel. In that case the joint law is part of the dynamics.
The update can also be written as a composition of kernels, retaining only the information required by the next stage. With Dn={(k,j)∈S2:nj>0}, each arrow below is well defined on its displayed states: The final map is deterministic. This factorization makes conservation of total population size explicit. It also gives a route to composition with other mechanisms, such as scheduling several local updates, by composing their kernels in time. The conditional mean of the frequency increment is E[xnext−x∣n]=N1(c(n)−x). For this identity, simply average ek−ej against the joint law. Independence supplies the stated c and d, while the mean-increment formula needs only their marginals. The second moments can depend on the joint law even when the drift agrees.
In the independent case, writing diag(c) for the diagonal matrix with entries ck, the conditional covariance of the frequency increment is N21(diag(c)−ccT+diag(d)−ddT), with c,d taken as columns in this formula. Each removed or added type is represented by a random coordinate vector. Independence makes their cross covariance zero, so the two covariance matrices add. The matrix annihilates the all-ones vector, expressing that a frequency increment is tangent to the affine simplex. This covariance and the conditional drift answer different questions about the same update kernel.
Proposition 1.1 (Fixation and recurrence). For the update above with strictly positive reproductive weights, suppose first that M is the identity. The chain reaches a monomorphic population almost surely, and each monomorphic population is absorbing. If M instead has strictly positive entries, the chain is irreducible and aperiodic. It has a unique stationary law and converges to that law from every initial distribution.
Proof. When M is the identity, a currently absent type cannot be produced. In a monomorphic state all replacements preserve the type. Starting from any state containing i, choose a parent of type i and remove an individual of another type at each successive step until reaching the monomorphic state of type i. Each step has strictly positive conditional probability and the path has length at most N−1. The state set is finite, so there is a uniform positive lower bound on the probability of absorption within N−1 steps, considering one such path for each state. By the Markov property, the probability of avoiding absorption over successive blocks of N−1 steps decays geometrically.
If M is strictly positive, ck(n)>0 for every k and every n. A move removing a present type and adding any specified type is therefore possible. Starting from n, repeatedly move a unit from a coordinate in surplus relative to a target n′ to one in deficit. This reaches n′ in at most N steps. Every state also has a self-loop because at least one dj is positive and every cj is positive. Thus the N-step transition matrix is strictly positive. A stationary law exists as a limit point of Cesàro averages. The N-step matrix has a common strictly positive minorant in every row. Removing it leaves the same residual mass in every row and gives a total-variation contraction by that residual mass, which is strictly less than one. This proves uniqueness and convergence for the full chain. ◻
Fixation probabilities retain the update law
When mutation is absent, let hi(n) be the probability of fixation at the monomorphic state of type i. Conditioning on the first update makes hi harmonic away from the absorbing states. Its boundary values are one at fixation of i and zero at fixation of any other type. These conditions characterize it. Indeed, a harmonic function with these boundary values has constant conditional mean until absorption, and the absorption argument above gives almost sure convergence to the boundary values. Boundedness on the finite state space permits taking expectations in the limit.
Under neutral reproduction all present reproductive weights are equal, so c(n)=x in the mutation-free process. Equation (8) gives harmonicity of ni/N, and the boundary values give hi(n)=ni/N. For nonneutral reproduction these fixation probabilities depend on the transition law. The drift alone can be insufficient when different updates induce different hitting distributions, and a deterministic trajectory is a different observable from the absorption probability. The next chapter studies selection by rare mutation, in which fixation between departures from recurrent states supplies the effective transition information. Chapter [ch:mean-field] compares the frequency process with its deterministic limit on a fixed time interval.
Rare mutation and stochastic stability
A small mutation probability changes the meaning of a long-run prediction. The mutation-free chain can have many recurrent classes. With positive mutation the chain can leave those classes, and its stationary law can weight them differently even when mutation tends to zero. The stationary limit first takes a long-time average at positive mutation and then decreases the mutation parameter. This order is intrinsic to the prediction.
A theorem for stationary weights
Let E be a nonempty finite state set and P an irreducible transition matrix on E, with rows indexed by the starting state. An in-tree rooted at i assigns one outgoing edge to each vertex other than i, such that repeated traversal of the edges brings any vertex to i. The root has no outgoing edge. Define Wi(P)=τ rooted at i∑(x,y)∈τ∏P(x,y). The one-state case uses the weight one for the empty tree. Each weight is positive because the support of P has a path from every vertex to i. Choosing at each vertex an edge decreasing the shortest such path length gives a tree of positive weight.
Theorem 1.1 (Markov chain tree formula). The unique stationary distribution of P is π(i)=Wi(P)/∑jWj(P).
Proof. It suffices to show that the weighted inflow to i from distinct states equals the weighted outflow from i: j=i∑Wj(P)P(j,i)=Wi(P)k=i∑P(i,k). A term on the left adds the edge j→i to a tree rooted at j. This produces a directed graph with one outgoing edge at every vertex and a unique directed cycle, of length at least two, containing i. A term on the right adds i→k to a tree rooted at i. It gives a graph of the same kind. From such a graph, deleting the edge entering i on the cycle recovers the construction on the left. Deleting the outgoing edge at i recovers the construction on the right. The products are the same because they use all edges of the corresponding graph. The diagonal terms of stationarity cancel, so this bijection proves the displayed identity. Normalize and use uniqueness of the stationary law for a finite irreducible chain. ◻
This is the rooted-tree form of the Markov chain tree theorem [16]. It accounts for indirect paths between states. The stationary ranking can depend on several paths taken together and therefore need not coincide with a ranking of the individual edges leaving each state.
Leading powers and stable states
Suppose Pϵ is irreducible for every sufficiently small ϵ>0. Assume for each off-diagonal edge that its weight is either identically zero for small ϵ, or Pϵ(x,y)=cxyϵrxy+o(ϵrxy),cxy>0,rxy≥0. Call rxy the resistance. Omit identically zero edges and set γi=τ rooted at imin(x,y)∈τ∑rxy. For each root define ai>0 by summing ∏(x,y)∈τcxy over its minimizers. Each tree product has exponent given by the sum of its edge resistances. The tree sum is finite and its coefficients at the smallest exponent are positive. Hence Wi(Pϵ)=ϵγi(ai+o(1)). Writing γ∗=miniγi, the stationary mass tends to zero for γi>γ∗ and tends to ai/∑γj=γ∗aj for γi=γ∗. A state with positive limiting mass is called stochastically stable. Thus the minimum resistance determines the stable set, and the coefficients determine the limiting weights within it. The leading powers depend on the revision and mutation mechanism. An ordering of payoffs alone does not provide them.
This argument is a direct finite-state counterpart to resistance and stochastic potential in Young’s theory of regular perturbed Markov processes [15]. Mutation mechanisms with weights decaying faster than all powers may require a different asymptotic scale. Aperiodicity is needed for general convergence from arbitrary initial laws along discrete time. The stationary weights in the tree theorem are defined for every irreducible kernel.
Eliminating transient excursions
The state set of a population chain can be large. An exact reduction separates states recurrent under no mutation from transient excursion states. Suppose Pϵ→P0 and write E=K⨿T, where K is the set of recurrent states of P0 and T is the set of transient states. Define successive visits to K by counting each ordinary time step spent in K, including consecutive time steps there. With matrix blocks indexed by the starting and ending sets, this trace kernel is Pϵ=PKKϵ+PKTϵ(I−PTTϵ)−1PTKϵ. If T is empty, only the first term is needed. The geometric series for the inverse sums incomplete excursions according to the number of steps that stay in T. For each m≥0 the term with m such steps follows the block sequence Transience makes the series converge at ϵ=0, since the probability of staying in T decays geometrically in a finite chain. The inverse therefore exists and is continuous for sufficiently small ϵ.
The stationary equation on T is πTϵ=πKϵPKTϵ+πTϵPTTϵ. Solving it and substituting into the equation on K gives πKϵPϵ=πKϵ. Thus the trace stationary distribution is the normalized restriction of πϵ. The trace is irreducible for positive ϵ, since an original positive-probability path between visited states decomposes into consecutive trace steps. Moreover, PKT0=0, because each recurrent class is closed at zero mutation. The same stationary identity then gives πTϵ→0. This permits computing stochastic stability using only the trace, when its transitions satisfy the requisite asymptotics. A trace records return events in the original chain. Its stationary probabilities describe fractions of visits, and the normalization accounts for the time omitted in excursions.
The error in dropping transient mass can be estimated before computing the trace stationary law. For a nonnegative matrix B, let ∥B∥∞ be its maximum row sum. The stationary equation gives πϵ(T)≤∥PKTϵ∥∞(I−PTTϵ)−1∞. The second factor bounds the expected duration of an excursion from any state in T, and remains bounded as ϵ→0 at fixed population size. For example, when the probability of leaving K in one update is uniformly O(ϵ), the total stationary time outside K is O(ϵ).
Rare steps and fixation
If the recurrent classes of P0 are singleton absorbing states and Pϵ=P0+ϵR+o(ϵ) entrywise, formula (9) gives Pϵ=I+ϵ(RKK+RKT(I−PTT0)−1PTK0)+o(ϵ). The off-diagonal first-order terms are direct changes between absorbing states and changes that enter T once at order ϵ, then reach an absorbing state according to P0. The latter factor contains the unperturbed fixation probabilities. If several rare steps are needed, first order alone cannot recover all resistances. More generally the trace need not lump several recurrent states into a single class at positive mutation. One can either keep all states of K, check an exact lumpability criterion as in Chapter [ch:aggregation], or invoke a separate asymptotic class reduction with its hypotheses.
Call the first-order coefficient G. Nonnegativity of off-diagonal trace probabilities and normalization of their rows imply that G is a continuous-time generator on K. If it is irreducible, the normalized trace stationary laws converge to its unique stationary law. Indeed, divide their stationary equations by ϵ after subtracting the identity term. Every accumulation point annihilates G. Irreducibility makes that accumulation point unique, and compactness gives convergence. This argument identifies the scope of the first-order approximation.
These conclusions address a fixed population size. Chapter [ch:mean-field] treats a different limiting operation, in which population size increases over a bounded time interval. Neither proof interchanges large population, long time, and vanishing mutation. Such an interchange requires additional bounds, because absorption and mixing time can themselves depend strongly on population size and on mutation.
Mean-field limits and interaction structure
A deterministic evolution can describe the leading motion of a stochastic population over a bounded time interval. The connection is a convergence statement about processes, with a specified scale for the frequency increments and the update rate. It requires bounds on the microscopic drift and on fluctuations. Kurtz gave a general framework for the approximation of differential equations by jump processes [17]. Benaïm and Weibull studied the approximation and time scales for evolutionary games [18]. The result below isolates assumptions that can be checked by evaluating the local update law.
A generator on the frequency simplex
Let D⊆Rd be a compact convex set and let DN⊆D be a finite state set. The simplex lattice CN(S)/N is the main application. Fix a finite set V of possible jump directions. Write the generator as LNφ(x)=Nv∈V∑βvN(x)[φ(x+v/N)−φ(x)]. Each intensity βvN is nonnegative and is zero whenever the indicated transition is infeasible. Terms of zero intensity are omitted. Suppose B=N,x∈DNsupv∑βvN(x)∥v∥2<∞,FN(x)=v∑βvN(x)v. The vector FN(x) is the instantaneous conditional drift. The rate of quadratic variation from jumps is of order N−1 since each jump squares the factor N−1 and the generator contributes N. If FN converges uniformly on its state sets to a field F, Taylor’s formula gives LNφ→Dφ[F] for a smooth φ with bounded second derivative on a neighborhood of D. The Taylor remainder is at most Bsup∥D2φ∥/(2N). The convergence of generators on such functions identifies the candidate equation.
Theorem 1.1 (Bounded-time approximation). Assume F is Lipschitz on D with constant L, and set δN=supx∈DN∥FN(x)−F(x)∥. Suppose δN→0 and XN(0)→x0 in probability. If the solution of x˙=F(x) from x0 remains in D on [0,T], then 0≤t≤Tsup∥XN(t)−x(t)∥⟶0in probability.
Proof. The coordinate martingales of the generator give the vector decomposition XN(t)=XN(0)+∫0tFN(XN(s))ds+MN(t). For the squared Euclidean norm of the martingale, the compensator of the sum of squared jumps contributes at most BT/N in expectation up to time T. In detail, the jump v/N contributes ∥v∥2/N2 at rate NβvN(XN(s)). Their integrated sum bounds E∥MN(T)∥2. Thus Doob’s quadratic inequality, applied to the submartingale ∥MN(t)∥, gives E0≤t≤Tsup∥MN(t)∥2≤4BT/N. Subtracting the deterministic integral equation and using the Lipschitz bound yields, by Grönwall’s inequality, 0≤t≤Tsup∥XN(t)−x(t)∥≤eLT(∥XN(0)−x0∥+TδN+0≤t≤Tsup∥MN(t)∥). Every term in parentheses tends to zero in probability. ◻
The estimate is quantitative once drift and initial-state errors are bounded. The factor eLT also locates the difficulty of extending the comparison to long time. The theorem controls complete paths on each fixed interval. It does not compare stationary distributions, which involve behavior on a time scale that can increase with N.
For deterministic initial states with error aN=∥XN(0)−x0∥, choose any η>eLT(aN+TδN). The proof gives P(t≤Tsup∥XN(t)−x(t)∥>η)≤N(e−LTη−aN−TδN)24BT. The expression separates the initial discrepancy and the systematic drift error from the probability of a fluctuation. Different microscopic kernels can share the same limit F while differing in B or in the conditional covariance from Chapter [ch:markov].
For an update chain with transition matrix PN, an independent Poisson clock of rate N gives the generator N(PN−I). The chain is evaluated at the number of clock arrivals by time t, so a unit of macroscopic time corresponds on average to N updates. A self-loop consumes a clock arrival and produces a zero displacement. This convention makes the drift of the continuous-time chain N times the mean increment in one discrete update. It also preserves stationary laws because πN(PN−I)=0 is equivalent to πPN=π.
Recovering deterministic selection
Use the birth–death update of Chapter [ch:markov] at rate N. Its mean field is c(n)−x by Equation (8). Suppose the weights of present types converge uniformly to bounded strictly positive functions fi(x), with a common positive lower bound. The limiting offspring fractions and field are ck(x)=i∑f(x)xifi(x)Mik,Fk(x)=ck(x)−xk,f(x)=i∑xifi(x). If the weights are Lipschitz, the field is Lipschitz. Its components sum to zero, and ck(x)≥0 at a face with xk=0. A direct form of the integral equation is xk(t)=e−txk(0)+∫0te−(t−s)ck(x(s))ds. This exhibits the role of the offspring distribution. Invariance of the simplex also follows by constructing the solution locally as the fixed point of this integral map on paths in the simplex, where it is a contraction on short intervals and maps such paths to such paths. Short intervals can be concatenated because the simplex is compact and the constants can be chosen uniformly. With no mutation, Fk(x)=xk(fk(x)−f(x))/f(x). Because f(x)>0, this has the replicator trajectories after a positive change of time. The rates with respect to calendar time differ by this common factor. With mutation, normalization and transmission are combined in c and need not give a particular additive mutation convention on the replicator equation.
The distinction between finite and limiting payoffs can be bounded explicitly. For ni>0 the payoff in Chapter [ch:markov] satisfies uiN(n)−(Ax)i=N−1(Ax)i−Aii. If fi(n)=ϕ(uiN(n)) with a Lipschitz positive map ϕ on the interval covered by the payoffs, the difference from the weight ϕ((Ax)i) is O(N−1) uniformly over present types. The absent types contribute zero to the birth distribution. Positivity on the compact payoff interval gives a uniform lower bound for the normalization, so the corresponding drift approximation has order N−1. The martingale estimate controls the stochastic error separately at order N−1/2 in root mean square over fixed times.
When counts close over a graph
Suppose the population is an N-vertex weighted graph. For an oriented pair of vertices u,v, let λuvN≥0 be a sampling weight with ∑u,vλuvN=1. A type i at u replaces a type j at v at rate NλuvNφ(i,j), for fixed nonnegative bounded biases φ. The relevant statistic is the oriented edge frequency θijN(z)=u,v∑λuvN1{z(u)=i,z(v)=j}. The density drift on a labeled state z is ∑i,jθijN(z)φ(i,j)(ei−ej), where ei is the coordinate unit at i. For a general graph the edge frequencies can vary when the vertex frequencies are fixed. A closed equation for vertex frequencies requires additional information about their evolution.
An explicit sufficient bound exists for regular weighted sampling. Suppose λuvN=Wuv/N, with W a nonnegative matrix with row sums one, and write JN for the matrix with all entries 1/N. If 1i is the vertex indicator of type i, then θijN(z)−xixj=N11iT(W−JN)1j≤∥W−JN∥2→2xixj. Thus ∥W−JN∥2→2→0 is sufficient for uniform closure of the drift. The proof of Theorem 1.1 still applies to the projected coordinate processes, using the generator on labeled configurations to obtain their compensators. Uniform drift approximation substitutes for the exact Markov property of the projection. With uniform sampling of distinct ordered vertices, W=(NJN−I)/(N−1). Then W−JN=(JN−I)/(N−1) and its norm is 1/(N−1) for N≥2. For any graph family satisfying the norm bound, the limiting field is F(x)=i,j∑xixjφ(i,j)(ei−ej). It is Lipschitz on the simplex. Its coordinates sum to zero and each Fk(x) has the factor xk, so the simplex is invariant. A density path on labeled configurations therefore converges to this solution under the initial-state hypothesis of Theorem 1.1. The omitted edge data may persist on sparse or strongly clustered graphs. One can retain additional observables or prove a separate estimate of their effect. Chapter [ch:aggregation] gives the criterion for exact Markov closure at each population size.
Stochastic categories and population aggregation
Kernels as morphisms
A state description is useful only when its dynamics can be computed at the level of that description. For the exchangeable population rule in Chapter [ch:markov], counts describe a Markov chain exactly. In the network rule of Chapter [ch:mean-field], the count drift generally depends on edge frequencies. The category of stochastic kernels gives a common language for these two situations.
Let FinStoch have finite sets as objects and stochastic kernels as morphisms. A kernel K:X⇝Y assigns to each x∈X a probability K(x,y) on Y. The composite of K and H:Y⇝Z has entries (KH)(x,z)=y∈Y∑K(x,y)H(y,z). Matrix products follow the order in which updates are applied, since laws are row vectors. Identity kernels have entries 1{x=y}. Matrix associativity is the rule for summing over consecutive unobserved states.
The monoidal product X×Y models joint state spaces, with product kernels given by (K⊗H)((x,x′),(y,y′))=K(x,y)H(x′,y′). This product applies independent random draws to the two coordinates given their current states. The singleton is monoidal unit and terminal object. Every state space has a deterministic copy operation ΔX(x)=(x,x). A kernel K preserves copy when KΔY=ΔX(K⊗K). On the left, a sample from K(x,−) is copied. On the right, independent samples are drawn from the same row. Comparing off-diagonal probabilities shows that for distinct y,z one must have K(x,y)K(x,z)=0. Thus each row is a point mass, and K comes from a function. Every function preserves copy. This is the deterministic subcategory of FinStoch, an instance of the distinction between general and deterministic morphisms in a Markov category [20].
Maps of dynamics
A discrete-time system is a pair (X,P) with P:X⇝X. A stochastic map of dynamics from (X,P) to (Y,Q) is a kernel K:X⇝Y with PK=KQ. The equation says that updating the source and then observing it gives the same law as observing it and applying the target update. Iteration gives PnK=KQn for every n. If QH=HR as well, then P(KH)=(KH)R, so these maps compose. Equivalently, a system is a functor BN→FinStoch, where BN has one object and the monoid of nonnegative time steps as its endomorphisms. A stochastic map of dynamics is a natural transformation. The categorical equation for all times is generated by its component at time one.
If μP=μ, then (μK)Q=μPK=μK, so the map transports stationary laws. Independent dynamics can also be composed in parallel. For two maps satisfying PiKi=KiQi, the kernel K1⊗K2 intertwines P1⊗P2 and Q1⊗Q2. Here the tensor products of the transitions represent synchronous updates with conditional independence.
Strong lumpability
Let q:X→Y be a surjection recording a deterministic statistic, and write J(x,y)=1{q(x)=y}. Its values identify blocks By=q−1(y). Strong lumpability requires that the observed process q(Xn), for a P-chain Xn, have a fixed Markov update on Y for every initial law on X. This notion has a block characterization [19, §6.3].
Theorem 1.1 (Strong lumpability). For a kernel P on X, the following are equivalent.
There is a single kernel Q governing the process q(Xn) for every initial law.
There is a kernel Q on Y satisfying PJ=JQ.
For every y,y′ the sum ∑x′∈By′P(x,x′) is constant for x∈By.
When these conditions hold, Q is unique and Q(y,y′) is the common value in the third condition.
Proof. For x∈By, the (x,y′)-entries of PJ and JQ are the block sum and Q(y,y′), respectively. This proves equivalence of the last two conditions because the sums give nonnegative rows of total mass one. Given the original process history, the conditional probability that q(Xn+1)=y′ is the block sum for Xn. Under the third condition it is Q(q(Xn),y′). Conditioning on the observed history leaves that value unchanged, proving the first condition. Conversely, initialize at a point x and use the proposed update Q at time one. Its entry from q(x) to y′ must equal the block sum from x, hence the block sum agrees for all points in the block. ◻
A Markov description obtained for one specified initial law can depend on the microscopic conditional distributions within the blocks. This weaker statement does not imply an intertwining for deterministic J. For strong lumpability the intertwining suffices to give both the distributional identity μPnJ=(μJ)Qn and the assertion about the full observed process in the theorem.
The quotient also has a factorization property for stochastic maps of dynamics. Suppose H:(X,P)→(Z,R) intertwines the updates and its rows are constant on each q-fiber. Then the unique kernel H defined by those common rows satisfies H=JH. From PH=HR and PJ=JQ one gets JQH=JHR. Surjectivity of q lets one cancel J since left multiplication by J simply repeats rows. Thus QH=HR. Every observation of X determined by the quotient statistic, and compatible with its own time update, factors through the quotient as a map of dynamics.
Proposition 1.2 (Equivariance gives a quotient). If a group G acts on X and P(gx,gx′)=P(x,x′) for every g, the orbit map X→X/G is strongly lumpable.
Proof. Every target orbit B is preserved bijectively by g, so x′∈B∑P(gx,x′)=u∈B∑P(gx,gu)=u∈B∑P(x,u). The sums therefore depend only on the source orbit. ◻
For site-independent replacement and permutation-invariant payoffs, site permutations act equivariantly on the labeled chain. Their orbits are precisely the configurations with a given strategy-count vector, recovering the population quotient in Chapter [ch:markov]. In spatial dynamics the relevant permutation group is typically the automorphism group of the weighted interaction graph. Its orbits may record information beyond counts, such as whether a pair of strategy classes frequently meet across graph edges.
Generators and approximate quotients
A continuous-time finite-state chain has generator LX with nonnegative off-diagonal entries and zero row sums. A deterministic quotient intertwines the semigroups at every time if and only if LXJ=JLY. Differentiation at zero gives necessity. For sufficiency the generator equation gives LXkJ=JLYk by induction, and the result follows from the exponential series. Thus a quotient generator exists when each generator block sum is constant on a source block. For off-diagonal blocks the contributing entries are nonnegative. Row sums of the quotient generator are zero by those of LX. This condition can be checked on jump intensities directly, including when the total holding rate varies among states of a block.
When block sums almost agree, an error estimate controls the effect on a fixed observation horizon. For a proposed kernel Q on Y, assume x∈XsupP(x,−)J−Q(q(x),−)TV≤ε, where total variation is one half the ℓ1 norm. Writing E=PJ−JQ, an exact telescoping identity gives μPnJ−μJQn=j=0∑n−1μPn−1−jEQj. The row vector before E is a probability, so the norm of its product with E is at most ε by convexity. Propagation under Qj is a contraction in total variation, giving a bound of nε for every starting law μ. This estimate concerns the one-time observed laws and is uniform on a specified range of time steps. A stationary error estimate requires more information, such as a quantitative contraction for the quotient dynamics. In the limit theorem of Chapter [ch:mean-field], the target is a deterministic flow and the uniform drift error is what enters the finite-time comparison.
Open games and the continuation interface
Classical evolutionary game theory starts from payoffs on an entire population interaction. A component description needs an interface for an interaction whose surroundings are still unspecified. Open games supply that interface in the form of an input and a continuation. The continuation tells the component how each possible output would be evaluated by its surroundings. This makes off-equilibrium choices available for composition [21].
Lenses as the deterministic interface
Use (X,S) for an interface with forward type X and backward type S. A lens from (X,S) to (Y,R) consists of two maps v:X→Y,u:X×R→S. Its forward action sends a state x to v(x). Its backward action translates a return value r using the original state x. In game semantics a return value may contain utility information. The lens axioms here are the composition and identity laws for these pairs.
For a second lens (v′,u′):(Y,R)→(Z,Q), the composite has forward map v′v and backward map (x,q)⟼u(x,u′(v(x),q)). For three lenses, both parenthesizations substitute the forward values in the same order and then the backward values in reverse order. The lens (idX,(x,s)↦s) is a two-sided identity. This gives an ordinary category. The product of two lenses uses cartesian products on both types and applies the two maps separately.
A continuation on the target is an arbitrary function k:Y→R. An input at the source is an element x∈X. A lens sends an input forward by v and transports a continuation backward by k⟼(x⟼u(x,k(v(x)))). The backward transport reverses composition by direct substitution. Keeping k as a function matters. A single value k(v(x)) gives a return only at the chosen output. A component deciding whether to deviate may need the value at a different output.
Strategies and contextual responses
An open game G:(X,S)→(Y,R) consists of a strategy set ΣG, a play map PG, a coplay map CG, and best-response relations: PGCGBG(x,k):ΣG×X→Y,:ΣG×X×R→S,⊆ΣG×ΣG. Here k:Y→R. The strategies are chosen for the whole component. Fixing a strategy σ gives a lens with maps PG(σ,−) and CG(σ,−,−). A pair (σ,σ′)∈BG(x,k) means that σ′ is an admissible response to the incumbent σ in this context. The equilibrium set consists of the strategies σ with (σ,σ)∈BG(x,k). An open game in this generality specifies responses as data. Its response relation can be constructed from an optimization problem, a restricted revision protocol, or another stated selection rule [21].
For a decision component with observed type X and choice type Y, take Σ=YX and P(σ,x)=σ(x). With real return type and backward source type 1, the standard maximizing relation allows (σ,σ′) if k(σ′(x))≥k(y)for every y∈Y. The map C is the unique map to 1. Existence of a response requires attainment of a maximum. At a given observed x this relation constrains the strategy only at x. A stronger requirement at all possible observations needs another equilibrium test, as in subgame analysis. The distinction is visible in the typing of B(x,k).
Sequential composition
Let H:(Y,R)→(Z,Q) follow G, with outer continuation k:Z→Q. An incumbent (σ,τ) determines the intermediate forward value y=PG(σ,x). The downstream strategy determines the continuation seen upstream: kτ(y′)=CH(τ,y′,k(PH(τ,y′))),y′∈Y. The upper path carries states forward. The lower arrow carries the continuation back across the downstream component. Here QZ and RY denote sets of continuations. A composite strategy is a pair. Its play and coplay are the composition of the fixed-strategy lenses, and its response relation is ((σ,τ),(σ′,τ′))∈BH∘G(x,k)⟺(σ,σ′)∈BG(x,kτ)and(τ,τ′)∈BH(y,k). The two tests use the incumbent context. The second component sees the output generated by σ. The first component evaluates alternatives using τ as a fixed strategy, including its play and coplay at changed inputs. This is a prescription for testing responses with the other components’ strategies held fixed.
Proposition 1.1 (Associativity of the response construction). Sequential composition is associative under the canonical reassociation of strategy products. Singleton-strategy identity lenses with the sole identity response give identities.
Proof. Write J:(Z,Q)→(W,T) for the third component and υ for its incumbent. For an outer continuation ℓ:W→T, put kυ(z′)=CJ(υ,z′,ℓ(PJ(υ,z′))). With y=PG(σ,x) and z=PH(τ,y), either parenthesization tests the three responses in the contexts G:H:J:(x,y′↦CH(τ,y′,kυ(PH(τ,y′)))),(y,kυ),(z,ℓ). For the first parenthesization these contexts result from first transporting through J and then through H. For the second, the coplay of the downstream lens composite expands to the same substitution. The play and coplay agree by lens associativity. Inserting an identity leaves the transported state and continuation unchanged and adds the sole relation on a singleton. ◻
Taking strategy sets up to reindexing by these coherent bijections gives an ordinary category. Keeping actual strategy sets makes the reassociation visible as a comparison between games. The next chapter also keeps reindexings that need not be invertible.
A compositional equilibrium test
The diagonal of the response formula gives the equilibrium criterion σ∈EG(x,kτ),τ∈EH(PG(σ,x),k), where EG denotes the contextual equilibrium set. This is a system of mutually dependent tests. Retaining only EG is enough for this particular diagonal calculation when all contexts and the fixed-strategy lenses are retained. Off-diagonal responses are needed to study transformations that preserve response possibilities or to introduce an update rule using those possibilities. For evolutionary models the selected continuation can include frequency-dependent payoffs. One must still specify how the current population determines that continuation, which strategy information is available, and how admissible responses change frequencies. The interface supplies a way to keep these choices explicit.
For instance, an adapter can expose the population state as an input and compute a payoff vector as its output. A downstream selection component then evaluates each strategy against that vector. Compositional optimization tests whether a strategy passes the resulting response relation. A dynamical model adds a state transition or vector field on populations, whose state change is an additional map. These two observations about the system can then be checked for agreement under the hypotheses of Chapter [ch:lyapunov].
Deterministic adapters
The response formula gives a useful rule for changing the language in which an interface is described. Suppose L:(X′,S′)→(X,S) and M:(Y,R)→(Y′,R′) are lenses. Regard each as a singleton-strategy game with its sole response. For a context (x′,k′) of M∘G∘L, the response test of the middle game has context (vL(x′),y⟼uM(y,k′(vM(y)))). This follows by first applying the lens substitution to the return from M, then carrying the input forward through L. The singleton comparisons impose no further response condition, and the same formula therefore computes equilibria of the adapted game. If a downstream adapter translates every return by the same amount for the relevant decision context, a real maximizing response test is unchanged. More generally a strictly increasing transformation of every return in that test preserves the ordering of alternatives. The key hypothesis here is a common transformation of the returns being compared. A translation depending on the output may alter their order and must be included in the continuation itself.
Parallel games and higher comparisons
Sequence describes a change of context through an intermediate interface. Parallel composition retains two interfaces and asks how the environment evaluates their joint output. A joint continuation can couple the decisions even when their play maps factor.
Parallel composition
For G:(X,S)→(Y,R) and H:(U,T)→(V,Q), their product has object types (X×U,S×T) and (Y×V,R×Q), and strategy set ΣG×ΣH. Its play and coplay are the products of the fixed-strategy lenses. Given a context with forward state (x,u) and continuation k:Y×V→R×Q, an incumbent (σ,τ) produces y∗=PG(σ,x) and v∗=PH(τ,u). Define the two continuations seen by the components as k1(y)=πRk(y,v∗),k2(v)=πQk(y∗,v). The parallel best-response relation requires (σ,σ′)∈BG(x,k1) and (τ,τ′)∈BH(u,k2). A trial change in one component thus sees the same incumbent output from the other. In particular, even a product of games whose input states are independent can have coupled equilibrium conditions when the continuation is joint. If k(y,v)=(r(y),q(v)), the continuations are r and q independently of σ,τ, so the equilibrium sets multiply.
Proposition 1.1 (Interchange). Parallel composition commutes with sequential composition up to the canonical exchange of strategy factors. Its associativity, unit and symmetry are given by the corresponding product bijections.
Proof. Consider two chains, G followed by H and G followed by H, with incumbent strategies σ,τ and σ,τ, respectively. Let y∗,y∗ be the intermediate outputs, z∗,z∗ the final outputs, and k the joint outer continuation. In either order of assembly the test at H uses y∗ and z↦π1k(z,z∗). The test at G uses the continuation y⟼CH(τ,y,π1k(PH(τ,y),z∗)). Indeed, composing within the first chain propagates the first outer continuation through H. Composing across the parallel pair first propagates k through the product lens, and then freezes the second intermediate value at y∗. Its downstream output is precisely z∗. The formulas for the other chain follow by using π2 and holding z∗ fixed. Thus all four response tests agree. The play and coplay maps agree because composition of product lenses is componentwise.
For any number of parallel factors the continuation for one factor fixes all other incumbent outputs and projects to the return for that factor. This gives the same formula for either parenthesization and after permutation of factors. The singleton unit introduces neither another output choice nor another return choice. Coherence follows from the corresponding equalities of tuples. ◻
Strategy maps and two-dimensional structure
One can preserve more information by keeping morphisms between games. For games G,G′ on the same interface, take a map ϕ:ΣG→ΣG′ that preserves play and coplay at every state and return. Require also that (σ,η)∈BG(x,k)⟹(ϕ(σ),ϕ(η))∈BG′(x,k). This gives a category for each pair of interfaces. Composition of strategy maps preserves the relation by transitivity of implication, and identity maps preserve all the specified data. Products of these maps preserve the sequential and parallel response formulas. For the sequential case, play preservation gives the same intermediate incumbent state and coplay preservation gives the same transported continuation. One can then apply response preservation separately in the two games. The parallel argument uses preservation of the two incumbent outputs. This yields a bicategorical description after retaining the canonical product reassociations as invertible cells. For a more general morphism allowing lenses between the boundary interfaces, see the double category of open games in [22].
For sequential composition, the diagram below holds at every upstream state x. Set Ex(σ,τ)=PH(τ,PG(σ,x)), where Z is the final forward output space. Primes denote the corresponding maps for the target games. The backward returns commute by coplay preservation. These two identities compare the boundary behavior independently of whether a pair of strategies satisfies the response tests.
Response-preserving maps carry equilibria to equilibria: one applies the implication to (σ,σ). A noninvertible map can identify equilibria or fail to reach some equilibria of the target. An isomorphism of games instead preserves and reflects the response tests in every context. The geometric information carried by a morphism is therefore different from a claim about the size or structure of its image on equilibrium sets. This distinction matters when strategy representations are quotiented or multiple agents share an interface.
Probability and continuation transport
For a fixed strategy, one may model forward play by a probability kernel. For finitely supported probabilities P(y∣x) and Q(z∣y), their forward composite is (QP)(z∣x)=y∑P(y∣x)Q(z∣y). This has unit given by a point mass at the input. Associativity is the reassociation of the joint sum over intermediate values. If a continuation takes values in R, this kernel transports it by expectation. Indeed y∑P(y∣x)z∑Q(z∣y)k(z)=z∑(QP)(z∣x)k(z). Thus a component that compares expected returns can use the transported function of its possible inputs. Existence of a maximizing strategy is again an additional hypothesis.
Stochastic parallel composition requires a decision about joint laws. Tensoring two kernels using independent randomness gives the product law conditional on their inputs. That tensor does not have the universal property of a cartesian product in stochastic maps. A joint distribution is generally not determined by its two marginals. For example, a nondegenerate finite law paired with itself by sharing its sample and the product of two independent copies of the law have the same marginals. The continuation may detect their dependence. Consequently continual substitution at the level of marginal expected payoffs alone can lose information relevant for later decisions. Bayesian open games supply additional structure for stochastic play, hidden information and best responses [23]. The calculation above describes only the forward-kernel and expected-return part of that framework.
An equilibrium interface and a revision rule
Response relations are evaluated at a context rather than at a time. To extract a trajectory one supplies a protocol for when revisions occur, which states agents observe, and how allowed revisions become a state update. For a fixed context, let B(σ) denote the set of responses to σ. If it is nonempty at every incumbent, a deterministic revision rule can choose a single U(σ)∈B(σ) at each incumbent. A fixed point of U is an equilibrium because its selected response is itself. The converse holds if the rule retains the incumbent whenever σ∈B(σ). The relation alone leaves this decision open, as well as the choice among other responses and the possible regularity of the selector. A protocol can instead attach a probability law to responses, which adds another layer of dynamical data.
In continuous time a smooth population vector field F and its positive rescaling aF by a smooth function a have the same rest points. On any interval over which a solution of one field exists and the scalar remains positive, its trajectory can be reparametrized in time to a solution of the other. This follows from the chain rule by solving for the derivative of the new time parameter to equal a along the trajectory. Quantities measured at specified times can nevertheless change. A contextual equilibrium criterion does not specify the clock, the distribution of samples, or the size of a discrete update. These data enter the dynamical interface in the next chapter.
Open dynamics and network wiring
An open dynamical system specifies how its current state changes under an input from its surroundings. Its output can supply another system with an input. A wiring diagram determines which values are passed and, when species or states are identified, where their rates of change are combined. Different choices of interface give different composition rules.
Input and output ports
Consider a smooth state space M, an input space A, and an output space B. Write a module as v:M×A⟶TM,h:M→B,v(x,a)∈TxM. The state equation is x˙=v(x,a) and its output is h(x). A single time-independent wiring of modules indexed by i can be expressed by maps w:Aext×i∏Bi→i∏Ai,r:i∏Bi→Bext. At state x=(xi)i, put b=(hi(xi))i and a=w(aext,b). The new state velocity has components vi(xi,ai) and the output is r(b). A feedback wire can carry hi(xi) to the same or another module. As the output depends only on the current state, closing such a wire requires no simultaneous algebraic solution for the inputs.
For nested networks the rule is associative: to compute one component vi, evaluate the elementary outputs, substitute them into the wiring maps of each enclosing network, and apply the resulting input to vi. Flattening the network substitutes the same functions into one composite. Associativity of function composition and the canonical comparison of product state spaces give equality of the component velocities. The same calculation applies to outputs. This is the readout form of the wiring algebra developed in [24]. If a readout is also allowed to depend on its instantaneous input, a directed loop may require a separate condition guaranteeing that its algebraic equations have a unique solution.
Replicator modules
For a population on a strategy simplex, expose a state readout and accept an input a that parametrizes a payoff field. In coordinates the local velocity is vi(p,a)=pi(fi(p,a)−j∑pjfj(p,a)). The velocity sums to zero, and vi=0 on the face pi=0, for every a. Consequently feedback through the payoff input preserves these algebraic identities. With smooth, locally Lipschitz coefficients on a neighborhood of the simplex, the wired vector field has unique local solutions. The face identities and uniqueness prevent a coordinate from crossing zero in finite time, and the mass identity preserves the normalization. The same argument works for separately normalized populations on a product of simplices, since the identity applies to each local collection of coordinates. This is one way to give a dynamical meaning to interactions between modules. A relation that recognizes mutually admissible responses is a separate structure, as in Chapter [ch:open-games].
For a varying external input the solution concept requires regularity of the input in time and local integrable bounds for the coefficients. Composition of wiring gives a formula for the field under those assumptions. It gives no single guarantee of convergence to a selected equilibrium. Even on a product of compact simplices the connections can couple the fields and alter stability. With a fixed external input and smooth wiring, compactness does ensure continuation of solutions for all forward times.
Composition by state identification
For chemical reactions and transport, a different interface uses labels for shared species or sites. Represent an open network by a cospan I→S←O of finite sets. Its internal field is vS:RS→RS on concentrations, or the restriction to nonnegative concentrations. A map j:S→P induces pullback j∗ which duplicates concentrations and pushforward j∗ which sums their velocities. The transported field on P is j∗vSj∗.
Suppose IiSoO is followed by Oi′To′J. The new species set is the pushout. Denote the structure maps into it by jS,jT. The field of the composite is vP=(jS)∗vSjS∗+(jT)∗vTjT∗. Every species identified by a shared port gets the velocity contributed by both components. For a third composable cospan, the two iterated pushouts are canonically isomorphic. In the common pushout, the contributions from each original module are the transported fields along its composite structure map. This follows from (kj)∗=k∗j∗, (kj)∗=j∗k∗, and distributivity of linear pushforward over addition. The zero field on the identity cospan is a unit. Disjoint union puts two networks side by side without adding identifications between them.
For a mass-action reaction with input complex α∈NS, output complex β∈NS, and rate constant k, its field is vS(x)=kxα(β−α). The identity (j∗x)α=xj∗α shows term by term that transport of the field is the field for the reaction with complexes j∗α,j∗β. Therefore gluing networks and then taking the mass-action equations gives the same field as gluing the module fields. The general construction of [25] retains boundary interfaces and gives a functor from open reaction networks to open dynamical systems.
The same rule applies to linear master equations with a column-conservative generator Q on RS. The induced matrix j∗Qj∗ has nonnegative off-diagonal entries since those entries sum coefficients between disjoint fibers of j. Its columns sum to zero since 1⊤j∗=1⊤ on the source. Thus gluing in this sense preserves the generator conditions. A quotient of an existing time evolution needs an intertwining condition in addition. Open Markov processes and their coarse-graining are studied in [26].
Linear conservation laws also pass through the pushout when their coefficients agree on the shared species. Suppose ℓSTvS=0 and ℓTTvT=0 pointwise, and o∗ℓS=(i′)∗ℓT. There is a unique coefficient vector ℓP restricting to both local vectors. The identity ℓPT(jS)∗=(jS∗ℓP)T, and its counterpart for T, annihilate the two terms of the composite velocity separately. Hence ℓPTx is constant along the closed flow. When boundary fluxes are supplied, its derivative equals the weighted incoming flux minus the weighted outgoing flux. This compatibility criterion applies to conserved species weights even when the components use different internal coordinates.
Boundary behavior of steady states
A steady state with boundary fluxes a∈RI and d∈RO satisfies vS(x)+i∗a−o∗d=0. Its external data are the concentrations i∗x,o∗x and the fluxes a,d. They define a relation between boundary data. The relation of the composite is obtained by composing the two relations across the shared concentrations and fluxes. Here is the exact reason.
The pushout diagram means that a concentration vector on P is equivalent to two vectors on S,T agreeing after pullback to O. The shared flux d enters the two steady equations with opposite signs. Applying (jS)∗,(jT)∗ and adding cancels it and gives the composite steady equation. Conversely suppose the composite steady equation holds and restrict the concentrations to S,T. Form the two local residuals including their external fluxes but omitting the shared flux. These residuals lie in the kernel of the sum of the two pushforwards to P. The sequence RO(o∗,−i∗′)RS⊕RT(jS)∗+(jT)∗RP⟶0 is exact since the free vector space functor preserves the pushout. Thus the residuals are obtained from some common flux with opposite signs, giving two local steady equations. This proves the relation formula for unrestricted real port fluxes. Additional inequalities on fluxes require their own feasibility check for such a factorization. The relation records possible external observations of steady states. Trajectories and the conditions for reaching those states remain part of the dynamical description.
Semiconjugacy and deterministic aggregation
An aggregate description of selection groups several strategies under the same observed label. Its exactness is a differential question. For every microscopic population with a given aggregate, the instantaneous aggregate velocity must agree. A coarse field that fits a single trajectory need not describe nearby microscopic states with the same observed population.
Flow maps and projectable fields
Let F and G be smooth vector fields on M and N and let q:M→N be smooth. We call q a semiconjugacy when it carries solution curves of F to solution curves of G with the same time parameter. This is equivalent to DqxF(x)=G(q(x))(x∈M). Indeed, for a source solution x(t), the chain rule makes q(x(t)) a solution of the target initial-value problem. Uniqueness then gives commutation of the two flow maps on their common time domain. Conversely, differentiating this commutation at zero gives (10). On simplices we use smooth fields on relative neighborhoods whose forward flows preserve the simplices. In this case the derivative at zero may be taken from the right.
A complete forward flow determines a functor B(R≥0,+)→Set. Here B makes the monoid of nonnegative times into a one-object category, and the functor sends addition of times to composition of evolution maps. The square says precisely that q is the component of a natural transformation. Smooth semiconjugacies form a category with ordinary identities and composition. The naturality condition uses a common clock. A state-dependent time change such as the positive rescaling in Chapter [ch:compositional-games] requires a separate description.
Given F and a surjective submersion q, a suitable G exists exactly when DqxF(x) is constant on every fiber q−1(y), as a vector in TyN. Necessity is immediate from (10). For sufficiency, define the candidate at y to be this common value. A local smooth section of the submersion expresses it as a smooth field on a neighborhood of y, and the expressions from different sections agree. Surjectivity also gives uniqueness. For an affine quotient of simplices by a partition, a global affine section is available. Choose a probability vector supported in each block and lift coarse masses to these fixed proportions. Evaluating DqF on that section gives the smooth candidate whenever the fiber condition holds. The candidate preserves the coarse simplex whenever the original flow preserves the fine simplex, by applying the chain rule to a source solution through a lift of each coarse initial state.
Partition quotients of selection
Let b:S↠T be a partition map and let Bαi be 1 if b(i)=α, and 0 otherwise. Write z=Bp for the image of the population p, so zα=∑b(i)=αpi. Let Fi(p)=pi(fi(p)−fˉ(p)) with fˉ(p)=∑ipifi(p). For every block with zα>0 define fα(p)=zα1b(i)=α∑pifi(p). Summing the microscopic equations and regrouping the mean gives (BF(p))α=zαfα(p)−β:zβ>0∑zβfβ(p) on the active blocks. The other block velocities vanish, since all of their microscopic masses are zero.
Proposition 1.1. For a proposed coarse payoff field g(z), the partition sends the replicator field for f to the replicator field for g if and only if fα(p)−gα(Bp) is independent of α among the active blocks for every p.
Proof. Subtract the two velocities on each active block and divide by its positive mass. The resulting equality requires the difference between the two payoffs to equal the weighted mean of those differences on the active blocks. Hence all of the differences are equal there. A common difference cancels in the centered payoffs and yields equality of the velocities. Inactive blocks contribute neither mass nor velocity to either calculation. ◻
A sufficient condition is fi(p)=gb(i)(Bp)+c(p) with the same scalar c(p) for all strategies. For matrix payoffs, the corresponding form is A=BTAˉB+1cT, where c is a constant vector and Aˉ is the coarse payoff matrix. The proposition is weaker than pointwise payoff agreement within each block. Microscopic deviations from the proposed block payoff can cancel in the population average. This cancellation is tested at every microscopic population in the domain.
Comparison with linear lumpability
For a Markov equation p˙=Qp with a column-conservative generator, the same partition is exact for another generator Q if BQ=QB. The column of BQ at a microscopic state i records the total signed rate into each coarse block. A matrix Q exists precisely when these columns agree for all i in each source block. To see this, compare the matrices on each basis vector. If the equalities of columns hold, define the coarse column using any microscopic representative of the block. The sums of its entries vanish because the columns of Q sum to zero. Its off-diagonal entries are nonnegative because the corresponding target blocks contain no source state. Thus the resulting matrix is a Markov generator. Equivalently, the intertwining holds for the matrix exponentials at every nonnegative time, by applying it to each power and summing the exponential series. The implication in the reverse direction follows by differentiating the exponentials at zero. For open Markov processes, preservation of interfaces and fluxes adds constraints to the condition on the generators [26]. In the nonlinear selection equation the fiber test applies to the centered payoff averages, whose weights are themselves microscopic state variables.
Observed equilibria and stability
Semiconjugacy carries a rest point of F to a rest point of G. Over a target rest point the condition says DqxF(x)=0, allowing motion tangent to a fiber. If V is a Lyapunov function for G, then D(V∘q)xF(x)=DVq(x)G(q(x)). Thus the same monotonicity is inherited by the pullback, with strictness only where the coarse Lie derivative is strict. These facts identify the limit of equilibrium and stability information that can be recovered from aggregated coordinates alone.
Finite-time error under a nonzero defect
Suppose the target is represented in a Euclidean domain, as for an affine simplex. The defect of an observation is rq(x)=DqxF(x)−G(q(x)). Suppose G is L-Lipschitz on a region containing both the observed source solution q(x(t)) and a target solution z(t) over [0,T]. If the defect along the source solution has norm at most ϵ, then for 0≤t≤T, ∥q(x(t))−z(t)∥≤eLt∥q(x(0))−z(0)∥+{ϵ(eLt−1)/L,ϵt,L>0,L=0. The integral equations for the two curves, followed by the triangle inequality, bound their distance at t by its initial value plus the integral of L times the current distance and ϵ. The integral form of Gronwall’s inequality yields the claim. For polynomial fields, formal approximate differential equivalences give more specific ways to find and certify reductions [27].
If h:N→P compares G to a third field J, the chain rule gives rh∘q(x)=Dhq(x)rq(x)+rh(q(x)). In particular a bound on Dh controls the magnification of the first reduction’s defect under composition. The error estimate uses a specified time interval and hypotheses about the region visited by the comparison trajectories. Long-time conclusions need additional information. For example, if the target satisfies the contraction inequality ⟨u−v,G(u)−G(v)⟩≤−λ∥u−v∥2(λ>0) on the comparison region, the error in Euclidean norm satisfies ∥q(x(t))−z(t)∥≤e−λt∥q(x(0))−z(0)∥+λϵ(1−e−λt). For positive distance, differentiate its square, apply the contraction inequality and Cauchy–Schwarz, and divide by twice the distance. At distance zero the upper right derivative of the distance is at most ϵ, since the difference of velocities is then the defect itself. Integrating the resulting differential inequality gives the bound. If the two trajectories remain in the region, the bound holds on their whole common forward time domain.
Fitness systems over varying contexts
A population in an ecological or strategic context uses a set of available types, a state of its interactions, and a rule for evaluating prospective behavior. When contexts vary, these data can have nontrivial transition maps. A model can then be a family of games without having a single global numbering of the types. This chapter uses sheaves to state exactly what must agree between presentations.
An internal strategy object
Let B be a smooth manifold and let p:S→B be a smooth locally constant finite covering, with all fibers nonempty. Its sheets over an open set give types in that context. The set of types is a sheaf of finite sets over B. Think of an open subset as a stage on which type labels may be observed. On a trivializing open set U, the probability state space over U is U×Δ(IU), where IU indexes the local sheets and Δ(IU)={x∈RIU:xi≥0,i∑xi=1}. On an overlap the sheets are related by a locally constant permutation σUV(b):IV→IU. The transition (b,x)↦(b,σUV(b)x) carries faces to faces and preserves both the mass and the nonnegativity conditions. The transitions assemble a simplex bundle over B. Equivalently, the simplex is a sheaf of states, using smooth plots on each context. The closed fibers have corners. Smoothness of a fitness map on such a fiber means that its local components extend smoothly to a neighborhood in the affine span.
A fitness operator is a section of the relative function object from the simplex bundle to the bundle of real type-indexed vectors. Concretely it is a collection of functions fU(b,x)∈RIU with fU(b,σUV(b)x)=σUV(b)fV(b,x). This is a descent condition on maps between bundles. It can be applied equally to restricted subfamilies of fitness operators. For instance a collection of pairwise payoff matrices AU(b) gives the operators AU(b)x when the matrices satisfy AU(b)σUV(b)=σUV(b)AV(b). A constant affine shift of all payoffs leaves the replicator field unchanged, as in Chapter [ch:payoff-equivalence]. It can be convenient to take the corresponding quotient of fitness data before imposing compatibility.
Local generators and their gluing
For each context b and a local vector x, the replicator operator assigns RfU(b,x)i=xi(fU(b,x)i−⟨x,fU(b,x)⟩). Its values are tangent to the affine hyperplane of mass one. Every zero coordinate remains zero in that field. These identities are pointwise in b and are independent of the trivialization.
Proposition 1.1 (Descent of replicator fields). Under the fitness transition law above, the replicator fields glue to a vertical vector field on the simplex bundle. The flow on any fiber is compatible with relabeling and with restriction to context opens. If the fitness fields are smooth up to the boundary in the specified sense, the flows exist for all nonnegative times on each simplex fiber.
Proof. A permutation σ preserves the Euclidean pairing, preserves coordinatewise multiplication, and fixes the all-ones vector. Therefore RfU(b,σx)=σRfV(b,x). The derivative of the transition in the vertical coordinate is σ, so these equations are exactly the gluing condition for a vertical field. The equality is preserved under further open restriction.
Because the fitness is smooth in an affine neighborhood, the local differential equation has unique solutions. A solution for the V-presentation, carried across the overlap, is a solution in the U-presentation with the same initial condition as the one in that chart. Uniqueness gives the flow comparison. The normalized mass and support identities of Chapter [ch:replicator] keep each trajectory in its compact simplex. The continuation theorem for smooth ordinary differential equations then gives existence for all nonnegative time at every fixed b. The mass calculation only uses the finite set indexing the simplex. It does not require a distinguished order on that set. ◻
The commutative square records how a transition acts on the whole flow, rather than on its rest points alone: Here t is elapsed time in a fixed context, not a path in B. A model in which context itself changes with time needs a rule for that motion. A connection or transition process would be additional data.
Parametrization and effective descent
In an ∞-topos E, let X,Y,P be objects over a base B. A family of evaluations from X to Y with parameter P is a morphism P×BX→Y. By the adjunction for the exponential in E/B, such a family is classified by P→[YX]B, where [YX]B denotes the fiberwise exponential. An inverse-image or presentation change can preserve evaluation while changing parameter coordinates. The pullback formula follows by applying the functor composing structure maps to the exponential adjunction: for q:C→B and T→C, q!(T×Cq∗X)≃q!T×BX. Thus q∗[YX]B≃[(q∗Y)q∗X]C by Yoneda. If U→B is an effective epimorphism with Čech nerve U∙, then E/B≃[n]∈ΔlimE/Un. This is descent for slices in an ∞-topos [28, §6.1.3]. Objects and morphisms over B can thus be specified locally along with a homotopy-coherent chain of overlap comparisons. The comparison extends to evaluation families since their parameter maps land in the compatible exponential object.
This gives a semantics for local games. A family can be observed by changing the base, and the result of interpreting that family is independent of a selected cover. For the concrete smooth model the relevant gluing equations reduce to equalities; for spaces with groupoid-valued states a commuting triangle may instead carry an invertible comparison. One must still specify which points are probability states and which maps define fitness. The categorical descent theorem glues these structures once they are defined in the chosen semantics. It does not supply a probability law from the base topos.
Monodromy and restricted parameter choices
A loop in B transports its strategy fiber by a permutation. The image of the resulting representation of the fundamental group is a monodromy group M. In a connected component, a flat state is a locally constant probability vector in the trivializations. It is specified by a vector x at one base point that is fixed by every permutation in M. To prove it, transport a choice from that point along paths. This is independent of the path exactly when every based loop fixes the vector. Flat states therefore have a common value within each orbit of M on types. If O is such an orbit and wO is its mass, each of its coordinates has value wO/∣O∣. The flat parameter region is consequently a simplex on monodromy orbits. A general smooth section of the bundle can vary within a context and needs only to obey transition laws.
A flat payoff matrix, by the same argument, is fixed under conjugation by the monodromy permutations. Its entry depends on an orbit of the diagonal action on ordered pairs of types. These conditions can force parameter tying, even when local populations have several distinguishable types. Conversely, the pair orbits may retain interaction patterns that are lost if one replaces all types by their orbit labels before giving the fitness. Context determines which changes of presentation are legitimate. The next chapter measures a different failure, when a coarse description is a randomized channel rather than a permutation.
Selection under stochastic compression
An observer can group strategies by a classification rule. A deterministic grouping sends each strategy to one class, whereas a randomized classifier sends it to a distribution of classes. These two changes of description have different effects on replicator dynamics. When fitness is pulled back along the classifier, the difference is given by a conditional covariance.
Columns as conditional distributions
Let I,J be nonempty finite sets, and let A be a column-stochastic matrix from I to J. Its column ai is the conditional distribution of the label j∈J given a strategy i∈I. Write x∈Δ(I) and z=Ax. The pushforward of a distribution is Ax. The corresponding pullback of a payoff vector γ∈RJ is A⊤γ, since ⟨x,A⊤γ⟩=⟨Ax,γ⟩. This is expectation along a Markov kernel, expressed in dual coordinates [20]. If g:Δ(J)→RJ is the observed fitness, consider the fine fitness f(x)=A⊤g(Ax)+c(x)1I, where c is a common payoff offset. This assumption controls the dependence of a fine type’s payoff on all its possible labels. It must be checked in a proposed aggregation.
For a probability vector w, set M(w)=diag(w)−ww⊤. The replicator field is Rf(w)=M(w)f(w). This form includes the constraint that shifts by a common payoff have zero field, since M(w)1=0. For a channel A define CA(x)=diag(Ax)−Adiag(x)A⊤.
Theorem 1.1 (Covariance of a compressed replicator field). Under the payoff hypothesis above, Rg(Ax)−ARf(x)=CA(x)g(Ax). The matrix CA(x) is positive semidefinite. At a given x the defect vanishes exactly when g(Ax) is constant on the support of every column ai with xi>0. In particular, exact pushforward for all x and all coarse fitness maps holds precisely when A represents a deterministic function I→J.
Proof. The common offset disappears in M(x)f(x). Using Axx⊤A⊤=(Ax)(Ax)⊤ gives M(Ax)−AM(x)A⊤=CA(x). Multiplication by g(Ax) gives the claimed difference. For an arbitrary test vector u∈RJ, CA(x)u⊤CA(x)u=i∑xi(diag(ai)−aiai⊤),=i∑xij∑(ai)juj2−(j∑(ai)juj)2. The summands are conditional variances and are nonnegative. A variance is zero precisely when u is constant on the support of the corresponding column. For a positive semidefinite real symmetric matrix C, u⊤Cu=0 is equivalent to Cu=0 by diagonalization. Taking u=g(Ax) proves the equality criterion. Every deterministic column has zero covariance. Conversely choose x concentrated on any specified i and allow a constant target fitness taking an arbitrary vector value. Universal vanishing then forces every coordinate indicator to have zero variance under ai, so every coordinate probability is either zero or one. Their sum is one, proving determinism. ◻
Deterministic compression groups types without spreading any type across distinct observed labels. Its exactness is the coordinate expression of summing replicator derivatives over the fibers of a function, when the payoffs agree within fibers. In stochastic compression, equality can still hold for particular payoffs or particular support faces. It is then a condition on the conditional distribution and the payoff together.
Composition of two compressions
For a second channel B:J→K the defects obey CBA(x)=CB(Ax)+BCA(x)B⊤. Indeed, first add and subtract Bdiag(Ax)B⊤ in the expression for CBA(x). The two terms are the covariance from the second channel and the covariance of its conditional means induced by the first. This is the law of total covariance for the original sampling sequence. The formula treats loss of a selection signal as a two-stage correlation calculation.
The two summands are positive semidefinite. If the final fitness is v∈RK, exact pushforward through the composite holds at x precisely when CB(Ax)v=0 and CA(x)B⊤v=0. To see this, test the displayed expression in the quadratic form at v. It is a sum of two nonnegative numbers, each with the zero criterion of Theorem 1.1. The two equality conditions correspond to asking that the final payoff be constant on the occupied conditional supports of the second stage and that its first-stage conditional mean be constant on the relevant supports of the first. This criterion can be applied before composing the channels.
If each step is deterministic, the maps and the exact fitness pullbacks compose without defect. This yields a class of morphisms of selection systems. The same category of stochastic maps can be used for compression with a nonzero defect. In that case the formula identifies the specific obstruction to the map being a morphism of replicator dynamics.
Trajectory error on a bounded time interval
Write x(t) for the fine solution and z(t) for the coarse solution starting at Ax(0). Suppose both remain on their simplices and that Rg is Lipschitz there with constant L for a fixed norm. Define ct=CA(x(t))g(Ax(t)). The projected fine solution satisfies dtdAx(t)=Rg(Ax(t))−ct. Subtracting the coarse equation and applying the integral form of Grönwall’s inequality yields ∥Ax(t)−z(t)∥≤∫0teL(t−s)∥cs∥ds. One derivation is to integrate the difference of the two derivatives, bound the target field difference by L∥Ax−z∥, and iterate the resulting integral inequality. Its Volterra series sums to the displayed exponential. If the defect is identically zero, uniqueness gives exact semiconjugacy. If the defect is merely small, the estimate controls finite-time approximation. It does not give a uniform long-time guarantee when the bound grows with t. For other mechanisms that yield semiconjugacy and lumpability, see Chapters [ch:aggregation] and [ch:semiconjugacy].
The tangent and cotangent interpretation
On the relative interior of a simplex, the metric ⟨v,w⟩x=i∑xiviwi,i∑vi=i∑wi=0 identifies the centered payoff covector with its replicator vector. In fact ⟨Rf(x),v⟩x=∑ifi(x)vi. The metric is the familiar Shahshahani metric [7]. Its inverse on covectors modulo constant vectors has matrix M(x). If Ax is on the boundary of the target simplex, the same construction applies on the relative interior of its support face, with its inverse metric represented in the full coordinates by zero extension. The difference between the target inverse metric and the pushforward of the source inverse metric is CA(x). The covariance is thus also the change in the quadratic form used to convert a payoff into a selection direction. This observation explains why contraction of information under sampling need not make evolution commute with sampling. The statement proved here uses ordinary finite-dimensional distributions and an exact payoff pullback, which are strong enough to identify the defect in closed form.
Action groupoids and symmetric selection
A permutation group can act on a game by relabeling its types. The resulting symmetry has a fixed locus, an orbit space, and an action groupoid. Each answers a different question. This chapter puts these constructions alongside the probability simplex and uses them to separate symmetry of a state from the quotient of a family of states.
The fixed simplex and equilibrium
Let a group G act by permutations of a nonempty finite set I, and suppose f:Δ(I)→RI is continuous and equivariant. The image of G in the permutation group is finite. The fixed simplex Δ(I)G consists of the states with equal coordinates on each orbit, so it is naturally a simplex on the orbit set. If p is fixed, f(p) is also fixed. The replicator field and its flow preserve this fixed simplex by the descent calculation in Chapter [ch:contextual-games], whenever the field satisfies the existence and uniqueness hypotheses there.
There is an equilibrium in the fixed simplex under the given continuity hypothesis, even before choosing a dynamic solution.
Proposition 1.1 (An invariant Nash state). Under these hypotheses there is a G-fixed p such that ⟨y,f(p)⟩≤⟨p,f(p)⟩ for every y∈Δ(I).
Proof. For any x put ai(x)=max{fi(x)−⟨x,f(x)⟩,0} and s(x)=∑iai(x). The continuous map Fi(x)=1+s(x)xi+ai(x) is equivariant because each operation in its definition commutes with permutation. It maps the fixed simplex to itself. By Brouwer’s theorem it has a fixed point p there. The fixed point equation gives ai(p)=s(p)pi. If s(p)>0, each occupied type has strictly higher fitness than the mean. Taking the mean over all occupied types would give a strict inequality with itself. Hence s(p)=0, and the individual inequalities fi(p)≤⟨p,f(p)⟩ give the result for all y by linearity in y. ◻
This calculation uses the ambient equilibrium inequalities. One can also compress along the orbit function and work with orbit masses, provided the fitness assigned to each orbit is the correct one. A restriction of the dynamics to symmetric states only tests attraction in the fixed simplex. For smooth fitness, at an interior fixed point the derivative of the field is a linear map commuting with the permutation action on the tangent space. The averaging projector onto the fixed vectors commutes with that map. The tangent space consequently splits into the fixed part and the kernel of averaging, both invariant under the derivative. Stability on the fixed part leaves the transverse block to be tested separately. At the boundary, feasible perturbations are governed by the tangent cone of the ambient simplex and can enter larger faces.
What the quotient records
For an action on a space X, write X//G for the realization of the action bar object [n]↦X×Gn. It maps to the delooping BG and the fiber over its distinguished point is X. If X is an ordinary discrete set, this is the groupoid of elements and symmetry arrows. After choosing orbit representatives x, it is equivalent to the coproduct of BGx over those representatives, where Gx is the subgroup fixing x. Every orbit is a connected groupoid and inclusion of the full subgroupoid at one object is an equivalence. The set of connected components is the coarse orbit set. The fuller quotient retains the automorphism groups of pure states.
The whole probability simplex makes the role of the acted-on object clear. It has an equivariant contraction λx+(1−λ)u to the uniform state u. Its homotopy quotient over BG is therefore equivalent to BG. The pure strategy set, a rest-point subspace, or a space of trajectories may have different quotient types. Equivariant contractibility of the whole simplex gives its quotient type. Each subspace requires a separate computation.
For a smooth equivariant field, the space of trajectories on a fixed time interval carries the action by transport of each trajectory. Initial and terminal evaluation maps are equivariant. In a well-posed deterministic system the initial evaluation is an equivariant homeomorphism to the initial state space, using continuous dependence on initial conditions. Terminal evaluation is then the flow map expressed through this identification. For a set-valued best response or a multivalued update (see Chapter [ch:revision]), the corresponding initial evaluation can have multiple trajectories over a state. In that situation its fibers encode a choice of resolution of ties or admissible continuations.
Coherent actions in an ∞-topos
An internal group object in an ∞-topos E has a reduced bar groupoid G∙ and a delooping BG. Effectivity identifies G∙ with the Čech nerve of 1→BG. An action on X can be specified by a cartesian transformation into G∙ with degree-zero object X. Its realization is the map X//G→BG. Conversely any object over BG pulls back along the Čech nerve to a cartesian transformation. Descent for slices shows that these assignments are inverse [28, 29, §6.1.3]. In particular, ActG(E)≃E/BG. The corresponding square is a homotopy pullback. This describes quotient families while retaining their lifts and all compatibility conditions for change of context.
An evaluator P×X→Y between equivariant objects is represented in this slice, so its local evaluations must transform both their parameters and inputs. For a discrete group acting on spaces, the space of equivariant maps from X to Y is the homotopy fixed-point space of Map(X,Y) with conjugation transport. For example, the cosimplicial description has in degree n a family of maps for each string in Gn. Transporting the domain along the string identifies it with the usual cosimplicial diagram for the homotopy limit of the conjugation action. Totalization imposes naturality and its higher coherence. The assertion concerns the space of possible equivariant maps. The existence of a natural dynamical invariant, a distinguished equilibrium, or a numerical update requires its own mathematical conditions.
In these constructions the context can be an ordinary parameter space, a classifying groupoid, or an internal base for locally varying types. Equivariance and descent describe how locally constructed fitness and revision laws assemble. They do not determine a revision law without assumptions on its behavioral or statistical meaning. Weight updates provide a particularly direct case where those assumptions and the resulting equations can be specified together.
Learning rules and populations of agents
Selection can act on candidate policies or programs as well as on inherited traits. In a learning system, three choices enter explicitly: what counts as a strategy, how its score is obtained, and how that score changes its frequency. The replicator field is an infinitesimal revision rule. A discrete-time learner can approximate it or use a different update with its own statistical guarantee.
An update from relative entropy
Let x be a probability vector with positive coordinates on a finite set I. Given a payoff vector a and a step η>0, exponential weights defines xi+=∑jxjeηajxieηai. A common additive shift of the payoff cancels. The update uniquely maximizes on the simplex y⟼⟨y,a⟩−η−1D(y∥x),D(y∥x)=i∑yilog(yi/xi), where a zero summand is interpreted as zero. Indeed, substitution of x+ writes the objective as a constant minus η−1D(y∥x+). Nonnegativity of relative entropy and its equality condition follow from Jensen’s inequality for log, with terms of zero mass handled by continuity. If the initial state has zeros, the same definition is used on its support face. This is a multiplicative weights update [30, 31].
Differentiation at η=0 gives ∂η∂xi+η=0=xi(ai−⟨x,a⟩). Putting a=f(x) yields the replicator field. If f is Lipschitz on the compact simplex, it is bounded there and the local truncation error relative to the Euler step is uniformly O(η2). Indeed the denominator is bounded away from zero for small η and the second derivative with respect to η is uniformly bounded. On a bounded time interval, compare with the Lipschitz ordinary differential equation. The local flow also differs from its Euler step by O(η2). The discrete error then satisfies a recursion bounded by (1+Lη) times the previous error plus O(η2). Summing the geometric series gives an O(η) error at grid times for fixed elapsed time. This connects the update to continuous selection without drawing a conclusion about arbitrarily long training.
The pathwise regret bound
Suppose payoffs at are observed each round, with ∣ait∣≤M for every coordinate and round. Their choice may depend on previous weights. Let xt be the pre-update state and η a fixed step size.
Proposition 1.1 (External regret of exponential weights). For a comparator p supported on the initial support, after T updates one has t=0∑T−1⟨p−xt,at⟩≤ηD(p∥x0)+2ηM2T.
Proof. Write Zt=∑ixiteηait. The update equation gives D(p∥xt+1)−D(p∥xt)=−η⟨p,at⟩+logZt. For a fixed round, the second derivative of log∑ixitesait is the variance of at under the distribution tilted by s. It is at most M2 since all of its values belong to [−M,M]. The first derivative at zero is ⟨xt,at⟩. Integrating the second derivative bound gives logZt≤η⟨xt,at⟩+η2M2/2. The claimed inequality follows by summing and dropping the nonnegative final relative entropy. ◻
This is a comparison to a fixed behavior in hindsight. With an appropriate step as a function of the horizon, the bound divided by T tends to zero. It allows an arbitrary sequence of bounded revealed scores and does not require that a single game produce them. For continuous-time regularized reinforcement learning and its relation to replicator and other dynamics, see [32].
Averaged joint play
The regret theorem yields a statement about a game when the scores are the actual counterfactual payoffs of that game. For a finite action product, let ui be player i’s utility. Each round the players independently use their mixed strategies conditioned on the history, giving a product law νt. If player i receives the payoff vector of expected utilities of each of its possible actions against the other players’ current choices of distributions, its payoff under its own distribution is Eνtui(a). Its payoff from a fixed unilateral deviation bi is Eνtui(bi,a−i). Set νT=T−1∑tνt. Then the coarse correlated equilibrium inequalities take the form EνT[ui(bi,a−i)−ui(a)]≤TRegreti(T) for every i,bi, with regret defined as the maximum over the fixed deviations (or a bound on that maximum). This is obtained by adding the per-round deviations. When these bounds tend to zero every accumulation point of the averaged joint laws is a coarse correlated equilibrium, by continuity of the finitely many linear inequalities. The conclusion concerns joint laws, which can carry correlation through their common time index. A Nash statement on the last product distribution would require other hypotheses. This distinction is important in agent populations whose behavior continues to cycle or react to changing opponents.
Outer selection and the evaluation interface
A program strategy can be represented by a fixed artifact, with an evaluation map from its identity and the current context to a score. The test distribution, budget, counterfactual comparison class, and timing determine whether the score estimates the intended fitness. An inner process can generate or train candidate artifacts. An outer process can then keep a distribution over verified candidates and update that distribution using scores. Exponential weights keeps the distribution in the simplex and is insensitive to score shifts common to all candidates in a round. These facts are properties of the update. They alone say nothing about the behavior or validity of any candidate.
There is also a compositional concern. If an evaluator reports the same payoff for every strategy in a coarse fiber, deterministic aggregation can commute with selection. If it reports only a randomized label, the calculation of Chapter [ch:stochastic-compression] identifies a covariance defect even when the fine payoffs are conditional expectations of coarse scores. This gives a test of a proposed reporting interface before using it as a summary of population behavior. Delayed reports, context changes, or evaluation mismatch introduce separate errors. A learning guarantee is conditional on specifying those mechanisms and bounding their effect.
The constructions in the book put equilibrium, dynamics, stochastic modeling, and presentation changes into compatible types. A model of interacting agents adds an interpretation of strategies and a mechanism for producing scores at the chosen interface. The resulting learning statements remain relative to that mechanism.
Notation
Type set, its probability simplex, and mass assigned to type i. See Chapter [ch:population].
The current fitness vector and its mean under x.
The replicator field and the matrix diag(x)−xx⊤ that converts a payoff covector into a selection direction. See Chapters [ch:replicator] and [ch:stochastic-compression].
Relative entropy of a comparator p from the state x, finite when the support of p is contained in that of x. See Chapters [ch:lyapunov] and [ch:agent-learning].
The types with positive mass in x. The associated support face contains trajectories from x for the replicator field without mutation.
In stochastic chapters, transition kernels or generator matrices with conventions specified at their use. In parametrized games, P can also denote a parameter object.
The pullback of payoff vectors along a column-stochastic channel A, and its conditional covariance diag(Ax)−Adiag(x)A⊤. See Chapter [ch:stochastic-compression].
Forward and backward types of an open-game interface. See Chapter [ch:open-games].
The slice over a context object B and its fiber product. See Chapter [ch:contextual-games].
The delooping of an internal group object and the homotopy quotient of a G-object. See Chapter [ch:higher-equivariance].
Machine-checked selection identities
The accompanying Lean project formalizes the algebra that makes deterministic aggregation compatible with selection. It uses Lean 4.19.0, pinned mathlib, and 23 audited theorems. The index types are arbitrary finite types. The statements apply uniformly to every cardinality, including reindexing along a map that leaves some target coordinates empty. See [33] for the proof assistant.
Drift and the normalized update
Let x,u:I→R, where I has a finite type structure. The formal definitions are m(x)=i∑xi,μx(u)=i∑xiui,F(x,u)i=xi(ui−μx(u)). Over a commutative ring the kernel checks i∑F(x,u)i=(1−m(x))μx(u). It follows that the drift is tangent to the mass-one hyperplane. The zero-coordinate property, invariance under adding a common payoff on that hyperplane, and linearity under uniform payoff scaling are also checked. These are identities of algebraic expressions used in Chapter [ch:replicator].
Over a field the project defines the reproductive update S(x,u)i=μx(u)xiui. It verifies unit mass when the denominator is nonzero. Over a linearly ordered field with x in the simplex and strictly positive fitness u, Lean derives positivity of the denominator and proves that the output lies in the simplex. For affine reproductive weights wi=1+ηui, under m(x)=1 and 1+ημx(u)=0, it also checks the exact discrete-to-drift identity S(x,w)i−xi=1+ημx(u)ηF(x,u)i. No time limit is needed to obtain this equality.
Deterministic aggregation
For a map q:I→J, the Lean definition of aggregation is (q∗x)j=∑q(i)=jxi. The project proves preservation of mass and of simplex membership, functoriality under composition, and the pullback-pushforward pairing μx(v∘q)=μq∗x(v). For v:J→R it proves the two commutation identities q∗F(x,v∘q)=F(q∗x,v),q∗S(x,v∘q)=S(q∗x,v). A state-dependent form and a specialization to pullbacks of payoff matrices make the first equation applicable to the deterministic semiconjugacy of Chapter [ch:semiconjugacy]. Each result uses finite sums and function extensionality. The second identity is algebraic as an equality of field terms. Its population interpretation uses the nonzero normalizer as in the preceding section.
Reproducing the check
From the extracted Lean source, run lake exe cache get, lake build, and lake env lean Audit.lean. The manifest pins the mathlib revision. The last command prints the transitive axioms of the audited declarations. The supplied audit records standard Lean axioms and no admitted proofs. The appendix covers the algebraic vector field and reproductive update. The main text gives the analytical and higher-categorical arguments, including existence of flows, stochastic limits, and the channel covariance formula in Chapter [ch:stochastic-compression].
Subject index
References give the chapters or appendices where each term is developed.
J. McKenzie Alexander. Evolutionary Game Theory. Elements in Decision Theory and Philosophy, Cambridge University Press, 2023. doi:10.1017/9781108582063.
J. W. Weibull. Evolutionary Game Theory. MIT Press, 1995. Publisher.
J. Maynard Smith and G. R. Price. The logic of animal conflict. Nature 246 (1973), 15–18. doi:10.1038/246015a0.
P. D. Taylor and L. B. Jonker. Evolutionarily stable strategies and game dynamics. Mathematical Biosciences 40 (1978), 145–156. doi:10.1016/0025-5564(78)90077-9.
J. Hofbauer, P. Schuster, and K. Sigmund. A note on evolutionary stable strategies and game dynamics. Journal of Theoretical Biology 81 (1979), 609–612. doi:10.1016/0022-5193(79)90058-4.
J. Hofbauer and K. Sigmund. Evolutionary Games and Population Dynamics. Cambridge University Press, 1998. doi:10.1017/CBO9781139173179.
S. Shahshahani. A New Mathematical Framework for the Study of Linkage and Selection. Memoirs of the AMS 17, no. 211 (1979). doi:10.1090/memo/0211.
J. Hofbauer and W. H. Sandholm. Stable games and their dynamics. Journal of Economic Theory 144 (2009), 1665–1693.e4. doi:10.1016/j.jet.2009.01.007.
W. H. Sandholm. Pairwise comparison dynamics and evolutionary foundations for Nash equilibrium. Games 1 (2010), 3–17. doi:10.3390/g1010003.
W. H. Sandholm. Population Games and Evolutionary Dynamics. MIT Press, 2010. Publisher.
J.-P. Aubin and A. Cellina. Differential Inclusions: Set-Valued Maps and Viability Theory. Springer, 1984. doi:10.1007/978-3-642-69512-4.
J. Hofbauer and S. Sorin. Best response dynamics for continuous zero-sum games. Discrete and Continuous Dynamical Systems B 6 (2006), 215–224. doi:10.3934/dcdsb.2006.6.215.
H. Ohtsuki and M. A. Nowak. The replicator equation on graphs. Journal of Theoretical Biology 243 (2006), 86–97. doi:10.1016/j.jtbi.2006.06.004.
P. A. P. Moran. Random processes in genetics. Mathematical Proceedings of the Cambridge Philosophical Society 54 (1958), 60–71. doi:10.1017/S0305004100033193.
H. P. Young. The evolution of conventions. Econometrica 61 (1993), 57–84. doi:10.2307/2951783.
F. T. Leighton and R. L. Rivest. Estimating a probability using finite memory. IEEE Transactions on Information Theory 32 (1986), 733–742. doi:10.1109/TIT.1986.1057250.
T. G. Kurtz. Solutions of ordinary differential equations as limits of pure jump Markov processes. Journal of Applied Probability 7 (1970), 49–58. doi:10.2307/3212147.
M. Benaïm and J. W. Weibull. Deterministic approximation of stochastic evolution in games. Econometrica 71 (2003), 873–903. doi:10.1111/1468-0262.00429.
J. G. Kemeny and J. L. Snell. Finite Markov Chains. Springer, 1976. Publisher.
T. Fritz. A synthetic approach to Markov kernels, conditional independence, and theorems on sufficient statistics. Advances in Mathematics 370 (2020), 107239. doi:10.1016/j.aim.2020.107239.
N. Ghani, J. Hedges, V. Winschel, and P. Zahn. Compositional game theory. Proceedings of LICS (2018), 472–481. doi:10.1145/3209108.3209165.
J. Hedges. Morphisms of open games. Electronic Notes in Theoretical Computer Science 341 (2018), 151–177. doi:10.1016/j.entcs.2018.11.008.
D. Vagner, D. I. Spivak, and E. Lerman. Algebras of open dynamical systems on the operad of wiring diagrams. Theory and Applications of Categories 30 (2015), 1793–1822. Journal.
J. C. Baez and B. S. Pollard. A compositional framework for reaction networks. Reviews in Mathematical Physics 29 (2017), 1750028. doi:10.1142/S0129055X17500283.
J. C. Baez and K. Courser. Coarse-graining open Markov processes. Theory and Applications of Categories 33 (2018), 1223–1268. Journal.
L. Cardelli, G. Squillace, M. Tribastone, M. Tschaikowski, and A. Vandin. Formal lumping of polynomial differential equations through approximate equivalences. Journal of Logical and Algebraic Methods in Programming 134 (2023), 100876. doi:10.1016/j.jlamp.2023.100876.
J. Lurie. Higher Topos Theory. Annals of Mathematics Studies 170, Princeton University Press, 2009. Author manuscript.
T. Nikolaus, U. Schreiber, and D. Stevenson. Principal ∞-bundles: general theory. Journal of Homotopy and Related Structures 10 (2015), 749–801. doi:10.1007/s40062-014-0083-6.
Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55 (1997), 119–139. doi:10.1006/jcss.1997.1504.
S. Arora, E. Hazan, and S. Kale. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing 8 (2012), 121–164. doi:10.4086/toc.2012.v008a006.
P. Mertikopoulos and W. H. Sandholm. Learning in games via reinforcement and regularization. Mathematics of Operations Research 41 (2016), 1297–1324. doi:10.1287/moor.2016.0778.
L. de Moura and S. Ullrich. The Lean 4 theorem prover and programming language. Automated Deduction – CADE 28 (2021), 625–635. doi:10.1007/978-3-030-79876-5_37.
Complete flattened LaTeX source
\documentclass[11pt,oneside,openany]{book}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage[
paperwidth=7in,paperheight=10in,left=0.87in,right=0.87in,
top=0.90in,bottom=0.86in,headheight=15pt,headsep=14pt,footskip=25pt
]{geometry}
\usepackage{microtype}
\usepackage{mathtools,amssymb,amsthm}
\usepackage{xcolor}
\usepackage{tikz-cd}
\usepackage{enumitem}
\usepackage{caption,needspace,multicol}
\usepackage{titlesec}
\usepackage{fancyhdr}
\usepackage{xurl}
\usepackage{hyperref}
\usepackage{bookmark}
\definecolor{bookink}{HTML}{182C2B}
\definecolor{bookaccent}{HTML}{13615A}
\definecolor{bookmuted}{HTML}{546967}
\definecolor{bookrule}{HTML}{A3BBB7}
\hypersetup{
pdftitle={Evolutionary Game Theory in Higher Categories}, pdfauthor={Anonymous},
pdfsubject={Stability, stochastic selection, and compositional models},
colorlinks=true,linkcolor=bookaccent,citecolor=bookaccent,urlcolor=bookaccent,
bookmarksdepth=2
}
\AtBeginDocument{\color{bookink}}
\linespread{1.055}
\setlength{\parskip}{2pt plus 1pt}
\setlength{\parindent}{1em}
\setlength{\emergencystretch}{1.2em}
\raggedbottom
\clubpenalty=3000
\widowpenalty=3000
\allowdisplaybreaks[1]
\titleformat{\chapter}[display]
{\normalfont\huge\bfseries\raggedright}
{\sffamily\small\bfseries\color{bookaccent}\MakeUppercase{\chaptertitlename}\ \thechapter}
{8pt}{}
[\vspace{4pt}{\color{bookrule}\titlerule}]
\titlespacing*{\chapter}{0pt}{12pt}{20pt}
\titleformat{\section}{\Large\bfseries\raggedright}
{\color{bookaccent}\thesection}{0.65em}{}
\titlespacing*{\section}{0pt}{15pt plus 2pt minus 1pt}{6pt}
\titleformat{\subsection}{\large\bfseries\raggedright}
{\color{bookaccent}\thesubsection}{0.65em}{}
\captionsetup{font=small,labelfont={bf,color=bookaccent},skip=6pt}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{\small\color{bookmuted}\nouppercase{\leftmark}}
\fancyfoot[R]{\sffamily\small\color{bookmuted}\thepage}
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\chaptermark}[1]{\markboth{#1}{}}
\fancypagestyle{plain}{ \fancyhf{} \fancyfoot[R]{\sffamily\small\color{bookmuted}\thepage} \renewcommand{\headrulewidth}{0pt}}
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[chapter]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
\DeclareMathOperator{\Hom}{Hom}
\DeclareMathOperator{\Ob}{Ob}
\DeclareMathOperator{\id}{id}
\DeclareMathOperator{\Id}{Id}
\DeclareMathOperator{\Nat}{Nat}
\newcommand{\Set}{\mathbf{Set}}
\newcommand{\FinSet}{\mathbf{FinSet}}
\newcommand{\Cat}{\mathcal{C}}
\setcounter{tocdepth}{1}
\DeclareMathOperator{\tr}{tr}
\DeclareMathOperator{\Tr}{Tr}
\DeclareMathOperator{\Ad}{Ad}
\DeclareMathOperator{\Map}{Map}
\DeclareMathOperator{\Tor}{Tor}
\DeclareMathOperator{\Ext}{Ext}
\DeclareMathOperator{\Rep}{Rep}
\DeclareMathOperator{\End}{End}
\newcommand{\hrepPart}{}
\newcommand{\bookpart}[2]{ \addtocontents{toc}{\protect\hrepPartEntry{#1}{#2}} \gdef\hrepPart{{\sffamily\small\color{bookaccent}Part #1\enspace #2\par\vspace{12pt}}}}
\newcommand{\hrepPartEntry}[2]{ \Needspace{6\baselineskip} \par\addvspace{0.7\baselineskip} \noindent\textcolor{bookaccent}{\textbf{#1\enspace #2}}\par\nopagebreak[4]}
\titleformat{\chapter}[display]
{\normalfont\huge\bfseries\raggedright}
{\hrepPart\sffamily\small\bfseries\color{bookaccent}\MakeUppercase{\chaptertitlename}\ \thechapter}
{8pt}{}
[\global\let\hrepPart\empty\vspace{4pt}{\color{bookrule}\titlerule}]
\begin{document}
\frontmatter
\begin{titlepage}
\thispagestyle{empty}
\vspace*{0.16\textheight}
{\noindent\color{bookaccent}\rule{\textwidth}{1pt}\par}
\vspace{17pt}
{\fontsize{27}{34}\selectfont\bfseries\raggedright
Evolutionary Game Theory\\in Higher Categories\par}
\vspace{14pt}
{\noindent\Large Stability, stochastic selection,\\and compositional models\par}
\vspace{30pt}
{\noindent\large Anonymous\par}
\vfill
{\noindent\small November 2026\par}
\end{titlepage}
\tableofcontents
\mainmatter
\bookpart{I}{Fitness, invasion, and stability}
\chapter{Population states and payoff maps}
\label{ch:population}
Evolutionary game theory uses a payoff comparison to describe how the
composition of a population changes the prospects of its members. The
state of a population is a distribution of strategic types. The same
set of types can support many different models, since changing the
matching process or the rule for converting payoffs to growth can
change the resulting dynamics. This chapter isolates the distribution
and payoff operations used in the pairwise, randomly matched model
\cite[Chs.~1--2]{alexander}.
\section{Distributions and relabeling}
For a nonempty finite set \(I\) of types, the population state space is
the simplex
\[
\Delta(I)=\left\{x\in\mathbb R^I:
x_i\geq0,\quad \sum_{i\in I}x_i=1\right\}.
\]
Write \(\delta_i\) for the state concentrated on \(i\).
The support of \(x\) is the set of its positive coordinates.
For a nonempty subset \(S\subseteq I\), the inclusion
\(\Delta(S)\to\Delta(I)\) inserts zero in the omitted coordinates.
Its image is a face, and its relative interior consists precisely
of the states of support \(S\). The affine hull of the full simplex
is the hyperplane \(\sum_i x_i=1\), whose translation space is
\(T_I=\{v:\sum_i v_i=0\}\). Feasible directions at a boundary state
are those \(v\in T_I\) with \(v_i\geq0\) whenever \(x_i=0\).
Indeed, this is necessary for \(x+t v\) to have nonnegative
coordinates at positive \(t\). It is sufficient for small \(t\)
because only finitely many of the positive coordinates can decrease.
A map of type sets \(f:I\to J\) pushes a distribution forward by
\[
(f_*x)_j=\sum_{f(i)=j}x_i.
\]
Its linear extension uses the same formula on all vectors.
If \(g:J\to K\), collecting the terms in the sums indexed by each
fiber of \(g f\) gives \((gf)_*=g_*f_*\). The identity map gives
the identity on distributions. Thus simplex formation is a functor
from finite sets to convex sets and affine maps. It extends on all
sets to \(\mathsf D I\), the set of finitely supported probability
distributions. A map on Dirac states extends uniquely to an affine
map out of \(\mathsf D I\), since every element is a finite
combination \(\sum_i x_i\delta_i\) and its coefficient function
is uniquely determined by \(x\). Accordingly a stochastic
assignment of an output distribution to each type is determined
by its values on types and then acts affinely on populations.
There is also a canonical way to combine mixtures of populations.
For \(w\in\mathsf D\mathsf D I\), its barycenter has coordinates
\(\mu_I(w)_i=\sum_x w_x x_i\). Taking the barycenter of a Dirac
distribution on a population returns that population. Mixing
Dirac distributions on individual types returns their original
coefficient distribution. The two orders of flattening three
layers of mixtures agree because in either order the coefficient
of \(i\) is obtained by adding the same products of three weights.
This gives the unit and multiplication of the finite-distribution
monad. The construction uses only finite sums even when \(I\)
is infinite. It differs from fixing a population size, since
the weights may be any nonnegative reals adding to one.
\section{Random matching and expected fitness}
A symmetric two-player game on \(I\) assigns the row player's
payoff \(A_{ij}\) when the row type is \(i\) and its opponent is
\(j\). The opponent receives \(A_{ji}\) in that encounter.
Symmetry is the exchange of player roles in this description.
No symmetry of the entries of \(A\) is required.
If the players draw types independently with distributions \(p,q\),
the expected payoff is
\[
u_A(p,q)=\sum_{i,j}p_i A_{ij}q_j=p^{\mathsf T}Aq .
\]
Its two separate affine laws follow by collecting the coefficients
of the product distribution. In particular, a randomly sampled
opponent from a population \(x\) gives the fitness vector \(Ax\),
while the population average is \(\overline u_A(x)=x^{\mathsf T}Ax\).
A correlated matching prescription instead supplies a conditional
distribution of opponents for each row type. If these distributions
are \(q(i,x)\), the corresponding fitness is
\(F_i(x)=\sum_j A_{ij}q(i,x)_j\). General matching rules
therefore need not produce a linear fitness map, even when the
encounter payoffs are fixed.
The pairing in \(u_A\) is functorial with respect to probability
aggregation. It uses the transpose of the aggregation map to
transport a score. For a score \(b\in\mathbb R^J\), define
\((f^*b)_i=b_{f(i)}\). Then
\[
\sum_i x_i(f^*b)_i=\sum_j (f_*x)_j b_j.
\]
This follows by collecting the left-hand terms over each fiber of
\(f\). The formula distinguishes the direction in which
population weights move from the direction in which a fitness
score is read back.
\section{Exact maps between games}
Consider the category whose objects are pairs \((I,A)\) and whose
maps \((I,A)\to(J,B)\) are type maps \(f:I\to J\) satisfying
\(A_{ij}=B_{f(i),f(j)}\) for every \(i,j\).
Identity maps satisfy the same equation. If \(f\) and \(g\) are
payoff-preserving, the comparison for \(gf\) follows by applying
the two equalities successively. This category describes exact
games with strategy maps. There are useful coarser notions of
comparison, including maps that preserve only payoff
differences, developed in Chapter~\ref{ch:payoff-equivalence}.
\begin{proposition}[Fitness under exact aggregation]
\label{prop:exact-fitness}
If \(f:(I,A)\to(J,B)\) is payoff-preserving, then
\[
Ax=f^*(Bf_*x),\qquad
u_A(p,q)=u_B(f_*p,f_*q).
\]
Conversely, either the fitness identity at all population states
or the payoff identity at all pairs of distributions implies
payoff preservation.
\end{proposition}
\begin{proof}
Regroup the terms of \((Ax)_i=\sum_k B_{f(i),f(k)}x_k\)
over the fibers of \(f\). This gives the claimed fitness identity.
The identity for payoffs then follows from the pairing between
aggregation and score pullback. Conversely, evaluating the first
identity at \(\delta_j\) gives equality in each component \(i\).
Evaluating the second identity at \((\delta_i,\delta_j)\) also
gives equality of entries.
\end{proof}
\begin{figure}[htbp]
\centering
\[
\begin{tikzcd}[column sep=3.2em,row sep=2.4em]
\Delta(I)\times\Delta(I)
\arrow[r,"f_*\times f_*"]\arrow[d,"u_A"']
& \Delta(J)\times\Delta(J)\arrow[d,"u_B"]\\
\mathbb R\arrow[r,equal]&\mathbb R
\end{tikzcd}
\]
\caption{An exact type map commutes with expected payoff.}
\label{fig:exact-payoff-map}
\end{figure}
A type bijection satisfying the comparison is a change of
labels. A noninjective type map can identify distinct population
states, while a nonsurjective map may leave types of the target
unrepresented. If all target types have preimages, every
target state can be represented by lifting its mass to selected
types above its support. Such a lift need not be unique.
Expected payoff agrees for all lifts by
Proposition~\ref{prop:exact-fitness}. A selection rule using
the labels of the lifts can still see the difference. Passing
to aggregated population states must therefore be justified for
the particular rules one wants to transport.
\section{Parameterized populations}
Suppose payoffs vary continuously with an environmental
parameter \(s\) in a space \(S\), giving an array \(A(s)\).
Evaluation \((s,p,q)\mapsto u_{A(s)}(p,q)\) is continuous
by its finite-sum formula. Substituting a map \(r:S'\to S\)
pulls back the family to \(A(r(s'))\), and evaluations agree
after the same substitution. If local descriptions use different
names for the types, a transition is a bijection of the
corresponding type sets. An exact transition then gives the
comparison in Figure~\ref{fig:exact-payoff-map}, and these
comparisons compose because pushforward does. The simplex
may consequently vary as a bundle with transition maps
permuting its coordinates. Expected payoff is then a single
continuous function on pairs of states over the same parameter,
as long as the local payoff matrices obey the transition
comparisons. The groupoid of labels keeps track of both the
state and its change of presentation.
This setup will support two further comparisons. Static
stability asks which distributions do well against populations
with a small mutant component. A dynamics asks where
trajectories go after an actual change in the population.
The payoff operation is common to the two questions, and
additional assumptions on the trajectory rule will enter later.
The exact payoff condition is closed under
pullback of the type labels. Given \(B\) on \(J\)
and any \(f:I\to J\), there is a unique matrix on
\(I\) for which \(f\) is exact. Its entries are
\(B_{f(i),f(j)}\). Pulling back twice gives the same
matrix as pulling back along the composite. Thus the
payoff spaces form a contravariant system over type
sets, whereas the population spaces form a covariant
system. Their comparison in the payoff square explains
why treating both as covariant under a quotient of
labels can change the model.
\par
\chapter{Evolutionary stability and invasion}
\label{ch:ess}
Fix a population simplex \(\Delta(I)\) and an expected payoff
\(u_A\) as in Chapter~\ref{ch:population}. We use the symmetric
Nash condition
\[
u_A(p,p)\geq u_A(q,p)\qquad(q\in\Delta(I)).
\]
It says that no alternative earns more against \(p\) than a
resident earns against \(p\). An evolutionary stability
test also asks what happens when the alternative changes
the population against which these payoffs are evaluated.
Maynard Smith and Price introduced the stability idea in
the study of animal conflict \cite{maynard-price}. Alexander
develops its different formulations in \cite[Ch.~2]{alexander}.
\section{Best replies and the resident population}
Let \(M_p=\max_i (Ap)_i\). The value \(u_A(p,p)\) is the
\(p\)-weighted average of the numbers \((Ap)_i\). A pure maximizer
earns \(M_p\) against \(p\), while the average cannot exceed
\(M_p\). Hence \(p\) satisfies the symmetric Nash condition
if and only if \(u_A(p,p)=M_p\). Equivalently, every type in
the support of \(p\) attains \(M_p\). If any type with
positive weight earned less, the weighted average would be
strictly smaller than \(M_p\).
This calculation also identifies the complete best-reply
correspondence at \(p\). It is the face spanned by those pure
types at which \(Ap\) reaches its maximum. Strict symmetric
Nash means all other population states \(q\ne p\) earn less
against \(p\). In the mixed strategy simplex this forces
the best-reply face to consist of a single point. In
particular, a resident supported on several types can
satisfy the Nash condition without satisfying its strict
version.
\section{The invasion test and the two inequalities}
For a resident \(p\), a competitor \(q\ne p\), and a mutant
share \(0<\varepsilon<1\), put
\[
x_{\varepsilon}=(1-\varepsilon)p+\varepsilon q.
\]
The pair \(p,q\) may consist of mixed strategies. The
parameter \(\varepsilon\) specifies the share of the
competitor in the population. It does not describe the
probabilities within either of those two strategies.
Define the payoff advantage in the resulting population by
\[
D_{p,q}(\varepsilon)
=u_A(p,x_{\varepsilon})-u_A(q,x_{\varepsilon}).
\]
The resident \(p\) is evolutionarily stable if for each
\(q\ne p\) there exists a threshold \(b(q)>0\) such that
\(D_{p,q}(\varepsilon)>0\) whenever
\(0<\varepsilon<\min\{1,b(q)\}\).
The threshold in this definition can depend on \(q\).
A threshold valid for all competitors is a different
quantifier placement, addressed in
Chapter~\ref{ch:superiority}.
\begin{theorem}[Invasion criterion for pairwise games]
\label{thm:invasion-criterion}
A resident \(p\) is evolutionarily stable if and only
if it is symmetric Nash and, for each \(q\ne p\) with
\(u_A(p,p)=u_A(q,p)\), one has
\[
u_A(p,q)>u_A(q,q).
\]
\end{theorem}
\begin{proof}
For each fixed competitor write
\(d_0=u_A(p,p)-u_A(q,p)\) and
\(d_1=u_A(p,q)-u_A(q,q)\). The affine law in the opponent
argument gives
\[
D_{p,q}(\varepsilon)=(1-\varepsilon)d_0+\varepsilon d_1 .
\]
If this is positive on a sufficiently small interval of
positive \(\varepsilon\), continuity gives \(d_0\geq0\).
When \(d_0=0\), the same interval condition requires
\(d_1>0\) by the displayed identity. Conversely, if
\(d_0>0\), the advantage remains positive for all
sufficiently small \(\varepsilon\) by continuity. If
\(d_0=0\) and \(d_1>0\), the advantage is
\(\varepsilon d_1>0\) throughout the range of positive
mutant shares. Applying the argument to every \(q\ne p\)
gives the equivalence.
\end{proof}
The two terms are evaluated in different environments.
The first evaluates both strategies against a purely
resident population. The second evaluates both against
the competing strategy. In the event of a first-order
tie, the strictly positive second comparison explains
why the resident gains a strict advantage when a
positive mutant share is present. Every strict symmetric
Nash equilibrium is evolutionarily stable, since none
of its competitors ties in the first comparison.
For a general continuous fitness map the first comparison
still gives a necessary condition for positive advantage
near zero mutant share. The exact tie test in
Theorem~\ref{thm:invasion-criterion} uses affine dependence
on the population. With nonlinear dependence, the advantage
along the segment can have a nontrivial shape. An endpoint
comparison then requires additional hypotheses before it
determines the sign near the resident.
\section{Neutral stability and sets of solutions}
Neutral stability uses the weak inequality
\(D_{p,q}(\varepsilon)\geq0\) in the invasion test. This
standard variant allows a competing strategy to match the
resident's score when its share is small
\cite[Ch.~2]{weibull}. The same proof, with a weak inequality
on the tie condition, shows that \(p\) is neutrally stable
exactly when it is symmetric Nash and \(d_1\geq0\) for each
distinct \(q\) with \(d_0=0\). Thus the solution predicates
satisfy
\[
\operatorname{ESS}(A)\subseteq
\operatorname{NS}(A)\subseteq
\operatorname{Nash}(A).
\]
These are subsets of the state space. An equality between
local payoffs in the invasion test need not select one
of the tied states. Persistence or attraction of the
resident depends on the dynamics chosen to resolve or
retain such ties.
The symmetric Nash condition defines a closed relation
between payoff matrices and population states. Indeed,
for each fixed pure type \(i\) it is the nonnegativity
of the continuous difference \(u_A(p,p)-(Ap)_i\), and
the pure inequalities imply the conditions for all
mixtures by averaging. An evolutionary stability
relation adds a conditional strict inequality. For
instance, along the parameter ray \(A(t)=tA\) for
\(t>0\), all comparisons in
Theorem~\ref{thm:invasion-criterion} retain their sign.
At \(t=0\) all comparisons vanish. If the simplex has
more than one point, every resident is then neutral and
no resident is strictly evolutionarily stable. Whenever
the positive ray has an ESS this supplies a limiting
failure of strict stability. Thus a family of games
with a continuous payoff map may have different solution
fibers at limiting parameter values.
\section{Transporting the comparison}
A label change transports all these predicates. To see
this, apply a payoff-preserving bijection \(f:(I,A)\to(J,B)\).
The map \(f_*\) respects the mixture \(x_\varepsilon\) by
affinity and preserves its payoff by
Proposition~\ref{prop:exact-fitness}. It also gives a
bijection between the competitors of \(p\) and of \(f_*p\).
The inequalities and their quantifiers therefore agree.
For a family with label transitions the stable locus
can be described on each chart and glued as a subset
of the simplex family by this argument. It retains
the topology inherited from that family when the
payoffs vary in a topological base.
For a payoff-preserving surjection \(f\), the Nash
predicate obeys the same comparison, since every
competitor of \(f_*p\) can be lifted. Strict stability
has an additional obstruction. If \(q\ne p\) has
\(f_*q=f_*p\), the payoff identity makes
\(D_{p,q}(\varepsilon)=0\) for every share. The lifted
matrix can thus preserve all coarse payoff information
and have additional tied alternatives upstairs. This
is one reason to record the class of game maps before
using it to compare solution sets.
\section{A population formulation of best replies}
The Nash relation has a useful description by the subset
of types with maximal score. Let \(B\subseteq I\) be
nonempty, and consider the part of the simplex where
\(B_p=B\). On this part, belonging to the best-reply
face is equivalent to the coordinate condition \(p_i=0\)
for all \(i\notin B\). The region \(B_p=B\) is specified
by equality of the scores on \(B\) and strict inequalities
between the common score and each score outside \(B\).
For a fixed matrix these are linear equality and
inequality constraints on \(p\). Thus the support
condition can be checked on these regions without
a search over all mixed competitors. Evolutionary
stability requires a further comparison on the face
of tied competitors. That is a quadratic condition
and need not reduce to checking only its vertices.
If \(p\) is strictly Nash, its payoff gap on the region
of competitors at distance at least \(r>0\) from \(p\)
has a positive minimum whenever that region is nonempty.
This follows from continuity on a closed subset of the
compact simplex. There is generally no positive minimum
over every distinct competitor, since a sequence of
competitors can approach \(p\). The existence of a uniform
invasion barrier is instead a uniform statement about
the relative contributions of the first and second
payoff differences. The next chapter establishes it by
examining the directions in which the first difference
vanishes.
\par
\chapter{Local superiority and mixed strategies}
\label{ch:superiority}
The invasion test fixes a competitor and then varies its share.
Local superiority compares the resident with every nearby
population against that population itself. In finite-dimensional
matrix games, critical directions at the resident relate these
two descriptions. They also isolate the part of the payoff matrix
that controls stability of a mixed resident. The equivalence with
a uniform invasion barrier is discussed in
\cite[Ch.~2]{alexander} and \cite[Ch.~2]{weibull}.
\section{The critical cone}
Write \(T_I=\{v:\sum_i v_i=0\}\). For \(p\in\Delta(I)\)
let \(C_p=\{v\in T_I:v_i\geq0\text{ if }p_i=0\}\).
Every displacement \(q-p\) belongs to \(C_p\), and every
\(v\in C_p\) is an actual displacement up to multiplication by
a small positive scalar. Introduce the linear and quadratic forms
\[
L_p(v)=-v^{\mathsf T}Ap,\qquad Q_A(v)=v^{\mathsf T}Av
\]
and the critical cone \(K_p=C_p\cap\ker L_p\).
The Nash condition is exactly \(L_p\geq0\) on \(C_p\).
In one direction one scales each feasible vector to an actual
displacement and applies Nash. In the other direction one
applies the inequality to \(q-p\) for arbitrary \(q\).
If \(q=p+v\), the two terms in the invasion criterion are
\(d_0=L_p(v)\) and \(d_1=L_p(v)-Q_A(v)\).
It follows from Theorem~\ref{thm:invasion-criterion} and
scaling of the quadratic form that
\[
p\text{ is an ESS}
\quad\iff\quad L_p|_{C_p}\geq0,\qquad
Q_A(v)<0\quad(0\ne v\in K_p).
\]
The critical cone consists of directions with no initial loss
against the resident. The quadratic condition applies only
on that cone. At a pure resident with a unique best response,
for example, there is no nonzero critical direction.
At a boundary resident with additional tied best responses,
the cone can include directions that add previously absent
types. Restricting the calculation to the resident's support
can omit those directions.
\section{Uniform superiority}
Call \(p\) locally superior if \(u_A(p,x)>u_A(x,x)\)
for every \(x\ne p\) sufficiently near \(p\) in the simplex.
A uniform invasion barrier is one positive threshold that
works for all distinct competitors in the definition of
evolutionary stability.
\begin{theorem}
\label{thm:uniform-ess}
For a matrix game on a finite type set, evolutionary
stability, local superiority, and the existence of a
uniform invasion barrier are equivalent.
\end{theorem}
\begin{proof}
Suppose \(p\) is an ESS. For \(v=x-p\) the superiority
difference is
\[
u_A(p,x)-u_A(x,x)=L_p(v)-Q_A(v).
\]
If there is no nonzero feasible direction, the local
condition is vacuous. Otherwise put
\(S=C_p\cap\{v:\|v\|=1\}\), a compact set. On its
critical subset \(S\cap K_p\), the quadratic form is
negative. The relatively open subset of \(S\) on which
\(Q_A<0\) contains this critical set. On its complement,
if nonempty, \(L_p\) has a strictly positive minimum \(m\)
by compactness and the fact that its zeros are critical.
Also \(|Q_A|\) has a finite maximum \(M\) on \(S\).
For \(w\in S\) and \(r>0\) we have
\[
L_p(rw)-Q_A(rw)=r\bigl(L_p(w)-rQ_A(w)\bigr).
\]
It is positive wherever \(Q_A(w)<0\) because \(L_p\geq0\).
On the complementary subset it is positive for
\(0<r<m/(1+M)\). This proves local superiority, including
the case in which the complementary subset is empty.
Now assume local superiority on a neighborhood of radius
\(\delta>0\). The simplex has finite diameter \(D\).
For each mutant \(q\ne p\), the invader mixture
\(x_\varepsilon=(1-\varepsilon)p+\varepsilon q\) satisfies
\(0<\|x_\varepsilon-p\|\leq\varepsilon D\). The identity
\[
u_A(p,x_\varepsilon)-u_A(x_\varepsilon,x_\varepsilon)
=\varepsilon
\bigl(u_A(p,x_\varepsilon)-u_A(q,x_\varepsilon)\bigr)
\]
follows from affinity in the first argument. Hence every
share below \(\min\{1,\delta/(1+D)\}\) gives positive
invasion advantage independently of \(q\). This yields
a uniform invasion barrier. A uniform threshold gives
the pointwise thresholds of evolutionary stability
immediately.
\end{proof}
Two features of the proof have distinct roles. Compactness
of the unit directions controls approach to the critical
cone. Boundedness of the simplex controls the passage from
nearby populations to a common mutant share for every
\(q\). General strategy spaces can lose either property.
The finite-dimensional proof cannot then be transferred
without a replacement estimate.
\section{Interior residents and the quadratic test}
In the relative interior of \(\Delta(I)\), the cone \(C_p\)
is the whole tangent space \(T_I\). A linear form nonnegative
on a vector space is zero. The Nash condition therefore says
that \(Ap\) is constant in the type index. For an interior
Nash resident, \(K_p=T_I\), and evolutionary stability is
equivalent to the restriction of the symmetric part
\((A+A^{\mathsf T})/2\) being negative definite on \(T_I\).
Indeed \(v^{\mathsf T}Av\) only sees that symmetric part.
When \(A=A^{\mathsf T}\), this is the Hessian test for the
quadratic function \(x\mapsto x^{\mathsf T}Ax/2\) restricted
to the affine hull of the simplex. Its differential at a
Nash state vanishes along \(T_I\), and its exact expansion
has quadratic term \(Q_A/2\). When \(A\) is not symmetric,
the argument establishes the sign of the critical quadratic
form. The derivative of this same quadratic function
contains the symmetric part and in general differs from
the fitness vector \(Ax\).
Negative definiteness on \(T_I\) also implies uniqueness of
a symmetric Nash state whenever one exists. If \(p,q\)
both satisfy Nash, adding \(u_A(p,p)\geq u_A(q,p)\)
and \(u_A(q,q)\geq u_A(p,q)\) gives
\((p-q)^{\mathsf T}A(p-q)\geq0\). Since \(p-q\in T_I\),
this forces equality of the states under negative definiteness.
\section{Persistence in payoff families}
The interior test has a direct parameter consequence.
Let \(\Pi\) be the Euclidean projection onto \(T_I\).
If \(Q_A<0\) on \(T_I\setminus\{0\}\), the operator
\(\Pi A|_{T_I}\) is invertible. Indeed, a vector \(v\)
in its kernel satisfies \(v^{\mathsf T}Av=0\).
Injectivity then gives invertibility on the
finite-dimensional space \(T_I\).
For any fixed \(b\) in the affine hull of the simplex,
the unique solution of \(\Pi A p=0\) in that affine hull
is
\[
p_A=b-(\Pi A|_{T_I})^{-1}\Pi A b .
\]
The expression varies continuously with \(A\) near a
matrix for which the operator is invertible.
Strict positivity of each coordinate is open. Negative
definiteness is also open, by taking the maximum of
\(Q_A\) over the compact unit sphere in \(T_I\)
when \(T_I\ne0\). Thus an interior ESS satisfying
this test persists through sufficiently small payoff
perturbations, and in that neighborhood the displayed
formula gives the unique Nash state. Relabeling commutes
with the zero-mean projection, and uniqueness makes
these locally computed states agree wherever the labels
change. This is an explicit instance of transport of a
solution in a parameter family.
\section{A local margin and continuation on a face}
The compact-direction argument yields a useful quantitative
version. If \(p\) is an ESS and the feasible unit set is
nonempty, there are constants \(c,\delta>0\) such that
\[
u_A(p,x)-u_A(x,x)\ \geq\ c\|x-p\|^2
\qquad(\|x-p\|<\delta).
\]
Indeed, if the critical unit set is nonempty, choose \(a>0\)
with \(Q_A\leq-a\) there. On the directions where
\(Q_A<-a/2\), the difference at \(v=rw\) is at least
\((a/2)r^2\). On the closed complement the minimum of
\(L_p\) is some \(m>0\). For
\(r<\min\{1,m/(2(1+M))\}\), the difference is at least
\((m/2)r\geq(m/2)r^2\). If either region is empty only
the other bound is needed. When the critical unit set
is empty the second estimate applies on all directions.
The lower bound is an estimate on payoffs. Its use
as an estimate on motion requires a dynamics whose
drift is related to those payoffs.
At a Nash point, let \(B_p=\{i:(Ap)_i=M_p\}\), where
\(M_p=\max_i(Ap)_i\). Since \(\sum_i v_i=0\), one has
\[
L_p(v)=\sum_{i\notin B_p}\bigl(M_p-(Ap)_i\bigr)v_i .
\]
All terms on the right are nonnegative for feasible
directions because every omitted index has zero weight
in \(p\). Consequently \(v\in K_p\) precisely when
\(v\in C_p\) and \(v_i=0\) outside \(B_p\).
If every absent type has a strictly lower payoff,
then \(B_p=\operatorname{supp}(p)=S\). The critical
cone is simply the tangent space to \(\Delta(S)\).
An ESS of this kind with a strict quadratic test on that
tangent space persists on the same face under small
matrix perturbations: solve the interior equation for
the restricted matrix, extend by zero, and use continuity
to preserve the payoff gaps to all absent types.
Any Nash state sufficiently near \(p\) for a sufficiently
small perturbation still has positive coordinates on
\(S\), and the same payoff gaps prevent extra positive
coordinates outside \(S\). Its uniqueness therefore follows
from the restricted affine equation. Here the strict gaps
and the curvature on the face control different aspects of
the same continuation.
\par
\chapter{Payoff equivalence and spaces of games}
\label{ch:payoff-equivalence}
The payoff array contains information about the absolute score
of each encounter. The static comparisons in Chapter~\ref{ch:ess}
and the usual replicator field use only relative scores against
the same population. This difference suggests an equivalence
relation on games with fixed type labels. Once labels are also
allowed to vary, the resulting space of games has a groupoid
presentation. Its arrows identify numerical presentations of
the same relative-fitness model.
\section{The quotient by common fitness increments}
Fix a nonempty finite type set \(I\), put \(n=|I|\), and
write \(\mathbf 1\) for its constant vector. Let
\[
\mathcal A_I=\operatorname{End}(\mathbb R^I),
\qquad W_I=\{\mathbf 1 b^{\mathsf T}:b\in\mathbb R^I\}.
\]
If \(B=A+\mathbf 1 b^{\mathsf T}\), then
\[
Bx=Ax+(b^{\mathsf T}x)\mathbf 1 .
\]
The increment is common to every row type, though its
size can depend on the opponent distribution \(x\).
For \(p,q,x\) in the simplex we have
\((p-q)^{\mathsf T}(B-A)x=0\), since \(p,q\) have the
same total mass. Thus comparing \(p\) and \(q\) against
\(x\) gives identical advantage in the two games.
This includes both comparisons in the ESS test.
Define the zero-mean projection
\(\Pi_I v=v-\mathbf 1(\sum_i v_i)/n\). Its kernel
is the span of \(\mathbf 1\) and its image is \(T_I\).
The map \(A\mapsto\Pi_I A\) has kernel \(W_I\):
each column is killed precisely when that column is
constant across the rows. It maps onto the linear maps
\(\mathbb R^I\to T_I\), since the projection acts
as the identity on their image. It follows that
\[
\mathcal Q_I=\mathcal A_I/W_I
\simeq\operatorname{Hom}(\mathbb R^I,T_I).
\]
Each class has a canonical representative with zero
column sums. Alternatively, knowing the relative scores
against every population state determines this class.
Indeed the relative scores give all row differences
in \((A-B)x\). If these vanish for all \(x\), evaluation
at each pure state shows \(A-B\in W_I\).
\begin{proposition}[The quotient determined by replicator motion]
\label{prop:replicator-payoff-quotient}
Write \(X_A\) for the polynomial vector field on the
simplex given by
\[
(X_A(x))_i=x_i\bigl((Ax)_i-x^{\mathsf T}Ax\bigr).
\]
Then \(X_A=X_B\) on the whole simplex if and only if
\(A-B\in W_I\). If \(B=\lambda A+\mathbf 1b^{\mathsf T}\)
for constant \(\lambda>0\), then \(X_B=\lambda X_A\)
and the static stability predicates agree.
\end{proposition}
\begin{proof}
The common fitness increment cancels in the expression
for \(X_A\) because the weights of \(x\) add to one.
Scaling every fitness by \(\lambda\) scales both terms
in the difference and every payoff comparison by the
same positive number. This proves the forward formulas.
Conversely, take \(x\) in the relative interior of the
simplex. Each \(x_i>0\), so equality of the fields
implies \(((A-B)x)_i=x^{\mathsf T}(A-B)x\) for all \(i\).
Thus \(\Pi_I(A-B)x=0\) on the relative interior.
Continuity gives zero on its closure, and in particular
on every pure state. The column argument then places
\(A-B\) in \(W_I\).
\end{proof}
If a solution curve for \(X_A\) is parameterized by
\(t\), changing its parameter to \(\lambda t\) gives a
solution for \(X_B\) on the corresponding time interval.
Positive scaling therefore gives a comparison of time
units. Keeping time fixed gives the sharper quotient
\(\mathcal Q_I\). Neither equivalence identifies the
unadjusted absolute payoffs. Those retain a role if
another process, such as a baseline contribution to
fitness, is sensitive to their absolute levels.
\section{Relabeling as a groupoid action}
Let \(\mathfrak S_I\) be the permutation group of the
type set. A permutation \(\sigma\) acts by
\(A\mapsto P_\sigma A P_\sigma^{-1}\), where
\(P_\sigma\delta_i=\delta_{\sigma(i)}\).
It preserves \(W_I\), since it fixes \(\mathbf 1\).
It also commutes with \(\Pi_I\), since it preserves
the sum of the coordinates. The action therefore
passes to \(\mathcal Q_I\). Its action groupoid has
objects \(q\in\mathcal Q_I\) and arrows
\(q\to q'\) given by permutations \(\sigma\) for
which \(q'=\sigma q\). Two matrices in the same orbit
determine isomorphic objects. The permutations fixing
a class are the automorphisms of that object. Passing
to the set of orbits forgets this group. Accordingly,
the orbit set classifies a single game up to relabeling,
while the groupoid can also record changes of
presentation along a parameter space.
One can equivalently keep the raw matrices and allow
translation by \(W_I\) in the arrows. The group
\(W_I\rtimes\mathfrak S_I\) acts on \(\mathcal A_I\)
by \((w,\sigma)A=\sigma A+w\). Its multiplication
uses the evident permutation action on \(W_I\).
Consider the functor of action groupoids
\[
\mathcal A_I//(W_I\rtimes\mathfrak S_I)
\longrightarrow
\mathcal Q_I//\mathfrak S_I
\]
sending \(A\) to its class and an arrow \((w,\sigma)\)
to \(\sigma\). It is fully faithful: if the classes of
\(A,B\) satisfy \([B]=\sigma[A]\), then
\(w=B-\sigma A\) belongs to \(W_I\) and is the unique
translation giving an arrow above \(\sigma\). It is
essentially surjective by the canonical normalized
representative \(\Pi_I A\) in each class. A quasi-inverse
uses this representative, sending each permutation to an
arrow with translation zero. The comparison back to a
raw matrix is translation by its difference from its
normalized form. The formulas are continuous and even
smooth as functions of the matrices. Consequently they
also give equivalent presentations for families with
continuous or smooth transition data. Here the removal
of redundant payoff coordinates gives an equivalence of
groupoids because the arrows over each permutation are
recovered uniquely.
\begin{figure}[htbp]
\centering
\[
\begin{tikzcd}[column sep=3em,row sep=2.4em]
\mathcal A_I\arrow[r,"\sigma"]\arrow[d,"\Pi_I(-)"']
&\mathcal A_I\arrow[d,"\Pi_I(-)"]\\
\mathcal Q_I\arrow[r,"\sigma"]&\mathcal Q_I
\end{tikzcd}
\]
\caption{Canonical normalization respects every permutation of labels.}
\label{fig:payoff-normalization}
\end{figure}
\section{Aggregation and local labels}
For a parameter base \(S\), a family with local labels
can be given on an open cover by maps to \(\mathcal Q_I\)
along with permutations on overlaps satisfying the
cocycle law. The comparison on overlaps is the one
in Figure~\ref{fig:payoff-normalization}, and population
states transform by the same permutations. Thus the
payoff family and population bundle can carry
nontrivial label transport. For example, a loop can
act by a permutation fixing the payoff class and
moving some points of the simplex. A solution family
that is required to be a section must respect that
transport. A numerical solution set at a single
parameter point does not specify how to continue the
individual solutions around such a loop.
A payoff-preserving surjection of type sets is a
different operation. It can remove duplicate labels by
summing their population weights. Its fibers over
interior mass assignments can have positive dimension.
It therefore fails to identify the underlying state
spaces even when the expected payoff descends. In
particular, if a state puts positive mass above a
target type that has at least two preimages, a nonzero
transfer of mass within that fiber leaves its image
unchanged. Chapter~\ref{ch:ess} shows that such a transfer
prevents the specified lift from being an ESS. The
groupoid equivalence for normalization uses actual
comparisons with inverses. A coarse aggregation needs
a separate argument for each invariant transported
along it.
\section{Games with selected residents}
The total solution space is more informative for families
than a list of stable residents for each matrix. For the
ESS predicate, form the subspace
\[
\mathcal S_I=
\{(q,p)\in\mathcal Q_I\times\Delta(I):
p\text{ is evolutionarily stable at }q\}.
\]
The definition is independent of the representative of \(q\).
Permuting the game and the resident together preserves this
subspace, by the payoff computation in
Chapter~\ref{ch:ess}. Projection therefore gives a functor
\(\mathcal S_I//\mathfrak S_I\to
\mathcal Q_I//\mathfrak S_I\). In a parameter family, a
selected resident is a lift along this projection.
In local coordinates it consists of a choice on each chart
and compatibility of the choices on every overlap.
Over a chosen game class with a specified identification of
its labels, the stable residents form the ordinary set
\(\operatorname{ESS}(q)\). If one instead restricts to the
full subgroupoid of games isomorphic to \(q\), then the
category of marked residents is equivalent to the action
groupoid of \(\operatorname{Stab}(q)\) on
\(\operatorname{ESS}(q)\). Indeed, bringing a game in that
orbit back to \(q\) takes its resident to one at \(q\),
and different choices of a permutation differ by a
stabilizer element. The groupoid records how residents can
be identified by changes of labels. This construction only
uses the stated stability predicate and its invariance
under permutations. It applies equally to the Nash and
neutral predicates.
\par
\bookpart{II}{Dynamics on population states}
\chapter{Replicator flows and mutation}
\label{ch:replicator}
\section{From abundance to proportion}
Let \(S\) be a nonempty finite strategy set and write \(\Delta(S)\)
for its probability simplex. A fitness assignment is a map
\(F:\Delta(S)\to\mathbb R^S\). Its \(i\)-th entry is the rate
associated with strategy \(i\), evaluated in the population \(x\).
For a matrix \(A\), the assignment \(F(x)=Ax\) evaluates the
expected payoff of each strategy against the population. The
derivation below applies to nonaffine assignments as well.
Taylor and Jonker give an early dynamical treatment, and
Alexander explains the abundance and revision interpretations
of the equation \cite{taylor-jonker,alexander}.
Suppose \(n_i\) is the abundance of strategy \(i\) and that
\(\dot n_i=n_iF_i(x)\), where \(x_i=n_i/\sum_j n_j\).
Writing \(M=\sum_j n_j\) and \(\phi(x)=\sum_jx_j F_j(x)\),
one obtains \(\dot M=M\phi(x)\). The quotient rule gives
\begin{equation}
\dot x_i = x_i\bigl(F_i(x)-\phi(x)\bigr).
\label{eq:replicator}
\end{equation}
The subtraction of mean fitness is thus the correction due to
normalization. Conversely, if \(x\) solves \eqref{eq:replicator},
then \(M(t)=M(0)\exp(\int_0^t\phi(x(s))\,ds)\) recovers a
positive total abundance and \(n_i=Mx_i\) solves the abundance
equation. The abundance formulation uses independence of the
fitness assignment from the total. If the rates depend on total
abundance, the proportions generally retain that dependence
and need an additional state variable.
There are two simple symmetries of \eqref{eq:replicator}.
Adding \(c(x)\) to every \(F_i(x)\) adds \(c(x)\) to \(\phi(x)\)
and leaves the field fixed. Relabeling \(S\) transports \(x\)
and \(F\) together. For an affine assignment, adding the same
row vector to each row of the payoff matrix is one instance
of the first symmetry. Equality of vector fields records a
specific observational equivalence of fitness data.
\section{The simplex and its faces}
Assume the fitness assignment extends to a locally Lipschitz
map on a neighborhood of \(\Delta(S)\). On that neighborhood
form \(\phi(x)=\sum_j x_j F_j(x)\) without requiring \(\sum_jx_j=1\).
For a local solution \(x\) set \(m(t)=\sum_jx_j(t)\).
Then \(\dot m=(1-m)\phi(x)\). If \(m(0)=1\), the integrating
factor for this scalar equation implies \(m(t)=1\) throughout
its interval. Independently, each coordinate satisfies
\begin{equation}
x_i(t)=x_i(0)\exp\left(
\int_0^t\bigl(F_i(x(s))-\phi(x(s))\bigr)\,ds
\right).
\label{eq:coordinate-factor}
\end{equation}
This identity includes coordinates starting at zero, because
the scalar linear equation along a given solution has the
zero solution for such an initial value.
\begin{proposition}[Existence and restriction to faces]
For the locally Lipschitz assignment just specified,
\eqref{eq:replicator} has a unique solution for every initial
state of \(\Delta(S)\), defined for all real times. Throughout
any finite time interval its support equals the initial
support. On the face indexed by a nonempty subset \(T\subseteq S\),
the solution is the replicator solution for the restricted
assignment on \(\Delta(T)\).
\end{proposition}
\begin{proof}
Local existence and uniqueness follow from the locally
Lipschitz field. Conservation of mass and
\eqref{eq:coordinate-factor} keep a solution in the simplex
throughout its interval of existence, in either direction
of time. A solution confined to this compact subset of the
domain can be continued at each finite endpoint. Support
preservation follows from the positivity of the exponential.
On the face the terms \(x_jF_j(x)\) with \(j\notin T\) vanish
from the mean, and each missing coordinate has zero velocity.
Uniqueness identifies the restricted trajectory with the
solution calculated directly in the face.
\end{proof}
In particular, if \(j_T:\Delta(T)\to\Delta(S)\) extends a
state by zero, and \(F_T\) restricts \(F\circ j_T\) to the
coordinates in \(T\), the solution operators obey the
commutative square
\[
\begin{tikzcd}[column sep=large]
\Delta(T)\arrow[r,"\Phi_t^{F_T}"]\arrow[d,hook] &
\Delta(T)\arrow[d,hook]\\
\Delta(S)\arrow[r,"\Phi_t^F"] & \Delta(S).
\end{tikzcd}
\]
Here both vertical maps are \(j_T\). The group law
\(\Phi_{s+t}=\Phi_s\circ\Phi_t\) follows by uniqueness applied
after a change of time origin. A relabeling yields the same
square with the vertical maps replaced by permutations.
These comparisons fix the sense in which the construction
respects the data of a game. A map between strategy sets that
identifies distinct types needs separate hypotheses to carry
the differential equation through the induced map on states.
A rest state has equal fitness \(\phi(x)\) on its support.
This condition is sufficient because the velocity of every
absent type is already zero. A Nash state also bounds the
payoffs of absent types above by \(\phi(x)\). The distinction
is caused by support preservation. In the interior the two
conditions coincide. For any pair of present types,
\[
\frac{d}{dt}\log\frac{x_i}{x_j}=F_i(x)-F_j(x).
\]
Thus a constant lower bound \(\gamma>0\) for the payoff
difference along an interval \([0,t]\) gives
\(x_i(t)/x_j(t)\ge e^{\gamma t}x_i(0)/x_j(0)\).
The inference concerns the relative growth of two present
types. It yields no positive mass for a type initially absent.
\section{Two mutation conventions}
A revision generator \(Q\) has nonnegative off-diagonal
entries and zero row sums, with \(Q_{ji}\) denoting the rate
from \(j\) to \(i\) for \(j\ne i\). Independent mutation
during the evolution of the proportions gives
\begin{equation}
\dot x_i=x_i(F_i(x)-\phi(x))+\sum_jx_jQ_{ji}.
\label{eq:additive-mutation}
\end{equation}
The generator contributes zero to the mass derivative.
On the simplex the contribution to a vanishing coordinate
is the sum of the incoming rates weighted by the masses.
This property yields forward invariance even when the
initial state lies on the boundary. To prove it, in the
ambient equation replace \(x_j\) by \(\max(x_j,0)\) in the
incoming terms with \(j\ne i\). The modified problem is
locally Lipschitz. In the equation for coordinate \(i\) the
terms involving \(x_i\) are linear in \(x_i\) along a given
solution, and the remaining forcing is nonnegative.
Variation of constants proves that all coordinates stay
nonnegative for \(t\ge0\). The modification consequently
agrees with the original equation on this solution. The
mass identity then retains \(m=1\), and compactness permits
global forward continuation.
For a constant \(Q\), a positive coordinate remains positive
by the same variation formula. If \(Q_{ji}>0\) and coordinate
\(j\) is positive for \(t>0\), its forcing makes \(i\) positive
for \(t>0\) as well. Induction along directed edges shows
that irreducibility, meaning directed reachability between
all strategies, produces full support at every strictly
positive time. Mutations change the invariant face structure
present in \eqref{eq:replicator}. They also destroy its
backward invariance on the closed simplex in general.
Another convention incorporates mutation into reproduction.
Let \(R\) be a row stochastic matrix of offspring probabilities
and let \(b_j(x)\ge0\) be an absolute birth rate. With fitness
used in these units the normalized equation is
\begin{equation}
\dot x_i=\sum_j x_jb_j(x)R_{ji}
-x_i\sum_j x_jb_j(x).
\label{eq:birth-mutation}
\end{equation}
This follows by normalizing the abundance equation with
incoming births \(\dot n_i=\sum_jn_jb_j(x)R_{ji}\). Setting
\(R\) equal to the identity recovers selection, and the sum
of the velocities on the simplex is zero. Under the same
local Lipschitz hypothesis, forward invariance follows by
the incoming-rate argument. For that argument the rates
can be extended off the simplex by their positive parts so
that incoming forcing remains nonnegative. For the diagonal
term the coefficient is \(b_iR_{ii}-\sum_jx_jb_j\).
A common baseline added to \(b_j\) changes
\eqref{eq:birth-mutation} by \(c\bigl(R^{\mathsf T}x-x\bigr)\).
This describes mutation occurring during additional births.
In \eqref{eq:additive-mutation}, the corresponding baseline
change to \(F\) has no effect. The two conventions therefore
require different information about which transformations of
fitness preserve the model. The formulation of an evolution
operator should specify where mutation enters before passing
to equivalence classes of payoff data \cite{hofbauer-sigmund}.
\section{Time changes and the strength of mutation}
Multiplying a fitness assignment by a positive constant \(a\)
changes the replicator solution from \(x(t)\) to \(x(at)\).
The geometric trajectories are the same and their
parametrizations differ. The same rule holds for a positive
continuous scalar \(a(x)\) on the simplex. Along a trajectory
of the rescaled field, the clock for the original field has
rate \(a(x(t))\). On the compact simplex, the positive
minimum of \(a\) ensures that this clock advances without
bound as physical time grows. Thus this kind of rescaling
preserves forward limit sets as well as rest points.
In the additive mutation model, the same clock rescales
both selection and the generator. Holding \(Q\) fixed while
rescaling \(F\) can change the stationary states. More
precisely, if \(x\) is stationary for both fitness \(F\)
and fitness \(aF\) with the same \(Q\) and a constant
\(a\ne1\), subtraction of the two stationary equations gives
\(R_F(x)=0\). The original equation then gives
\(Q^{\mathsf T}x=0\). Conversely, a common zero of these two
fields is stationary for every relative scaling. When \(Q\)
is irreducible this test picks out at most its unique
stationary distribution \cite{hofbauer-sigmund}. Consequently,
fitness units and mutation rates carry joint information
when the model includes both mechanisms.
\par
\chapter{Invasion and Lyapunov geometry}
\label{ch:lyapunov}
\section{Relative entropy near a resident state}
A state \(p\in\Delta(S)\) is locally superior for \(F\) if
\[
(p-x)\cdot F(x)>0
\]
for every distinct \(x\) sufficiently close to \(p\) in the
simplex. This formulation compares the resident with a
nearby population in the environment of that population.
For symmetric matrix games it is equivalent to the usual
evolutionarily stable strategy condition discussed in
Chapter~\ref{ch:ess}. The implication to dynamical stability
is the classical result studied by Hofbauer, Schuster, and
Sigmund \cite{hofbauer-schuster-sigmund,alexander}.
Set \(U_p=\{x\in\Delta(S):x_i>0\text{ whenever }p_i>0\}\).
The quantity
\[
D(p\Vert x)=\sum_{i:p_i>0}p_i\log\frac{p_i}{x_i}
\]
is finite on \(U_p\), including when \(p\) lies in a proper
face. The bound \(-\log z\ge1-z\) at \(z=x_i/p_i\) yields
\[
D(p\Vert x)\ge1-\sum_{i:p_i>0}x_i\ge0.
\]
Equality demands \(x_i=p_i\) on the support of \(p\), which
also forces the remaining entries of \(x\) to vanish.
The derivative along \eqref{eq:replicator} is
\begin{equation}
\frac{d}{dt}D(p\Vert x(t))
= x(t)\cdot F(x(t))-p\cdot F(x(t)).
\label{eq:kl-dissipation}
\end{equation}
The cancellation uses \(\sum_i p_i=1\). Support preservation
makes the logarithms legitimate for all finite times
whenever they are defined initially.
\begin{theorem}[Local stability from superiority]
\label{thm:superiority-stability}
For the replicator equation of Chapter~\ref{ch:replicator},
a locally superior state \(p\) is a locally asymptotically
stable rest point relative to \(\Delta(S)\).
\end{theorem}
\begin{proof}
For any \(q\in\Delta(S)\), approach \(p\) along the segment
\(x=p+\epsilon(q-p)\). Dividing the superiority inequality
by \(\epsilon>0\) and taking its limit shows
\((q-p)\cdot F(p)\le0\). Thus \(p\) is Nash and is a rest
point of the replicator field. Take a closed relative
ball around \(p\) contained in \(U_p\) and in the neighborhood
where superiority holds. Taking the ball small enough
is possible even when \(p\) lies on the boundary because
only its positive coordinates need to stay positive.
If the simplex is a singleton, stability is immediate.
Otherwise, for a sufficiently small radius the relative
sphere is nonempty, and the minimum there of \(D(p\Vert-)\)
is positive by compactness. Any trajectory starting inside
with entropy below this minimum stays inside. By taking the
radius arbitrarily small we obtain Lyapunov stability.
For such a trajectory, the entropy is nonincreasing and
bounded below. If a point \(z\ne p\) were a limit point,
its entropy derivative would be strictly negative.
The trajectory has bounded speed on the closed ball.
It would therefore spend intervals of a fixed positive
duration where the derivative is bounded above by a
negative constant, on arbitrarily late visits near \(z\).
Taking disjoint visits contradicts convergence of the
entropy values. The limit points are consequently all \(p\).
The trajectory lies in a compact set and hence converges
to \(p\).
\end{proof}
The same argument supplies a global conclusion on \(U_p\)
if \(p\) is a Nash state and
\[
(x-p)\cdot\bigl(F(x)-F(p)\bigr)<0
\qquad(x\ne p)
\]
on the whole simplex. The Nash inequality implies
\((x-p)\cdot F(x)<0\). A finite sublevel set of
\(D(p\Vert-)\) stays away from zeros of the required
coordinates, so the compact-set argument applies there.
For completeness, each term with \(p_i>0\) is bounded below
by \(p_i\log p_i\), while the term for \(x_i\) tends to
infinity at \(x_i=0\). This gives positive lower bounds
for the required coordinates on any one sublevel set.
This strict inequality is a special sufficient condition.
Hofbauer and Sandholm establish broader geometric criteria
for convergence in stable population games
\cite{hofbauer-sandholm-stable}.
\section{Selection as a metric gradient}
At an interior state \(x\), tangent vectors are vectors \(u\)
with \(\sum_i u_i=0\). The Shahshahani metric is
\[
g_x(u,v)=\sum_i\frac{u_i v_i}{x_i}.
\]
The same definition applies to the relative interior of
each face by restricting the coordinates to its support
\cite{shahshahani,hofbauer-sigmund}. Define the fitness
one-form on a face by \(\alpha_x(u)=\sum_iF_i(x)u_i\).
If \(R_F\) denotes the replicator vector field, its
metric pairing with \(u\) is
\[
g_x(R_F(x),u)=\sum_i(F_i(x)-\phi(x))u_i=\alpha_x(u).
\]
The field is the metric dual of the payoff one-form.
Adding a state-dependent baseline to all payoffs leaves
the one-form unchanged, because all tangent vectors
have zero sum.
Pointwise, every tangent covector has a representative
in the ambient strategy coordinates, and two such
representatives differ by a multiple of \(\mathbf 1\).
Writing \(X\) for the relative interior of the full simplex,
the fitness, one-form, and field correspond by
\[
\begin{tikzcd}[column sep=huge]
\mathbb R^S/\mathbb R\mathbf 1
\arrow[r,"{[f]\mapsto f\cdot(-)}"] \arrow[dr,"R_x"']
& T_x^*X \arrow[d,"g_x^{-1}"] \\
& T_xX .
\end{tikzcd}
\]
Here \(R_x\) denotes the linear map that projects a fitness
vector by its resident mean and multiplies by the current
frequencies. At a proper face the corresponding metric
duality uses only supported coordinates. All payoff
coordinates outside the support then disappear from
the tangent covector, even though they are needed to
evaluate invasion by absent types.
If \(\alpha=dV\) for a differentiable potential \(V\), this
duality says that \(R_F\) is the Riemannian gradient of \(V\).
The rate of increase of the potential is
\begin{equation}
\frac{d}{dt}V(x(t))=g_x(R_F,R_F)
=\sum_i x_i(F_i(x)-\phi(x))^2 .
\label{eq:potential-rate}
\end{equation}
Only supported coordinates enter this identity. For a
potential continuously differentiable on the closed
simplex in its affine hull, the right side extends
continuously to every face. Compactness bounds the
potential and the speed of the trajectory. As in the
entropy argument, a nonrest limit point would give
repeated fixed-duration intervals of strictly positive
increase, which is impossible. This shows that each
limit point is a rest point. It leaves convergence to a
particular rest point open when the rest set has several
possible limit points.
For matrix payoffs, write \(A=S+K\) with \(S\) symmetric
and \(K\) skew-symmetric. When \(K=0\), one can take
\(V(x)=x^{\mathsf T}Sx/2\). More generally \(A\) defines a
potential on the interior precisely when the bilinear form
\(u^{\mathsf T}Av\) is symmetric on tangent vectors.
Indeed, the exterior derivative of the payoff one-form
on constant tangent vectors is
\(v^{\mathsf T}Au-u^{\mathsf T}Av\). This must vanish
for a potential, and it suffices because the open
simplex is convex. Thus a skew part matters for
integrability only through its restriction to
tangent directions.
\section{Dissipation and a conservative obstruction}
At an interior rest point \(p\) of a matrix game,
\(Ap=c\mathbf{1}\). Consequently
\[
(x-p)\cdot Ax=(x-p)^{\mathsf T}S(x-p).
\]
The right side depends only on the symmetric part
because \(u^{\mathsf T}Ku=0\). Strict negative
definiteness of \(S\) on the tangent hyperplane gives
strict entropy dissipation. The skew part can
nevertheless change the direction of the velocity.
For a purely skew field \(A=K\) with \(Kp=0\), one
instead has \(\phi(x)=0\) and \(p^{\mathsf T}Kx=0\)
everywhere. Equation~\eqref{eq:kl-dissipation} makes
\(D(p\Vert x)\) a first integral on the interior.
Every trajectory from a distinct interior state has
positive constant entropy, so it cannot converge to
\(p\). This obstructs asymptotic stability regardless
of whether individual level sets carry periodic motion.
Suppose that \(p\) is interior and that there is a strict
uniform bound of the form
\((x-p)\cdot F(x)\le -c\lVert x-p\rVert^2\)
near \(p\), with \(c>0\). The entropy Hessian in the
ambient positive orthant is
\(\operatorname{diag}(p_i/x_i^2)\). This Hessian is
bounded above and below by positive definite diagonal
matrices on a small closed neighborhood of \(p\). The first
derivative of entropy at \(p\) vanishes on the tangent
space. Taylor's theorem therefore gives positive constants
\(a,b\) such that
\[
a\lVert x-p\rVert^2\le D(p\Vert x)
\le b\lVert x-p\rVert^2 .
\]
An initial state close enough stays in this neighborhood
by Lyapunov stability. Then \eqref{eq:kl-dissipation}
implies exponential decay at rate at least \(c/b\) for
the entropy. This argument uses the full support of the
resident. At a boundary resident, redistributing mass to
an absent strategy can give a linear term in the expansion.
For a fixed \(q\) with mass \(m\) outside the support of
\(p\), direct differentiation yields
\[
D(p\Vert p+\epsilon(q-p))
=\epsilon m+O(\epsilon^2)
\qquad(\epsilon\downarrow 0).
\]
The general stability implication accommodates this boundary
term through the entropy sign and compactness.
These statements concern the vector field associated
with a given payoff rule. Equation~\eqref{eq:potential-rate}
uses the exact derivative of a potential and
\eqref{eq:kl-dissipation} uses the comparison with a
resident. They give separate ways to turn a strategic
inequality into a dynamical conclusion, with distinct
hypotheses at the boundary and on the rest set.
\par
\chapter{Revision protocols and response dynamics}
\label{ch:revision}
\section{Response as a differential inclusion}
A payoff assignment gives current incentives. A revision
protocol determines which of those incentives are used when
a strategy changes. This extra datum affects boundary motion,
the identification of rest points, and convergence.
Alexander and Sandholm develop the passage from revision
protocols to population dynamics
\cite{alexander,sandholm-population}.
Write
\[
B(x)=\operatorname*{arg\,max}_{y\in\Delta(S)} y\cdot F(x).
\]
Because the objective is linear in \(y\), this is the
nonempty convex face spanned by the maximizers of
\(F_i(x)\). Best-response revision uses the differential
inclusion
\[
\dot x(t)\in B(x(t))-x(t).
\]
The solution convention is an absolutely continuous path
satisfying the inclusion almost everywhere. If \(F\) is
continuous, the graph of \(B\) is closed. In fact, taking a
limit in the maximizing inequalities for \(y_n\in B(x_n)\)
shows that any limit \(y\) maximizes against the limit \(x\).
Compactness of the simplex then gives upper semicontinuity.
One may extend the correspondence to the ambient Euclidean
space by precomposing \(B\) with the nearest-point projection
onto the simplex. The standard existence theorem for locally
bounded upper semicontinuous differential inclusions with
nonempty compact convex values supplies a local solution
\cite{aubin-cellina}.
For a solution, \(b(t)=\dot x(t)+x(t)\) belongs to the
simplex almost everywhere. The integrating factor gives
\[
x(t)=e^{-t} x(0)+\int_0^t e^{-(t-s)}b(s)\,ds .
\]
The coefficients form a probability weighting. This keeps
the path in the simplex and hence within the region where
the original correspondence is defined. The velocity is
bounded there, so the path extends to every forward time
by continuation. Existence requires no choice of best
response at a tie. There may be several paths from the same
initial state. A point admits a constant path exactly when
\(x\in B(x)\), the Nash condition. Convergence results for
this inclusion require substantial further structure.
For instance, Hofbauer and Sorin study continuous
concave--convex zero-sum games and identify convergence
to the saddle-point set \cite{hofbauer-sorin}.
\section{Pairwise comparison}
Suppose a reviser currently using \(j\) switches towards
\(i\) at rate \(\rho(F_i(x)-F_j(x))\), absorbing any common
uniform sampling frequency into time. Let \(\rho\) be
nonnegative and locally Lipschitz, strictly positive for
positive arguments and zero for nonpositive arguments.
The drift of \(x_i\) is the difference between incoming
and outgoing flux:
\begin{equation}
W_i(x)=\sum_j x_j\rho(F_i(x)-F_j(x))
-x_i\sum_j\rho(F_j(x)-F_i(x)).
\label{eq:pairwise-drift}
\end{equation}
Each individual transfer contributes with opposite signs
to the two relevant coordinates. Summing therefore gives
zero, and the rate at an absent coordinate is nonnegative.
If \(F\) is locally Lipschitz, the incoming-term variation
argument of Chapter~\ref{ch:replicator} shows that the
resulting equation has a unique global forward trajectory
from each simplex state.
\begin{proposition}[Rest states under comparison]
A state is a rest point of \eqref{eq:pairwise-drift}
if and only if it is Nash.
\end{proposition}
\begin{proof}
Reindexing the sum of the outgoing terms gives
\[
F(x)\cdot W(x)=
\sum_{i,j} x_j\rho(F_i(x)-F_j(x))
(F_i(x)-F_j(x))\ge0.
\]
Every summand is nonnegative, and it is positive
exactly when \(x_j>0\) and \(F_i(x)>F_j(x)\).
At a rest point no such pair exists. Thus every used
strategy maximizes payoff, so \(x\in B(x)\).
Conversely, if \(x\in B(x)\), every transition away
from a used strategy has rate zero. Both flux terms
vanish.
\end{proof}
This is the positive-correlation argument for pairwise
comparison dynamics \cite{sandholm-pairwise}. It also yields
a strict increase of the potential along nonstationary
paths when the payoff one-form is exact. In a related
protocol, sample the candidate from the current population
and use \([F_i-F_j]_+=\max(F_i-F_j,0)\) as the switch
rate conditional on the candidate. The flux from \(j\) to
\(i\) is then \(x_j x_i[F_i-F_j]_+\).
Combining it with its reverse flux and using
\([a]_+-[-a]_+=a\) recovers \eqref{eq:replicator}.
This latter sampling convention cannot create an absent
type. The state-independent availability of candidates is
the hypothesis that makes the preceding characterization
include the boundary Nash inequalities.
\section{Logit revision and regularization}
Fix a noise parameter \(\eta>0\). The logit response is
\[
B^\eta_i(x)=\frac{\exp(F_i(x)/\eta)}
{\sum_j\exp(F_j(x)/\eta)} .
\]
The smooth response map from fitness vectors to probabilities
preserves common payoff baselines and is equivariant under
strategy relabeling. For a bijection \(S\to S'\) with
permutation map \(P\) the latter statement is
\[
\begin{tikzcd}[column sep=large]
\mathbb R^S \arrow[r,"\operatorname{softmax}_{\eta}"]\arrow[d] &
\Delta(S)\arrow[d]\\
\mathbb R^{S'}\arrow[r,"\operatorname{softmax}_{\eta}"]&
\Delta(S').
\end{tikzcd}
\]
The two vertical maps apply \(P\). The result follows because
the normalization is a sum over the strategy set. A
many-to-one map need not commute with the same response,
since several copied entries contribute several terms to
that sum.
In detail, let \(q\colon S'\twoheadrightarrow S\) be a
surjection and pull a payoff vector \(f\) back to \(f\circ q\).
The mass assigned to the fiber of \(i\) is proportional to
\(\lvert q^{-1}(i)\rvert\exp(f_i/\eta)\). The effective
coarse payoff includes a shift of \(\eta\log\lvert
q^{-1}(i)\rvert\), which cancels from the response when
the fiber sizes agree. More generally, using positive
reference weights \(w_j\) on \(S'\) makes the mass of a fiber
proportional to
\(\exp(f_i/\eta)\sum_{j:q(j)=i} w_j\).
The weighted response therefore commutes with lumping for
pulled-back fitness vectors when one pushes the reference
weights forward. To extend this calculation to a
semiconjugacy of the revision equations, the fine fitness
must be constant within each fiber and must be computed
from the coarse state by the same rule at every instant.
One can derive the logit rule by regularizing the response
optimization. Let \(H(y)=-\sum_i y_i\log y_i\), using the
continuous extension \(0\log0=0\). For fixed \(x\), write
\(Z=\sum_j\exp(F_j(x)/\eta)\). Substituting the definition
of \(B^\eta\) gives
\[
y\cdot F(x)+\eta H(y)
=\eta\log Z-\eta D(y\Vert B^\eta(x)).
\]
The identity is valid also for \(y\) on a proper face because
the second argument of entropy has full support. By the
nonnegativity established in Chapter~\ref{ch:lyapunov}, the
unique regularized maximizer is \(B^\eta(x)\). Compare its
objective value with that of a pure maximizing strategy,
which has entropy zero. Since \(H(y)\le\log |S|\),
\[
0\le \max_i F_i(x)-B^\eta(x)\cdot F(x)
\le\eta\log |S|.
\]
The entropy bound follows by computing relative entropy
against the uniform distribution. In particular, each
logit rest point is an \(\eta\log|S|\)-approximate Nash
state, in the sense that this is a bound on the gain
from any unilateral population-level deviation.
The dynamics \(\dot x=B^\eta(x)-x\) have a unique forward
solution for locally Lipschitz \(F\). The same integrating
factor used for best response shows forward invariance.
For a fixed \(\eta\), continuity on the compact simplex
gives a positive lower bound for every response coordinate.
Thus the solution is interior for all \(t>0\), with each
coordinate bounded uniformly away from zero after any
positive time. Brouwer's fixed point theorem ensures a
rest point for continuous \(F\). Every rest point is
interior and satisfies \(x=B^\eta(x)\).
If \(\eta_k\to0\) through positive parameters and
\(x^{(k)}=B^{\eta_k}(x^{(k)})\) converges, its limit is Nash.
For a strategy strictly suboptimal at the limit, continuity
gives a payoff gap of at least some \(\delta>0\) compared
with a maximizing strategy there, for all large \(k\).
Its probability in \(x^{(k)}\) is at most
\(\exp(-\delta/\eta_k)\), hence is zero in the limit.
Only maximizers can retain mass. This argument does not
choose which Nash states arise along branches of fixed points.
The potential case gives a convergence result at a fixed
noise level. Let \(V\) be a potential in the sense of
Chapter~\ref{ch:lyapunov}, continuously differentiable on
the simplex in its affine hull. On the interior the logit
dynamics satisfy
\begin{align*}
\frac{d}{dt}(V+\eta H)(x)
&=\eta\sum_i (B^\eta_i(x)-x_i)
\bigl(\log B^\eta_i(x)-\log x_i\bigr)\\
&=\eta\bigl[D(B^\eta(x)\Vert x)+D(x\Vert B^\eta(x))\bigr].
\end{align*}
The first equality uses
\(F_i(x)=\eta\log B^\eta_i(x)+\eta\log\sum_j e^{F_j(x)/\eta}\)
and the fact that the velocity sums to zero. Thus \(V+\eta H\)
strictly increases except at rest points. The interior bounds
after positive time make the compactness argument of
Chapter~\ref{ch:lyapunov} applicable. If \(V\) is concave,
then \(V+\eta H\) is strictly concave on the simplex. Each
interior logit rest point satisfies its constrained
first-order condition, so there is only one. All trajectories
converge to it. The existence, limit, and convergence
statements use different hypotheses on fitness and on the
response rule. More general perturbation and revision
theories are developed by Sandholm \cite{sandholm-population}.
\par
\chapter{Interacting populations and spatial structure}
\label{ch:structured}
The comparison neighborhood in Chapter~\ref{ch:revision} consisted of
individuals drawn from the same population. An individual can instead
interact frequently with nearby populations while drawing its behavioral
replacement from its own population. An individual who moves changes the
composition of both the source and destination. These mechanisms give
different equations and lead to different forms of reduction. We study
exact synchronized subspaces for a deterministic interaction model and
then derive a quantitative synchronization criterion for migration.
\section{Product simplices and assortative encounters}
Let $I$ be a finite set of populations and $S_r$ a finite set of types
for each $r\in I$. A state in
$\mathcal D_I=\prod_{r\in I}\Delta(S_r)$ records conditional
frequencies within the populations. The payoff to type $i\in S_r$ is
$F_i^r(x)$, potentially a function of the full profile. Local
normalization gives
\[
\phi_r(x)=\sum_{i\in S_r}x_i^r F_i^r(x),
\qquad
\dot x_i^r=x_i^r\bigl(F_i^r(x)-\phi_r(x)\bigr).
\]
Each population has its own conserved mass. Assuming a locally
Lipschitz extension near the product, the proof of
Chapter~\ref{ch:replicator} applies simultaneously. The equation for each
population's mass receives its own continuous coefficient $\phi_r(x)$,
and each coordinate still has an integrating factor. Compactness of the
product yields the global flow. A choice of one face per population
gives an invariant product face. If every $F^r$ depends solely on $x^r$,
the flow is the product of the separate population flows. If payoffs
couple populations, a product face can remain invariant while the flow
within it is coupled.
Take a common type set $S$, a matrix $A$ of row-player payoffs on $S$,
and a row stochastic matrix $W=(W_{rs})$ on $I$. An encounter by a
resident of $r$ draws its partner's population from row $r$ and then
draws the type from that population. The vector of expected payoffs is
\[
F^r_W(x)=A\sum_{s\in I} W_{rs}x^s.
\]
Diagonal weights quantify within-population encounters. Directed weights
allow asymmetric access to partners. The state equations apply the
selection step within each destination of the payoff map, retaining
separate local averages even when all populations see the same
distribution of partners. At a stationary profile each resident type in
each population has the same payoff. Comparison with absent types is
required to promote this equality to a population Nash condition.
Here $W$ is part of a deterministic model posed directly at the
population level. For an individual-based graph process, replacing, for
example, the conditional distribution of neighbors of type $i$ by the
unconditional population mixture is an additional modeling step.
Ohtsuki and Nowak derive update-dependent replicator approximations for
regular graphs using weak selection and pair
approximation~\cite{ohtsuki-nowak}. That derivation treats local
correlations explicitly. The present equation follows directly from the
specified partner sampling.
\section{Exact reduction along equitable partitions}
Let $\pi\colon I\twoheadrightarrow J$ partition $I$ into blocks, and
write $L\colon\mathcal D_J\longrightarrow\mathcal D_I$ for replication
of block values, $(Lz)^r=z^{\pi(r)}$. The partition is equitable for
$W$ if the total weight from $r$ to block $b$ depends only on the block
$a=\pi(r)$. Define $\bar W_{ab}$ to be that total. This gives a row
stochastic matrix $\bar W$ and, if $L$ also denotes the incidence matrix
$L_{ra}=\mathbf 1_{\pi(r)=a}$, the identity $WL=L\bar W$.
\begin{proposition}
Write $R_W$ and $R_{\bar W}$ for the replicator fields built with the
same payoff matrix $A$. For an equitable partition, $R_W(Lz)=
L R_{\bar W}(z)$. In particular the image of $L$ is invariant, and the
restriction of the flow to that image is conjugate to the reduced flow.
\end{proposition}
\begin{proof}
For $r$ in block $a$, the expanded profile has payoff
$F_W^r(Lz)=A\sum_b\bar W_{ab}z^b=F_{\bar W}^a(z)$. Its resident
frequencies are $z^a$, so its local mean payoff is the same as the block
mean payoff. Substituting those two equalities makes all derivatives
agree with the expanded block derivatives. A reduced solution lifts to
a full solution, which is unique from its initial condition.
\end{proof}
For the flows $\Phi$, this reduction is an intertwining relation.
\[
\begin{tikzcd}[column sep=large,row sep=large]
\mathcal D_J \arrow[r,"\Phi_t^{\bar W}"] \arrow[d,"L"']
& \mathcal D_J \arrow[d,"L"] \\
\mathcal D_I \arrow[r,"\Phi_t^W"]
& \mathcal D_I
\end{tikzcd}
\]
If $\bar W L'=L'\widetilde W$ for a second equitable partition,
then $W(LL')=(LL')\widetilde W$. Thus the embeddings and their
intertwining equations compose. The one-block partition is equitable for
any stochastic $W$. Its synchronous dynamics is the ordinary
replicator equation for $A$.
For a general profile, a weighted block average retains less information
than is needed to compute its derivative. For probability weights $\mu$
on a specified block, put $\bar x_i=\sum_r\mu_r x_i^r$ over that block
and $h_i^r=F_i^r(x)-\phi_r(x)$. Then
\[
\dot{\bar x}_i
=\bar x_i\sum_r\mu_r h_i^r
+\sum_r\mu_r(x_i^r-\bar x_i)
\left(h_i^r-\sum_s\mu_s h_i^s\right).
\]
The second term is a covariance. Its appearance precedes any further
attempt to express the averages of relative payoffs through the
coarse state. Equitability controls the aggregate encounter weights,
while the embedding supplies the synchronized state on which that
control implies an exact closed system.
\section{A potential from detailed balance}
Suppose $W$ admits a positive probability vector $\mu$ satisfying
$\mu_r W_{rs}=\mu_s W_{sr}$. If $A=A^{\mathsf T}$, define
\[
P(x)=\frac12\sum_{r,s}\mu_r W_{rs}
(x^r)^{\mathsf T}Ax^s .
\]
Differentiation of the first factor gives
$\tfrac12\sum_r\mu_r\langle u^r,F_W^r(x)\rangle$.
In the term from the second factor exchange $r$ and $s$, then apply
detailed balance and symmetry of $A$. This gives the other half of
$dP_x(u)=\sum_r\mu_r\langle u^r,F_W^r(x)\rangle$.
On the relative interior of each product face, give the tangent space
the weighted product of the Shahshahani metrics from
Chapter~\ref{ch:lyapunov}. Its inner product is
$\sum_r\mu_r\sum_{i:x_i^r>0}u_i^r v_i^r/x_i^r$.
For $v=R_W(x)$ the local means cancel against $\sum_i u_i^r=0$.
Therefore $R_W$ is the gradient of $P$ in this metric. The rate of
increase of $P$ equals the weighted sum of the within-population payoff
variances, and vanishes at stationary profiles. This argument uses
reciprocity of the encounter-weighted payoff couplings. A directed row
stochastic matrix by itself supplies the reduction theorem but does
not supply the symmetry used in the derivative of $P$.
\section{Migration and exponential synchronization}
Let $V=R_F$ be a common local replicator field with a Lipschitz constant
$C$ on $\Delta(S)$ in Euclidean norm. Consider the additional coupling
\[
\dot x^r=V(x^r)+\kappa\left(\sum_s W_{rs}x^s-x^r\right),
\qquad \kappa\geq 0.
\]
This term diffuses relative frequencies between equal-normalization
populations. It mixes a particular type across populations even when
its local frequency is initially zero. If the replicator has the
ambient form above, the total mass vector $m$ satisfies
$\dot m_r=(1-m_r)\phi_r+\kappa((Wm)_r-m_r)$ along any solution.
The constant unit vector solves this linear inhomogeneous equation,
with the coefficients taken along that solution. Uniqueness preserves
each mass. The incoming linear terms have nonnegative off-diagonal
coefficients for every type. Variation of constants, or local Picard
iteration of the resulting positive integral equation for each type,
preserves nonnegativity. Thus the model has a global flow on
$\mathcal D_I$.
\begin{proposition}
Let $|I|\geq 2$ and suppose $W$ is irreducible and reversible with a
positive stationary distribution $\mu$. Let $\lambda>0$ be the
smallest positive eigenvalue of $I-W$ in the $\mu$-weighted inner
product. If $\kappa\lambda>C$, the squared distance to the synchronous
subspace, in the product norm with weights $\mu$, decays at least at
rate $2(\kappa\lambda-C)$.
\end{proposition}
\begin{proof}
For the mean $\bar x=\sum_r\mu_rx^r$ set $\delta^r=x^r-\bar x$ and
$E=\tfrac12\sum_r\mu_r\lVert\delta^r\rVert^2$. Since the deviations
have weighted mean zero, the term from $\dot{\bar x}$ in $\dot E$
vanishes. The local contribution is
$\sum_r\mu_r\langle\delta^r,V(x^r)-V(\bar x)\rangle\leq 2CE$.
The migration contribution equals $-\kappa$ times the Dirichlet form
\[
\left\langle\delta,(I-W)\delta\right\rangle_\mu
=\frac12\sum_{r,s}\mu_rW_{rs}
\lVert\delta^r-\delta^s\rVert^2.
\]
Reversibility makes $I-W$ self-adjoint. Irreducibility forces every
zero of the displayed form to be constant across populations. Applying
the spectral gap on the mean-zero subspace bounds the form below by
$2\lambda E$. The claim follows by integrating
$\dot E\leq -2(\kappa\lambda-C)E$.
\end{proof}
Migration cancels from the mean equation by stationarity of $\mu$.
The mean therefore satisfies
$\lVert\dot{\bar x}-V(\bar x)\rVert\leq C\sqrt{2E}$. The estimate
controls the deviation from the local law along the mean path, and the
stability of any particular synchronous orbit is a further dynamical
question. The partition theorem gives an exact invariant subspace. The
migration estimate gives an attraction criterion for one such subspace.
Both properties specify what a spatial reduction preserves and under
which hypotheses it does so.
\par
\bookpart{III}{Stochastic populations and aggregation}
\chapter{Finite populations and Markov dynamics}
\label{ch:markov}
Alexander starts the finite-population analysis from individual replacement and then asks what changes when interaction is local \cite[Sec.~4]{alexander}. To make this distinction precise, separate the population state, the law of an update, and the statistic that describes the next state. A stochastic update assigns a probability to each possible successor. Its support determines what can happen, and its probability weights determine the relative likelihoods. Both are needed to describe extinction, fixation, and stationary behavior.
\section{Count states and fitness}
Let \(S\) be a nonempty finite strategy set. Fix a population size \(N\geq 2\). A labeled configuration is a function \(z:\{1,\ldots,N\}\to S\). Its count vector is \(n(z)\), with \(n_i(z)\) the number of individuals using strategy \(i\). The set of count states and the associated frequencies are
\[
C_N(S)=\left\{n\in\mathbb N^S:\sum_{i\in S}n_i=N\right\},
\qquad x_i=\frac{n_i}{N}.
\]
Count states are orbits of the action of label permutations on configurations. The induced action on an update law can be used to test whether the orbit statistic itself evolves by a Markov chain. Under exchangeable sampling, the law of the updated counts depends only on the current counts. On a graph, two configurations in the same orbit can place the same strategy at vertices with different neighborhoods. The Markov property for counts then requires a separate argument.
Suppose \(A_{ij}\) is the payoff to strategy \(i\) against \(j\), and that an individual's opponent is sampled uniformly from the other \(N-1\) individuals. The expected payoff of a present type \(i\) is
\[
u_i^N(n)=\sum_{j\in S}\frac{n_j-\mathbf 1_{\{i=j\}}}{N-1}A_{ij}.
\]
If \(n_i=0\), there is no individual of type \(i\) whose payoff is needed. Give each currently present type a strictly positive reproductive weight \(f_i(n)\). For instance a positive response \(f_i(n)=\phi(u_i^N(n))\) implements a prescribed mapping from payoffs to reproductive probabilities. The choice is part of the model. Adding the same constant to all payoffs preserves many static comparisons. Adding the same constant to positive reproductive weights changes ratios \(f_i/f_j\) in general and hence can change the stochastic dynamics. A common positive multiplication of all weights at a state leaves their normalized ratios unchanged. The distinction parallels the several meanings of payoff equivalence in Chapter~\ref{ch:payoff-equivalence}.
\section{Birth, mutation, and replacement}
The birth--death process updates one position at a time. It belongs to the overlapping-generation tradition of Moran's model \cite{moran-random}. Here the parent is chosen with probability proportional to its reproductive weight, the offspring type is sampled from a mutation kernel, and a uniformly chosen member of the current population is replaced. The parent is eligible for replacement in this convention. Let \(M_{ik}\) be the probability that a type \(i\) parent produces type \(k\), with each row of \(M\) summing to one. Define
\[
b_i(n)=\frac{n_i f_i(n)}{\sum_j n_j f_j(n)},\qquad
c_k(n)=\sum_i b_i(n)M_{ik},\qquad
d_j(n)=\frac{n_j}{N}.
\]
For an absent type we set \(n_i f_i(n)=0\) by convention. The offspring distribution is \(c(n)\). Let \(e_i\) denote the count vector with a unit at \(i\) and zero elsewhere. If \(k\neq j\) and \(n_j>0\), the transition from \(n\) to \(n+e_k-e_j\) has probability \(c_k(n)d_j(n)\). The probability of remaining at \(n\) is \(\sum_j c_j(n)d_j(n)\). These probabilities sum to one because all ordered pairs of offspring and removed types have been accounted for. A death rule depending on the parent or on its location replaces the product \(c_kd_j\) by a joint kernel. In that case the joint law is part of the dynamics.
The update can also be written as a composition of kernels, retaining only the information required by the next stage. With \(D_n=\{(k,j)\in S^2:n_j>0\}\), each arrow below is well defined on its displayed states:
\[
\begin{tikzcd}[column sep=large]
\{n\} \arrow[r,"c\otimes d"{above}] &
D_n \arrow[r,"{\text{replacement}}"{above}] &
C_N(S).
\end{tikzcd}
\]
The final map is deterministic. This factorization makes conservation of total population size explicit. It also gives a route to composition with other mechanisms, such as scheduling several local updates, by composing their kernels in time. The conditional mean of the frequency increment is
\begin{equation}
\mathbb E[x_{\mathrm{next}}-x\mid n]
=\frac{1}{N}\bigl(c(n)-x\bigr).
\label{eq:birth-death-drift}
\end{equation}
For this identity, simply average \(e_k-e_j\) against the joint law. Independence supplies the stated \(c\) and \(d\), while the mean-increment formula needs only their marginals. The second moments can depend on the joint law even when the drift agrees.
In the independent case, writing \(\operatorname{diag}(c)\) for the diagonal matrix with entries \(c_k\), the conditional covariance of the frequency increment is
\[
\frac{1}{N^2}\bigl(\operatorname{diag}(c)-cc^{\mathsf T}
+\operatorname{diag}(d)-dd^{\mathsf T}\bigr),
\]
with \(c,d\) taken as columns in this formula. Each removed or added type is represented by a random coordinate vector. Independence makes their cross covariance zero, so the two covariance matrices add. The matrix annihilates the all-ones vector, expressing that a frequency increment is tangent to the affine simplex. This covariance and the conditional drift answer different questions about the same update kernel.
\begin{proposition}[Fixation and recurrence]
\label{prop:fixation-recurrence}
For the update above with strictly positive reproductive weights, suppose first that \(M\) is the identity. The chain reaches a monomorphic population almost surely, and each monomorphic population is absorbing. If \(M\) instead has strictly positive entries, the chain is irreducible and aperiodic. It has a unique stationary law and converges to that law from every initial distribution.
\end{proposition}
\begin{proof}
When \(M\) is the identity, a currently absent type cannot be produced. In a monomorphic state all replacements preserve the type. Starting from any state containing \(i\), choose a parent of type \(i\) and remove an individual of another type at each successive step until reaching the monomorphic state of type \(i\). Each step has strictly positive conditional probability and the path has length at most \(N-1\). The state set is finite, so there is a uniform positive lower bound on the probability of absorption within \(N-1\) steps, considering one such path for each state. By the Markov property, the probability of avoiding absorption over successive blocks of \(N-1\) steps decays geometrically.
If \(M\) is strictly positive, \(c_k(n)>0\) for every \(k\) and every \(n\). A move removing a present type and adding any specified type is therefore possible. Starting from \(n\), repeatedly move a unit from a coordinate in surplus relative to a target \(n'\) to one in deficit. This reaches \(n'\) in at most \(N\) steps. Every state also has a self-loop because at least one \(d_j\) is positive and every \(c_j\) is positive. Thus the \(N\)-step transition matrix is strictly positive. A stationary law exists as a limit point of Ces\`aro averages. The \(N\)-step matrix has a common strictly positive minorant in every row. Removing it leaves the same residual mass in every row and gives a total-variation contraction by that residual mass, which is strictly less than one. This proves uniqueness and convergence for the full chain.
\end{proof}
\section{Fixation probabilities retain the update law}
When mutation is absent, let \(h_i(n)\) be the probability of fixation at the monomorphic state of type \(i\). Conditioning on the first update makes \(h_i\) harmonic away from the absorbing states. Its boundary values are one at fixation of \(i\) and zero at fixation of any other type. These conditions characterize it. Indeed, a harmonic function with these boundary values has constant conditional mean until absorption, and the absorption argument above gives almost sure convergence to the boundary values. Boundedness on the finite state space permits taking expectations in the limit.
Under neutral reproduction all present reproductive weights are equal, so \(c(n)=x\) in the mutation-free process. Equation~\eqref{eq:birth-death-drift} gives harmonicity of \(n_i/N\), and the boundary values give \(h_i(n)=n_i/N\). For nonneutral reproduction these fixation probabilities depend on the transition law. The drift alone can be insufficient when different updates induce different hitting distributions, and a deterministic trajectory is a different observable from the absorption probability. The next chapter studies selection by rare mutation, in which fixation between departures from recurrent states supplies the effective transition information. Chapter~\ref{ch:mean-field} compares the frequency process with its deterministic limit on a fixed time interval.
\par
\chapter{Rare mutation and stochastic stability}
\label{ch:stochastic-stability}
A small mutation probability changes the meaning of a long-run prediction. The mutation-free chain can have many recurrent classes. With positive mutation the chain can leave those classes, and its stationary law can weight them differently even when mutation tends to zero. The stationary limit first takes a long-time average at positive mutation and then decreases the mutation parameter. This order is intrinsic to the prediction.
\section{A theorem for stationary weights}
Let \(E\) be a nonempty finite state set and \(P\) an irreducible transition matrix on \(E\), with rows indexed by the starting state. An in-tree rooted at \(i\) assigns one outgoing edge to each vertex other than \(i\), such that repeated traversal of the edges brings any vertex to \(i\). The root has no outgoing edge. Define
\[
W_i(P)=\sum_{\tau\text{ rooted at }i}
\prod_{(x,y)\in\tau}P(x,y).
\]
The one-state case uses the weight one for the empty tree. Each weight is positive because the support of \(P\) has a path from every vertex to \(i\). Choosing at each vertex an edge decreasing the shortest such path length gives a tree of positive weight.
\begin{theorem}[Markov chain tree formula]
\label{thm:markov-trees}
The unique stationary distribution of \(P\) is
\(\pi(i)=W_i(P)/\sum_j W_j(P)\).
\end{theorem}
\begin{proof}
It suffices to show that the weighted inflow to \(i\) from distinct states equals the weighted outflow from \(i\):
\[
\sum_{j\ne i}W_j(P)P(j,i)
=W_i(P)\sum_{k\ne i}P(i,k).
\]
A term on the left adds the edge \(j\to i\) to a tree rooted at \(j\). This produces a directed graph with one outgoing edge at every vertex and a unique directed cycle, of length at least two, containing \(i\). A term on the right adds \(i\to k\) to a tree rooted at \(i\). It gives a graph of the same kind. From such a graph, deleting the edge entering \(i\) on the cycle recovers the construction on the left. Deleting the outgoing edge at \(i\) recovers the construction on the right. The products are the same because they use all edges of the corresponding graph. The diagonal terms of stationarity cancel, so this bijection proves the displayed identity. Normalize and use uniqueness of the stationary law for a finite irreducible chain.
\end{proof}
This is the rooted-tree form of the Markov chain tree theorem \cite{leighton-rivest}. It accounts for indirect paths between states. The stationary ranking can depend on several paths taken together and therefore need not coincide with a ranking of the individual edges leaving each state.
\section{Leading powers and stable states}
Suppose \(P^\epsilon\) is irreducible for every sufficiently small \(\epsilon>0\). Assume for each off-diagonal edge that its weight is either identically zero for small \(\epsilon\), or
\[
P^\epsilon(x,y)
=c_{xy}\epsilon^{r_{xy}}+o(\epsilon^{r_{xy}}),
\qquad c_{xy}>0,\quad r_{xy}\geq 0 .
\]
Call \(r_{xy}\) the resistance. Omit identically zero edges and set
\[
\gamma_i=\min_{\tau\text{ rooted at }i}
\sum_{(x,y)\in\tau}r_{xy}.
\]
For each root define \(a_i>0\) by summing \(\prod_{(x,y)\in\tau}c_{xy}\)
over its minimizers. Each tree product has exponent given by the sum of its edge resistances. The tree sum is finite and its coefficients at the smallest exponent are positive. Hence
\[
W_i(P^\epsilon)
=\epsilon^{\gamma_i}(a_i+o(1)).
\]
Writing \(\gamma_*=\min_i\gamma_i\), the stationary mass tends to zero for \(\gamma_i>\gamma_*\) and tends to \(a_i/\sum_{\gamma_j=\gamma_*}a_j\) for \(\gamma_i=\gamma_*\). A state with positive limiting mass is called stochastically stable. Thus the minimum resistance determines the stable set, and the coefficients determine the limiting weights within it. The leading powers depend on the revision and mutation mechanism. An ordering of payoffs alone does not provide them.
This argument is a direct finite-state counterpart to resistance and stochastic potential in Young's theory of regular perturbed Markov processes \cite{young-conventions}. Mutation mechanisms with weights decaying faster than all powers may require a different asymptotic scale. Aperiodicity is needed for general convergence from arbitrary initial laws along discrete time. The stationary weights in the tree theorem are defined for every irreducible kernel.
\section{Eliminating transient excursions}
The state set of a population chain can be large. An exact reduction separates states recurrent under no mutation from transient excursion states. Suppose \(P^\epsilon\to P^0\) and write \(E=K\amalg T\), where \(K\) is the set of recurrent states of \(P^0\) and \(T\) is the set of transient states. Define successive visits to \(K\) by counting each ordinary time step spent in \(K\), including consecutive time steps there. With matrix blocks indexed by the starting and ending sets, this trace kernel is
\begin{equation}
\widehat P^\epsilon
= P^\epsilon_{KK}
+ P^\epsilon_{KT}(I-P^\epsilon_{TT})^{-1}P^\epsilon_{TK}.
\label{eq:trace-kernel-egt}
\end{equation}
If \(T\) is empty, only the first term is needed. The geometric series for the inverse sums incomplete excursions according to the number of steps that stay in \(T\). For each \(m\geq0\) the term with \(m\) such steps follows the block sequence
\[
\begin{tikzcd}[column sep=large]
K \arrow[r,"P^\epsilon_{KT}"{above}] &
T \arrow[r,"(P^\epsilon_{TT})^m"{above}] &
T \arrow[r,"P^\epsilon_{TK}"{above}] &
K .
\end{tikzcd}
\]
Transience makes the series converge at \(\epsilon=0\), since the probability of staying in \(T\) decays geometrically in a finite chain. The inverse therefore exists and is continuous for sufficiently small \(\epsilon\).
The stationary equation on \(T\) is
\(\pi_T^\epsilon=\pi_K^\epsilon P^\epsilon_{KT}
+\pi_T^\epsilon P^\epsilon_{TT}\).
Solving it and substituting into the equation on \(K\) gives
\(\pi_K^\epsilon\widehat P^\epsilon=\pi_K^\epsilon\).
Thus the trace stationary distribution is the normalized restriction of \(\pi^\epsilon\). The trace is irreducible for positive \(\epsilon\), since an original positive-probability path between visited states decomposes into consecutive trace steps. Moreover, \(P^0_{KT}=0\), because each recurrent class is closed at zero mutation. The same stationary identity then gives \(\pi_T^\epsilon\to0\). This permits computing stochastic stability using only the trace, when its transitions satisfy the requisite asymptotics. A trace records return events in the original chain. Its stationary probabilities describe fractions of visits, and the normalization accounts for the time omitted in excursions.
The error in dropping transient mass can be estimated before computing the trace stationary law. For a nonnegative matrix \(B\), let \(\|B\|_\infty\) be its maximum row sum. The stationary equation gives
\[
\pi^\epsilon(T)
\leq \|P^\epsilon_{KT}\|_\infty
\big\|(I-P^\epsilon_{TT})^{-1}\big\|_\infty .
\]
The second factor bounds the expected duration of an excursion from any state in \(T\), and remains bounded as \(\epsilon\to 0\) at fixed population size. For example, when the probability of leaving \(K\) in one update is uniformly \(O(\epsilon)\), the total stationary time outside \(K\) is \(O(\epsilon)\).
\section{Rare steps and fixation}
If the recurrent classes of \(P^0\) are singleton absorbing states and \(P^\epsilon=P^0+\epsilon R+o(\epsilon)\) entrywise, formula~\eqref{eq:trace-kernel-egt} gives
\[
\widehat P^\epsilon=I+\epsilon
\bigl(R_{KK}+R_{KT}(I-P^0_{TT})^{-1}P^0_{TK}\bigr)
+o(\epsilon).
\]
The off-diagonal first-order terms are direct changes between absorbing states and changes that enter \(T\) once at order \(\epsilon\), then reach an absorbing state according to \(P^0\). The latter factor contains the unperturbed fixation probabilities. If several rare steps are needed, first order alone cannot recover all resistances. More generally the trace need not lump several recurrent states into a single class at positive mutation. One can either keep all states of \(K\), check an exact lumpability criterion as in Chapter~\ref{ch:aggregation}, or invoke a separate asymptotic class reduction with its hypotheses.
Call the first-order coefficient \(G\). Nonnegativity of off-diagonal trace probabilities and normalization of their rows imply that \(G\) is a continuous-time generator on \(K\). If it is irreducible, the normalized trace stationary laws converge to its unique stationary law. Indeed, divide their stationary equations by \(\epsilon\) after subtracting the identity term. Every accumulation point annihilates \(G\). Irreducibility makes that accumulation point unique, and compactness gives convergence. This argument identifies the scope of the first-order approximation.
These conclusions address a fixed population size. Chapter~\ref{ch:mean-field} treats a different limiting operation, in which population size increases over a bounded time interval. Neither proof interchanges large population, long time, and vanishing mutation. Such an interchange requires additional bounds, because absorption and mixing time can themselves depend strongly on population size and on mutation.
\par
\chapter{Mean-field limits and interaction structure}
\label{ch:mean-field}
A deterministic evolution can describe the leading motion of a stochastic population over a bounded time interval. The connection is a convergence statement about processes, with a specified scale for the frequency increments and the update rate. It requires bounds on the microscopic drift and on fluctuations. Kurtz gave a general framework for the approximation of differential equations by jump processes \cite{kurtz-limit}. Bena\"im and Weibull studied the approximation and time scales for evolutionary games \cite{benaim-weibull}. The result below isolates assumptions that can be checked by evaluating the local update law.
\section{A generator on the frequency simplex}
Let \(D\subseteq\mathbb R^d\) be a compact convex set and let \(D_N\subseteq D\) be a finite state set. The simplex lattice \(C_N(S)/N\) is the main application. Fix a finite set \(\mathcal V\) of possible jump directions. Write the generator as
\[
\mathcal L_N\varphi(x)
=N\sum_{v\in\mathcal V}\beta_v^N(x)
[\varphi(x+v/N)-\varphi(x)] .
\]
Each intensity \(\beta_v^N\) is nonnegative and is zero whenever the indicated transition is infeasible. Terms of zero intensity are omitted. Suppose
\[
B=\sup_{N,x\in D_N}\sum_v\beta_v^N(x)\|v\|^2<\infty,
\qquad
F_N(x)=\sum_v\beta_v^N(x)v .
\]
The vector \(F_N(x)\) is the instantaneous conditional drift. The rate of quadratic variation from jumps is of order \(N^{-1}\) since each jump squares the factor \(N^{-1}\) and the generator contributes \(N\). If \(F_N\) converges uniformly on its state sets to a field \(F\), Taylor's formula gives \(\mathcal L_N\varphi\to D\varphi[F]\) for a smooth \(\varphi\) with bounded second derivative on a neighborhood of \(D\). The Taylor remainder is at most \(B\sup\|D^2\varphi\|/(2N)\). The convergence of generators on such functions identifies the candidate equation.
\begin{theorem}[Bounded-time approximation]
\label{thm:egt-fluid}
Assume \(F\) is Lipschitz on \(D\) with constant \(L\), and set
\(\delta_N=\sup_{x\in D_N}\|F_N(x)-F(x)\|\). Suppose \(\delta_N\to0\) and \(X_N(0)\to x_0\) in probability. If the solution of \(\dot x=F(x)\) from \(x_0\) remains in \(D\) on \([0,T]\), then
\[
\sup_{0\leq t\leq T}\|X_N(t)-x(t)\|\longrightarrow0
\quad\text{in probability}.
\]
\end{theorem}
\begin{proof}
The coordinate martingales of the generator give the vector decomposition
\[
X_N(t)=X_N(0)+\int_0^t F_N(X_N(s))\,ds+M_N(t).
\]
For the squared Euclidean norm of the martingale, the compensator of the sum of squared jumps contributes at most \(BT/N\) in expectation up to time \(T\). In detail, the jump \(v/N\) contributes \(\|v\|^2/N^2\) at rate \(N\beta_v^N(X_N(s))\). Their integrated sum bounds \(\mathbb E\|M_N(T)\|^2\). Thus Doob's quadratic inequality, applied to the submartingale \(\|M_N(t)\|\), gives
\[
\mathbb E\sup_{0\leq t\leq T}\|M_N(t)\|^2\leq 4BT/N .
\]
Subtracting the deterministic integral equation and using the Lipschitz bound yields, by Gr\"onwall's inequality,
\[
\sup_{0\leq t\leq T}\|X_N(t)-x(t)\|
\leq e^{LT}
\left(\|X_N(0)-x_0\|+T\delta_N+
\sup_{0\leq t\leq T}\|M_N(t)\|\right).
\]
Every term in parentheses tends to zero in probability.
\end{proof}
The estimate is quantitative once drift and initial-state errors are bounded. The factor \(e^{LT}\) also locates the difficulty of extending the comparison to long time. The theorem controls complete paths on each fixed interval. It does not compare stationary distributions, which involve behavior on a time scale that can increase with \(N\).
For deterministic initial states with error \(a_N=\|X_N(0)-x_0\|\), choose any \(\eta>e^{LT}(a_N+T\delta_N)\). The proof gives
\[
\mathbb P\!\left(\sup_{t\leq T}\|X_N(t)-x(t)\|>\eta\right)
\leq
\frac{4BT}{N\bigl(e^{-LT}\eta-a_N-T\delta_N\bigr)^2}.
\]
The expression separates the initial discrepancy and the systematic drift error from the probability of a fluctuation. Different microscopic kernels can share the same limit \(F\) while differing in \(B\) or in the conditional covariance from Chapter~\ref{ch:markov}.
For an update chain with transition matrix \(P_N\), an independent Poisson clock of rate \(N\) gives the generator \(N(P_N-I)\). The chain is evaluated at the number of clock arrivals by time \(t\), so a unit of macroscopic time corresponds on average to \(N\) updates. A self-loop consumes a clock arrival and produces a zero displacement. This convention makes the drift of the continuous-time chain \(N\) times the mean increment in one discrete update. It also preserves stationary laws because \(\pi N(P_N-I)=0\) is equivalent to \(\pi P_N=\pi\).
\section{Recovering deterministic selection}
Use the birth--death update of Chapter~\ref{ch:markov} at rate \(N\). Its mean field is \(c(n)-x\) by Equation~\eqref{eq:birth-death-drift}. Suppose the weights of present types converge uniformly to bounded strictly positive functions \(f_i(x)\), with a common positive lower bound. The limiting offspring fractions and field are
\[
c_k(x)=\sum_i
\frac{x_i f_i(x)}{\overline f(x)}M_{ik},
\qquad
F_k(x)=c_k(x)-x_k,\qquad
\overline f(x)=\sum_i x_i f_i(x).
\]
If the weights are Lipschitz, the field is Lipschitz. Its components sum to zero, and \(c_k(x)\geq0\) at a face with \(x_k=0\). A direct form of the integral equation is
\[
x_k(t)=e^{-t}x_k(0)+
\int_0^t e^{-(t-s)}c_k(x(s))\,ds .
\]
This exhibits the role of the offspring distribution. Invariance of the simplex also follows by constructing the solution locally as the fixed point of this integral map on paths in the simplex, where it is a contraction on short intervals and maps such paths to such paths. Short intervals can be concatenated because the simplex is compact and the constants can be chosen uniformly. With no mutation, \(F_k(x)=x_k(f_k(x)-\overline f(x))/\overline f(x)\). Because \(\overline f(x)>0\), this has the replicator trajectories after a positive change of time. The rates with respect to calendar time differ by this common factor. With mutation, normalization and transmission are combined in \(c\) and need not give a particular additive mutation convention on the replicator equation.
The distinction between finite and limiting payoffs can be bounded explicitly. For \(n_i>0\) the payoff in Chapter~\ref{ch:markov} satisfies
\[
u_i^N(n)-(Ax)_i=\frac{(Ax)_i-A_{ii}}{N-1}.
\]
If \(f_i(n)=\phi(u_i^N(n))\) with a Lipschitz positive map \(\phi\) on the interval covered by the payoffs, the difference from the weight \(\phi((Ax)_i)\) is \(O(N^{-1})\) uniformly over present types. The absent types contribute zero to the birth distribution. Positivity on the compact payoff interval gives a uniform lower bound for the normalization, so the corresponding drift approximation has order \(N^{-1}\). The martingale estimate controls the stochastic error separately at order \(N^{-1/2}\) in root mean square over fixed times.
\section{When counts close over a graph}
Suppose the population is an \(N\)-vertex weighted graph. For an oriented pair of vertices \(u,v\), let \(\lambda^N_{uv}\geq0\) be a sampling weight with \(\sum_{u,v}\lambda^N_{uv}=1\). A type \(i\) at \(u\) replaces a type \(j\) at \(v\) at rate \(N\lambda^N_{uv}\varphi(i,j)\), for fixed nonnegative bounded biases \(\varphi\). The relevant statistic is the oriented edge frequency
\[
\theta^N_{ij}(z)=\sum_{u,v}\lambda^N_{uv}
\mathbf1_{\{z(u)=i,\ z(v)=j\}}.
\]
The density drift on a labeled state \(z\) is
\(\sum_{i,j}\theta^N_{ij}(z)\varphi(i,j)(e_i-e_j)\), where \(e_i\) is the coordinate unit at \(i\). For a general graph the edge frequencies can vary when the vertex frequencies are fixed. A closed equation for vertex frequencies requires additional information about their evolution.
An explicit sufficient bound exists for regular weighted sampling. Suppose \(\lambda^N_{uv}=W_{uv}/N\), with \(W\) a nonnegative matrix with row sums one, and write \(J_N\) for the matrix with all entries \(1/N\). If \(\mathbf 1_i\) is the vertex indicator of type \(i\), then
\[
\left|\theta^N_{ij}(z)-x_i x_j\right|
= \frac1N\left|\mathbf1_i^{\mathsf T}(W-J_N)\mathbf1_j\right|
\leq \|W-J_N\|_{2\to2}\sqrt{x_i x_j}.
\]
Thus \(\|W-J_N\|_{2\to2}\to0\) is sufficient for uniform closure of the drift. The proof of Theorem~\ref{thm:egt-fluid} still applies to the projected coordinate processes, using the generator on labeled configurations to obtain their compensators. Uniform drift approximation substitutes for the exact Markov property of the projection.
\[
\begin{tikzcd}[column sep=4.65em]
\text{labeled state}\arrow[r,"{\text{observe}}"{above}] &
(x,\theta^N)\arrow[r,"{\text{retain counts}}"{above}] & x
\end{tikzcd}
\]
With uniform sampling of distinct ordered vertices, \(W=(N J_N-I)/(N-1)\). Then \(W-J_N=(J_N-I)/(N-1)\) and its norm is \(1/(N-1)\) for \(N\geq2\). For any graph family satisfying the norm bound, the limiting field is
\[
F(x)=\sum_{i,j}x_i x_j\varphi(i,j)(e_i-e_j).
\]
It is Lipschitz on the simplex. Its coordinates sum to zero and each \(F_k(x)\) has the factor \(x_k\), so the simplex is invariant. A density path on labeled configurations therefore converges to this solution under the initial-state hypothesis of Theorem~\ref{thm:egt-fluid}. The omitted edge data may persist on sparse or strongly clustered graphs. One can retain additional observables or prove a separate estimate of their effect. Chapter~\ref{ch:aggregation} gives the criterion for exact Markov closure at each population size.
\par
\chapter{Stochastic categories and population aggregation}
\label{ch:aggregation}
\section{Kernels as morphisms}
A state description is useful only when its dynamics can be computed at the level of that description. For the exchangeable population rule in Chapter~\ref{ch:markov}, counts describe a Markov chain exactly. In the network rule of Chapter~\ref{ch:mean-field}, the count drift generally depends on edge frequencies. The category of stochastic kernels gives a common language for these two situations.
Let \(\mathbf{FinStoch}\) have finite sets as objects and stochastic kernels as morphisms. A kernel \(K:X\rightsquigarrow Y\) assigns to each \(x\in X\) a probability \(K(x,y)\) on \(Y\). The composite of \(K\) and \(H:Y\rightsquigarrow Z\) has entries
\[
(KH)(x,z)=\sum_{y\in Y}K(x,y)H(y,z).
\]
Matrix products follow the order in which updates are applied, since laws are row vectors. Identity kernels have entries \(\mathbf 1_{\{x=y\}}\). Matrix associativity is the rule for summing over consecutive unobserved states.
The monoidal product \(X\times Y\) models joint state spaces, with product kernels given by
\[
(K\otimes H)\big((x,x'),(y,y')\big)=K(x,y)H(x',y').
\]
This product applies independent random draws to the two coordinates given their current states. The singleton is monoidal unit and terminal object. Every state space has a deterministic copy operation \(\Delta_X(x)=(x,x)\). A kernel \(K\) preserves copy when \(K\Delta_Y=\Delta_X(K\otimes K)\). On the left, a sample from \(K(x,-)\) is copied. On the right, independent samples are drawn from the same row. Comparing off-diagonal probabilities shows that for distinct \(y,z\) one must have \(K(x,y)K(x,z)=0\). Thus each row is a point mass, and \(K\) comes from a function. Every function preserves copy. This is the deterministic subcategory of \(\mathbf{FinStoch}\), an instance of the distinction between general and deterministic morphisms in a Markov category \cite{fritz-markov}.
\section{Maps of dynamics}
A discrete-time system is a pair \((X,P)\) with \(P:X\rightsquigarrow X\). A stochastic map of dynamics from \((X,P)\) to \((Y,Q)\) is a kernel \(K:X\rightsquigarrow Y\) with \(PK=KQ\). The equation says that updating the source and then observing it gives the same law as observing it and applying the target update.
\[
\begin{tikzcd}[column sep=large]
X \arrow[r,"P"{above}]\arrow[d,"K"{left}] & X\arrow[d,"K"{right}]\\
Y \arrow[r,"Q"{above}] & Y
\end{tikzcd}
\]
Iteration gives \(P^nK=KQ^n\) for every \(n\). If \(QH=HR\) as well, then \(P(KH)=(KH)R\), so these maps compose. Equivalently, a system is a functor \(B\mathbb N\to\mathbf{FinStoch}\), where \(B\mathbb N\) has one object and the monoid of nonnegative time steps as its endomorphisms. A stochastic map of dynamics is a natural transformation. The categorical equation for all times is generated by its component at time one.
If \(\mu P=\mu\), then \((\mu K)Q=\mu PK=\mu K\), so the map transports stationary laws. Independent dynamics can also be composed in parallel. For two maps satisfying \(P_iK_i=K_iQ_i\), the kernel \(K_1\otimes K_2\) intertwines \(P_1\otimes P_2\) and \(Q_1\otimes Q_2\). Here the tensor products of the transitions represent synchronous updates with conditional independence.
\section{Strong lumpability}
Let \(q:X\to Y\) be a surjection recording a deterministic statistic, and write \(J(x,y)=\mathbf 1_{\{q(x)=y\}}\). Its values identify blocks \(B_y=q^{-1}(y)\). Strong lumpability requires that the observed process \(q(X_n)\), for a \(P\)-chain \(X_n\), have a fixed Markov update on \(Y\) for every initial law on \(X\). This notion has a block characterization \cite[\S6.3]{kemeny-snell}.
\begin{theorem}[Strong lumpability]
For a kernel \(P\) on \(X\), the following are equivalent.
\begin{enumerate}
\item There is a single kernel \(Q\) governing the process \(q(X_n)\) for every initial law.
\item There is a kernel \(Q\) on \(Y\) satisfying \(PJ=JQ\).
\item For every \(y,y'\) the sum \(\sum_{x'\in B_{y'}}P(x,x')\) is constant for \(x\in B_y\).
\end{enumerate}
When these conditions hold, \(Q\) is unique and \(Q(y,y')\) is the common value in the third condition.
\end{theorem}
\begin{proof}
For \(x\in B_y\), the \((x,y')\)-entries of \(PJ\) and \(JQ\) are the block sum and \(Q(y,y')\), respectively. This proves equivalence of the last two conditions because the sums give nonnegative rows of total mass one. Given the original process history, the conditional probability that \(q(X_{n+1})=y'\) is the block sum for \(X_n\). Under the third condition it is \(Q(q(X_n),y')\). Conditioning on the observed history leaves that value unchanged, proving the first condition. Conversely, initialize at a point \(x\) and use the proposed update \(Q\) at time one. Its entry from \(q(x)\) to \(y'\) must equal the block sum from \(x\), hence the block sum agrees for all points in the block.
\end{proof}
A Markov description obtained for one specified initial law can depend on the microscopic conditional distributions within the blocks. This weaker statement does not imply an intertwining for deterministic \(J\). For strong lumpability the intertwining suffices to give both the distributional identity \(\mu P^n J=(\mu J)Q^n\) and the assertion about the full observed process in the theorem.
The quotient also has a factorization property for stochastic maps of dynamics. Suppose \(H:(X,P)\to(Z,R)\) intertwines the updates and its rows are constant on each \(q\)-fiber. Then the unique kernel \(\overline H\) defined by those common rows satisfies \(H=J\overline H\). From \(PH=HR\) and \(PJ=JQ\) one gets \(JQ\overline H=J\overline H R\). Surjectivity of \(q\) lets one cancel \(J\) since left multiplication by \(J\) simply repeats rows. Thus \(Q\overline H=\overline H R\). Every observation of \(X\) determined by the quotient statistic, and compatible with its own time update, factors through the quotient as a map of dynamics.
\begin{proposition}[Equivariance gives a quotient]
If a group \(G\) acts on \(X\) and \(P(gx,gx')=P(x,x')\) for every \(g\), the orbit map \(X\to X/G\) is strongly lumpable.
\end{proposition}
\begin{proof}
Every target orbit \(B\) is preserved bijectively by \(g\), so
\[
\sum_{x'\in B}P(gx,x')=\sum_{u\in B}P(gx,gu)
=\sum_{u\in B}P(x,u).
\]
The sums therefore depend only on the source orbit.
\end{proof}
For site-independent replacement and permutation-invariant payoffs, site permutations act equivariantly on the labeled chain. Their orbits are precisely the configurations with a given strategy-count vector, recovering the population quotient in Chapter~\ref{ch:markov}. In spatial dynamics the relevant permutation group is typically the automorphism group of the weighted interaction graph. Its orbits may record information beyond counts, such as whether a pair of strategy classes frequently meet across graph edges.
\section{Generators and approximate quotients}
A continuous-time finite-state chain has generator \(L_X\) with nonnegative off-diagonal entries and zero row sums. A deterministic quotient intertwines the semigroups at every time if and only if \(L_XJ=JL_Y\). Differentiation at zero gives necessity. For sufficiency the generator equation gives \(L_X^kJ=JL_Y^k\) by induction, and the result follows from the exponential series. Thus a quotient generator exists when each generator block sum is constant on a source block. For off-diagonal blocks the contributing entries are nonnegative. Row sums of the quotient generator are zero by those of \(L_X\). This condition can be checked on jump intensities directly, including when the total holding rate varies among states of a block.
When block sums almost agree, an error estimate controls the effect on a fixed observation horizon. For a proposed kernel \(Q\) on \(Y\), assume
\[
\sup_{x\in X}\big\|P(x,-)J-Q(q(x),-)\big\|_{\mathrm{TV}}
\leq \varepsilon,
\]
where total variation is one half the \(\ell^1\) norm. Writing \(E=PJ-JQ\), an exact telescoping identity gives
\[
\mu P^nJ-\mu J Q^n
=\sum_{j=0}^{n-1}\mu P^{n-1-j} E Q^j.
\]
The row vector before \(E\) is a probability, so the norm of its product with \(E\) is at most \(\varepsilon\) by convexity. Propagation under \(Q^j\) is a contraction in total variation, giving a bound of \(n\varepsilon\) for every starting law \(\mu\). This estimate concerns the one-time observed laws and is uniform on a specified range of time steps. A stationary error estimate requires more information, such as a quantitative contraction for the quotient dynamics. In the limit theorem of Chapter~\ref{ch:mean-field}, the target is a deterministic flow and the uniform drift error is what enters the finite-time comparison.
\par
\bookpart{IV}{Games and dynamics with interfaces}
\chapter{Open games and the continuation interface}
\label{ch:open-games}
Classical evolutionary game theory starts from payoffs on an entire
population interaction. A component description needs an interface for
an interaction whose surroundings are still unspecified. Open games
supply that interface in the form of an input and a continuation. The
continuation tells the component how each possible output would be
evaluated by its surroundings. This makes off-equilibrium choices
available for composition \cite{ghani-open-games}.
\section{Lenses as the deterministic interface}
Use \((X,S)\) for an interface with forward type \(X\) and backward
type \(S\). A lens from \((X,S)\) to \((Y,R)\) consists of two maps
\[
v:X\to Y,\qquad u:X\times R\to S.
\]
Its forward action sends a state \(x\) to \(v(x)\). Its backward action
translates a return value \(r\) using the original state \(x\). In game
semantics a return value may contain utility information. The lens
axioms here are the composition and identity laws for these pairs.
For a second lens \((v',u'):(Y,R)\to(Z,Q)\), the composite has forward
map \(v'v\) and backward map
\[
(x,q)\longmapsto u\bigl(x,u'(v(x),q)\bigr).
\]
For three lenses, both parenthesizations substitute the forward values
in the same order and then the backward values in reverse order.
The lens \((\operatorname{id}_X,(x,s)\mapsto s)\) is a two-sided identity.
This gives an ordinary category. The product of two lenses uses
cartesian products on both types and applies the two maps separately.
A continuation on the target is an arbitrary function \(k:Y\to R\).
An input at the source is an element \(x\in X\). A lens sends an
input forward by \(v\) and transports a continuation backward by
\[
k\longmapsto \bigl(x\longmapsto u(x,k(v(x)))\bigr).
\]
The backward transport reverses composition by direct substitution.
Keeping \(k\) as a function matters. A single value \(k(v(x))\) gives
a return only at the chosen output. A component deciding whether to
deviate may need the value at a different output.
\section{Strategies and contextual responses}
An open game \(\mathcal G:(X,S)\to(Y,R)\) consists of a strategy set
\(\Sigma_{\mathcal G}\), a play map \(P_{\mathcal G}\), a coplay map
\(C_{\mathcal G}\), and best-response relations:
\[
\begin{split}
P_{\mathcal G}&:\Sigma_{\mathcal G}\times X\to Y,\\
C_{\mathcal G}&:\Sigma_{\mathcal G}\times X\times R\to S,\\
B_{\mathcal G}(x,k)&\subseteq \Sigma_{\mathcal G}\times
\Sigma_{\mathcal G}.
\end{split}
\]
Here \(k:Y\to R\). The strategies are chosen for the whole
component. Fixing a strategy \(\sigma\) gives a lens with maps
\(P_{\mathcal G}(\sigma,-)\) and \(C_{\mathcal G}(\sigma,-,-)\).
A pair \((\sigma,\sigma')\in B_{\mathcal G}(x,k)\) means that
\(\sigma'\) is an admissible response to the incumbent \(\sigma\)
in this context. The equilibrium set consists of the strategies
\(\sigma\) with \((\sigma,\sigma)\in B_{\mathcal G}(x,k)\).
An open game in this generality specifies responses as data.
Its response relation can be constructed from an optimization
problem, a restricted revision protocol, or another stated selection
rule \cite{ghani-open-games}.
For a decision component with observed type \(X\) and choice type
\(Y\), take \(\Sigma=Y^X\) and \(P(\sigma,x)=\sigma(x)\). With real
return type and backward source type \(1\), the standard maximizing
relation allows \((\sigma,\sigma')\) if
\[
k(\sigma'(x))\ \ge\ k(y)\qquad\text{for every }y\in Y.
\]
The map \(C\) is the unique map to \(1\). Existence of a response
requires attainment of a maximum. At a given observed \(x\) this
relation constrains the strategy only at \(x\). A stronger requirement
at all possible observations needs another equilibrium test, as in
subgame analysis. The distinction is visible in the typing of
\(B(x,k)\).
\section{Sequential composition}
Let \(\mathcal H:(Y,R)\to(Z,Q)\) follow \(\mathcal G\), with outer
continuation \(k:Z\to Q\). An incumbent \((\sigma,\tau)\) determines
the intermediate forward value \(y=P_{\mathcal G}(\sigma,x)\).
The downstream strategy determines the continuation seen upstream:
\[
k_{\tau}(y')=
C_{\mathcal H}\bigl(\tau,y',k(P_{\mathcal H}(\tau,y'))\bigr),
\qquad y'\in Y.
\]
The upper path carries states forward. The lower arrow carries
the continuation back across the downstream component.
\[
\begin{tikzcd}[column sep=large]
X \arrow[r,"{P_{\mathcal G}(\sigma,-)}"{above}] & Y
\arrow[r,"{P_{\mathcal H}(\tau,-)}"{above}] & Z\\
& R^Y & Q^Z \arrow[l,"k\mapsto k_\tau"{above}]
\end{tikzcd}
\]
Here \(Q^Z\) and \(R^Y\) denote sets of continuations.
A composite strategy is a pair. Its play and coplay are the
composition of the fixed-strategy lenses, and its response relation is
\[
\begin{aligned}
&\bigl((\sigma,\tau),(\sigma',\tau')\bigr)
\in B_{\mathcal H\circ\mathcal G}(x,k)\\
&\quad\Longleftrightarrow\
(\sigma,\sigma')\in B_{\mathcal G}(x,k_\tau)
\quad\text{and}\quad
(\tau,\tau')\in B_{\mathcal H}(y,k).
\end{aligned}
\]
The two tests use the incumbent context. The second component sees
the output generated by \(\sigma\). The first component evaluates
alternatives using \(\tau\) as a fixed strategy, including its play
and coplay at changed inputs. This is a prescription for testing
responses with the other components' strategies held fixed.
\begin{proposition}[Associativity of the response construction]
Sequential composition is associative under the canonical
reassociation of strategy products. Singleton-strategy identity
lenses with the sole identity response give identities.
\end{proposition}
\begin{proof}
Write \(\mathcal J:(Z,Q)\to(W,T)\) for the third component and
\(\upsilon\) for its incumbent. For an outer continuation
\(\ell:W\to T\), put
\[
k_{\upsilon}(z')=
C_{\mathcal J}\bigl(\upsilon,z',\ell(P_{\mathcal J}(\upsilon,z'))\bigr).
\]
With \(y=P_{\mathcal G}(\sigma,x)\) and
\(z=P_{\mathcal H}(\tau,y)\), either parenthesization tests the
three responses in the contexts
\[
\begin{array}{rl}
\mathcal G:&
\bigl(x,\ y'\mapsto C_{\mathcal H}
(\tau,y',k_{\upsilon}(P_{\mathcal H}(\tau,y')))\bigr),\\
\mathcal H:& (y,k_{\upsilon}),\\
\mathcal J:& (z,\ell).
\end{array}
\]
For the first parenthesization these contexts result from first
transporting through \(\mathcal J\) and then through \(\mathcal H\).
For the second, the coplay of the downstream lens composite expands
to the same substitution. The play and coplay agree by lens
associativity. Inserting an identity leaves the transported state and
continuation unchanged and adds the sole relation on a singleton.
\end{proof}
Taking strategy sets up to reindexing by these coherent bijections
gives an ordinary category. Keeping actual strategy sets makes the
reassociation visible as a comparison between games. The next chapter
also keeps reindexings that need not be invertible.
\section{A compositional equilibrium test}
The diagonal of the response formula gives the equilibrium criterion
\[
\sigma\in E_{\mathcal G}(x,k_\tau),
\qquad
\tau\in E_{\mathcal H}(P_{\mathcal G}(\sigma,x),k),
\]
where \(E_{\mathcal G}\) denotes the contextual equilibrium set.
This is a system of mutually dependent tests. Retaining only
\(E_{\mathcal G}\) is enough for this particular diagonal calculation
when all contexts and the fixed-strategy lenses are retained. Off-diagonal
responses are needed to study transformations that preserve response
possibilities or to introduce an update rule using those possibilities.
For evolutionary models the selected continuation can include
frequency-dependent payoffs. One must still specify how the current
population determines that continuation, which strategy information is
available, and how admissible responses change frequencies. The
interface supplies a way to keep these choices explicit.
For instance, an adapter can expose the population state as an input
and compute a payoff vector as its output. A downstream selection
component then evaluates each strategy against that vector.
Compositional optimization tests whether a strategy passes the
resulting response relation. A dynamical model adds a state transition
or vector field on populations, whose state change is an additional
map. These two observations about the system can then be checked for
agreement under the hypotheses of Chapter~\ref{ch:lyapunov}.
\section{Deterministic adapters}
The response formula gives a useful rule for changing the language in
which an interface is described. Suppose
\(L:(X',S')\to(X,S)\) and \(M:(Y,R)\to(Y',R')\) are lenses.
Regard each as a singleton-strategy game with its sole response.
For a context \((x',k')\) of \(M\circ\mathcal G\circ L\), the
response test of the middle game has context
\[
\left(v_L(x'),\ y\longmapsto u_M(y,k'(v_M(y)))\right).
\]
This follows by first applying the lens substitution to the return
from \(M\), then carrying the input forward through \(L\). The
singleton comparisons impose no further response condition, and the
same formula therefore computes equilibria of the adapted game.
If a downstream adapter translates every return by the same amount
for the relevant decision context, a real maximizing response test
is unchanged. More generally a strictly increasing transformation
of every return in that test preserves the ordering of alternatives.
The key hypothesis here is a common transformation of the returns
being compared. A translation depending on the output may alter
their order and must be included in the continuation itself.
\par
\chapter{Parallel games and higher comparisons}
\label{ch:compositional-games}
Sequence describes a change of context through an intermediate
interface. Parallel composition retains two interfaces and asks how
the environment evaluates their joint output. A joint continuation
can couple the decisions even when their play maps factor.
\section{Parallel composition}
For \(\mathcal G:(X,S)\to(Y,R)\) and
\(\mathcal H:(U,T)\to(V,Q)\), their product has object types
\((X\times U,S\times T)\) and \((Y\times V,R\times Q)\), and
strategy set \(\Sigma_{\mathcal G}\times\Sigma_{\mathcal H}\).
Its play and coplay are the products of the fixed-strategy lenses.
Given a context with forward state \((x,u)\) and continuation
\(k:Y\times V\to R\times Q\), an incumbent \((\sigma,\tau)\)
produces \(y_* = P_{\mathcal G}(\sigma,x)\) and
\(v_* = P_{\mathcal H}(\tau,u)\). Define the two continuations
seen by the components as
\begin{align*}
k_1(y)&=\pi_R k(y,v_*),&
k_2(v)&=\pi_Q k(y_*,v).
\end{align*}
The parallel best-response relation requires
\((\sigma,\sigma')\in B_{\mathcal G}(x,k_1)\) and
\((\tau,\tau')\in B_{\mathcal H}(u,k_2)\).
A trial change in one component thus sees the same incumbent
output from the other. In particular, even a product of games
whose input states are independent can have coupled equilibrium
conditions when the continuation is joint. If
\(k(y,v)=(r(y),q(v))\), the continuations are \(r\) and \(q\)
independently of \(\sigma,\tau\), so the equilibrium sets multiply.
\begin{proposition}[Interchange]
Parallel composition commutes with sequential composition up to
the canonical exchange of strategy factors. Its associativity,
unit and symmetry are given by the corresponding product
bijections.
\end{proposition}
\begin{proof}
Consider two chains, \(\mathcal G\) followed by \(\mathcal H\)
and \(\widetilde{\mathcal G}\) followed by
\(\widetilde{\mathcal H}\), with incumbent strategies
\(\sigma,\tau\) and \(\widetilde\sigma,\widetilde\tau\),
respectively. Let \(y_*,\widetilde y_*\) be the intermediate
outputs, \(z_*,\widetilde z_*\) the final outputs, and \(k\) the
joint outer continuation. In either order of assembly the test at
\(\mathcal H\) uses \(y_*\) and \(z\mapsto\pi_1 k(z,\widetilde z_*)\).
The test at \(\mathcal G\) uses the continuation
\[
y\longmapsto C_{\mathcal H}
\bigl(\tau,y,\pi_1 k(P_{\mathcal H}(\tau,y),\widetilde z_*)\bigr).
\]
Indeed, composing within the first chain propagates the first
outer continuation through \(\mathcal H\). Composing across the
parallel pair first propagates \(k\) through the product lens,
and then freezes the second intermediate value at
\(\widetilde y_*\). Its downstream output is precisely
\(\widetilde z_*\). The formulas for the other chain follow by
using \(\pi_2\) and holding \(z_*\) fixed. Thus all four response
tests agree. The play and coplay maps agree because composition of
product lenses is componentwise.
For any number of parallel factors the continuation for one
factor fixes all other incumbent outputs and projects to the
return for that factor. This gives the same formula for either
parenthesization and after permutation of factors. The singleton
unit introduces neither another output choice nor another return
choice. Coherence follows from the corresponding equalities of
tuples.
\end{proof}
\section{Strategy maps and two-dimensional structure}
One can preserve more information by keeping morphisms between
games. For games \(\mathcal G,\mathcal G'\) on the same interface,
take a map \(\phi:\Sigma_{\mathcal G}\to\Sigma_{\mathcal G'}\)
that preserves play and coplay at every state and return. Require
also that
\[
(\sigma,\eta)\in B_{\mathcal G}(x,k)
\ \Longrightarrow\
(\phi(\sigma),\phi(\eta))\in B_{\mathcal G'}(x,k).
\]
This gives a category for each pair of interfaces. Composition of
strategy maps preserves the relation by transitivity of implication,
and identity maps preserve all the specified data. Products of
these maps preserve the sequential and parallel response
formulas. For the sequential case, play preservation gives
the same intermediate incumbent state and coplay preservation
gives the same transported continuation. One can then apply
response preservation separately in the two games. The parallel
argument uses preservation of the two incumbent outputs.
This yields a bicategorical description after retaining the
canonical product reassociations as invertible cells.
For a more general morphism allowing lenses between the
boundary interfaces, see the double category of open games
in \cite{hedges-morphisms}.
For sequential composition, the diagram below holds at every
upstream state \(x\). Set \(E_x(\sigma,\tau)=
P_{\mathcal H}(\tau,P_{\mathcal G}(\sigma,x))\), where
\(Z\) is the final forward output space. Primes denote the
corresponding maps for the target games.
\[
\begin{tikzcd}[column sep=large,row sep=large]
\Sigma_{\mathcal G}\times\Sigma_{\mathcal H}
\arrow[r,"E_x"{above}]\arrow[d,"\phi\times\psi"']
& Z\arrow[d,equal]\\
\Sigma_{\mathcal G'}\times\Sigma_{\mathcal H'}
\arrow[r,"E'_x"{above}]
& Z
\end{tikzcd}
\]
The backward returns commute by coplay preservation. These
two identities compare the boundary behavior independently of
whether a pair of strategies satisfies the response tests.
Response-preserving maps carry equilibria to equilibria:
one applies the implication to \((\sigma,\sigma)\).
A noninvertible map can identify equilibria or fail to reach
some equilibria of the target. An isomorphism of games
instead preserves and reflects the response tests in every
context. The geometric information carried by a morphism is
therefore different from a claim about the size or structure
of its image on equilibrium sets. This distinction matters
when strategy representations are quotiented or multiple
agents share an interface.
\section{Probability and continuation transport}
For a fixed strategy, one may model forward play by a
probability kernel. For finitely supported probabilities
\(P(y\mid x)\) and \(Q(z\mid y)\), their forward composite is
\[
(QP)(z\mid x)=\sum_y P(y\mid x)\,Q(z\mid y).
\]
This has unit given by a point mass at the input. Associativity
is the reassociation of the joint sum over intermediate values.
If a continuation takes values in \(\mathbb R\), this kernel
transports it by expectation. Indeed
\[
\sum_y P(y\mid x)\sum_z Q(z\mid y)k(z)
=\sum_z (QP)(z\mid x)k(z).
\]
Thus a component that compares expected returns can use the
transported function of its possible inputs. Existence of a
maximizing strategy is again an additional hypothesis.
Stochastic parallel composition requires a decision about joint
laws. Tensoring two kernels using independent randomness gives
the product law conditional on their inputs. That tensor does
not have the universal property of a cartesian product in
stochastic maps. A joint distribution is generally not determined
by its two marginals. For example, a nondegenerate finite law
paired with itself by sharing its sample and the product of
two independent copies of the law have the same marginals.
The continuation may detect their dependence. Consequently
continual substitution at the level of marginal expected
payoffs alone can lose information relevant for later decisions.
Bayesian open games supply additional structure for stochastic
play, hidden information and best responses
\cite{bolt-bayesian}. The calculation above describes only the
forward-kernel and expected-return part of that framework.
\section{An equilibrium interface and a revision rule}
Response relations are evaluated at a context rather than at a
time. To extract a trajectory one supplies a protocol for when
revisions occur, which states agents observe, and how allowed
revisions become a state update. For a fixed context, let
\(B(\sigma)\) denote the set of responses to \(\sigma\).
If it is nonempty at every incumbent, a deterministic revision
rule can choose a single \(U(\sigma)\in B(\sigma)\) at each
incumbent. A fixed point of \(U\) is an equilibrium because its
selected response is itself. The converse holds if the rule
retains the incumbent whenever \(\sigma\in B(\sigma)\).
The relation alone leaves this decision open, as well as
the choice among other responses and the possible regularity
of the selector. A protocol can instead attach a probability
law to responses, which adds another layer of dynamical data.
In continuous time a smooth population
vector field \(F\) and its positive rescaling \(aF\) by a smooth
function \(a\) have the same rest points. On any interval over
which a solution of one field exists and the scalar remains
positive, its trajectory can be reparametrized in time to a
solution of the other. This follows from the chain rule by
solving for the derivative of the new time parameter to equal
\(a\) along the trajectory. Quantities measured at specified
times can nevertheless change. A contextual equilibrium
criterion does not specify the clock, the distribution of
samples, or the size of a discrete update. These data enter
the dynamical interface in the next chapter.
\par
\chapter{Open dynamics and network wiring}
\label{ch:open-dynamics}
An open dynamical system specifies how its current state changes
under an input from its surroundings. Its output can supply another
system with an input. A wiring diagram determines which values
are passed and, when species or states are identified, where their
rates of change are combined. Different choices of interface give
different composition rules.
\section{Input and output ports}
Consider a smooth state space \(M\), an input space \(A\), and
an output space \(B\). Write a module as
\[
v:M\times A\longrightarrow TM,\qquad h:M\to B,
\qquad v(x,a)\in T_xM .
\]
The state equation is \(\dot x=v(x,a)\) and its output is
\(h(x)\). A single time-independent wiring of modules indexed
by \(i\) can be expressed by maps
\[
w:A_{\mathrm{ext}}\times\prod_i B_i\to\prod_i A_i,
\qquad
r:\prod_i B_i\to B_{\mathrm{ext}} .
\]
At state \(\boldsymbol x=(x_i)_i\), put
\(\boldsymbol b=(h_i(x_i))_i\) and
\(\boldsymbol a=w(a_{\mathrm{ext}},\boldsymbol b)\).
The new state velocity has components \(v_i(x_i,a_i)\) and
the output is \(r(\boldsymbol b)\). A feedback wire can carry
\(h_i(x_i)\) to the same or another module. As the output depends
only on the current state, closing such a wire requires no
simultaneous algebraic solution for the inputs.
For nested networks the rule is associative: to compute one
component \(v_i\), evaluate the elementary outputs, substitute
them into the wiring maps of each enclosing network, and apply
the resulting input to \(v_i\). Flattening the network substitutes
the same functions into one composite. Associativity of function
composition and the canonical comparison of product state spaces
give equality of the component velocities. The same calculation
applies to outputs. This is the readout form of the wiring algebra
developed in \cite{vagner-wiring}. If a readout is also allowed to
depend on its instantaneous input, a directed loop may require
a separate condition guaranteeing that its algebraic equations
have a unique solution.
\section{Replicator modules}
For a population on a strategy simplex, expose a state readout
and accept an input \(a\) that parametrizes a payoff field.
In coordinates the local velocity is
\[
v_i(p,a)=p_i\left(f_i(p,a)-\sum_j p_jf_j(p,a)\right).
\]
The velocity sums to zero, and \(v_i=0\) on the face \(p_i=0\),
for every \(a\). Consequently feedback through the payoff input
preserves these algebraic identities. With smooth, locally
Lipschitz coefficients on a neighborhood of the simplex, the
wired vector field has unique local solutions. The face
identities and uniqueness prevent a coordinate from crossing
zero in finite time, and the mass identity preserves the
normalization. The same argument works for separately normalized
populations on a product of simplices, since the identity applies
to each local collection of coordinates. This is one way to
give a dynamical meaning to interactions between modules.
A relation that recognizes mutually admissible responses is a
separate structure, as in Chapter~\ref{ch:open-games}.
For a varying external input the solution concept requires
regularity of the input in time and local integrable bounds
for the coefficients. Composition of wiring gives a
formula for the field under those assumptions. It gives no
single guarantee of convergence to a selected equilibrium.
Even on a product of compact simplices the connections can
couple the fields and alter stability. With a fixed external
input and smooth wiring, compactness does ensure continuation
of solutions for all forward times.
\section{Composition by state identification}
For chemical reactions and transport, a different interface uses
labels for shared species or sites. Represent an open network
by a cospan \(I\to S\leftarrow O\) of finite sets. Its internal
field is \(v_S:\mathbb R^S\to\mathbb R^S\) on concentrations,
or the restriction to nonnegative concentrations. A map
\(j:S\to P\) induces pullback \(j^*\) which duplicates concentrations
and pushforward \(j_*\) which sums their velocities. The
transported field on \(P\) is \(j_*v_Sj^*\).
Suppose \(I\xrightarrow{i} S\xleftarrow{o} O\) is followed by
\(O\xrightarrow{i'} T\xleftarrow{o'} J\). The new species set
is the pushout. Denote the structure maps into it by \(j_S,j_T\).
The field of the composite is
\[
v_P = (j_S)_*v_Sj_S^* + (j_T)_*v_Tj_T^* .
\]
Every species identified by a shared port gets the velocity
contributed by both components. For a third composable cospan,
the two iterated pushouts are canonically isomorphic. In the
common pushout, the contributions from each original module are
the transported fields along its composite structure map. This
follows from \((kj)_*=k_*j_*\), \((kj)^*=j^*k^*\), and distributivity
of linear pushforward over addition. The zero field on the
identity cospan is a unit. Disjoint union puts two networks
side by side without adding identifications between them.
For a mass-action reaction with input complex
\(\alpha\in\mathbb N^S\), output complex
\(\beta\in\mathbb N^S\), and rate constant \(k\), its field
is \(v_S(x)=k x^\alpha(\beta-\alpha)\). The identity
\((j^*x)^\alpha=x^{j_*\alpha}\) shows term by term that
transport of the field is the field for the reaction with
complexes \(j_*\alpha,j_*\beta\). Therefore gluing networks
and then taking the mass-action equations gives the same
field as gluing the module fields. The general construction
of \cite{baez-pollard-reactions} retains boundary interfaces and
gives a functor from open reaction networks to open
dynamical systems.
The same rule applies to linear master equations with a
column-conservative generator \(Q\) on \(\mathbb R^S\).
The induced matrix \(j_*Qj^*\) has nonnegative off-diagonal
entries since those entries sum coefficients between disjoint
fibers of \(j\). Its columns sum to zero since
\(\boldsymbol 1^\top j_*=\boldsymbol 1^\top\) on the source.
Thus gluing in this sense preserves the generator conditions.
A quotient of an existing time evolution needs an
intertwining condition in addition. Open Markov processes
and their coarse-graining are studied in
\cite{baez-courser-markov}.
Linear conservation laws also pass through the pushout when
their coefficients agree on the shared species. Suppose
\(\ell_S^{\mathsf T}v_S=0\) and \(\ell_T^{\mathsf T}v_T=0\)
pointwise, and \(o^*\ell_S=(i')^*\ell_T\). There is a unique
coefficient vector \(\ell_P\) restricting to both local vectors.
The identity \(\ell_P^{\mathsf T}(j_S)_*
= (j_S^*\ell_P)^{\mathsf T}\), and its counterpart for \(T\),
annihilate the two terms of the composite velocity separately.
Hence \(\ell_P^{\mathsf T}x\) is constant along the closed flow.
When boundary fluxes are supplied, its derivative equals
the weighted incoming flux minus the weighted outgoing flux.
This compatibility criterion applies to conserved species weights
even when the components use different internal coordinates.
\section{Boundary behavior of steady states}
A steady state with boundary fluxes \(a\in\mathbb R^I\) and
\(d\in\mathbb R^O\) satisfies
\[
v_S(x)+i_*a-o_*d=0.
\]
Its external data are the concentrations \(i^*x,o^*x\)
and the fluxes \(a,d\). They define a relation between boundary
data. The relation of the composite is obtained by composing
the two relations across the shared concentrations and fluxes.
Here is the exact reason.
The pushout diagram
\[
\begin{tikzcd}
O\arrow[r,"i'"{above}]\arrow[d,"o"']&T\arrow[d,"j_T"]\\
S\arrow[r,"j_S"{above}]&P
\end{tikzcd}
\]
means that a concentration vector on \(P\) is equivalent to
two vectors on \(S,T\) agreeing after pullback to \(O\).
The shared flux \(d\) enters the two steady equations with
opposite signs. Applying \((j_S)_*,(j_T)_*\) and adding
cancels it and gives the composite steady equation.
Conversely suppose the composite steady equation holds and
restrict the concentrations to \(S,T\). Form the two local
residuals including their external fluxes but omitting the
shared flux. These residuals lie in the kernel of the sum
of the two pushforwards to \(P\). The sequence
\[
\mathbb R^O
\xrightarrow{(o_*,-i'_*)}
\mathbb R^S\oplus\mathbb R^T
\xrightarrow{(j_S)_*+(j_T)_*}
\mathbb R^P\longrightarrow 0
\]
is exact since the free vector space functor preserves the
pushout. Thus the residuals are obtained from some common
flux with opposite signs, giving two local steady equations.
This proves the relation formula for unrestricted real port
fluxes. Additional inequalities on fluxes require their own
feasibility check for such a factorization. The relation
records possible external observations of steady states.
Trajectories and the conditions for reaching those states
remain part of the dynamical description.
\par
\chapter{Semiconjugacy and deterministic aggregation}
\label{ch:semiconjugacy}
An aggregate description of selection groups several strategies
under the same observed label. Its exactness is a differential
question. For every microscopic population with a given aggregate,
the instantaneous aggregate velocity must agree. A coarse field
that fits a single trajectory need not describe nearby microscopic
states with the same observed population.
\section{Flow maps and projectable fields}
Let \(F\) and \(G\) be smooth vector fields on \(M\) and \(N\) and
let \(q:M\to N\) be smooth. We call \(q\) a \emph{semiconjugacy}
when it carries solution curves of \(F\) to solution curves of
\(G\) with the same time parameter. This is equivalent to
\begin{equation}
Dq_x F(x)=G(q(x))\qquad (x\in M).
\label{eq:exact-semiconjugacy}
\end{equation}
Indeed, for a source solution \(x(t)\), the chain rule makes
\(q(x(t))\) a solution of the target initial-value problem.
Uniqueness then gives commutation of the two flow maps on their
common time domain. Conversely, differentiating this commutation
at zero gives \eqref{eq:exact-semiconjugacy}. On simplices we use
smooth fields on relative neighborhoods whose forward flows
preserve the simplices. In this case the derivative at zero may
be taken from the right.
\[
\begin{tikzcd}[column sep=large,row sep=large]
M \arrow[r,"\Phi_t"{above}] \arrow[d,"q"']
& M \arrow[d,"q"]\\
N \arrow[r,"\Psi_t"{above}]
& N
\end{tikzcd}
\]
A complete forward flow determines a functor
\(B(\mathbb R_{\geq 0},+)\to\mathbf{Set}\). Here \(B\) makes
the monoid of nonnegative times into a one-object category, and
the functor sends addition of times to composition of evolution
maps. The square says precisely that \(q\) is the component
of a natural transformation. Smooth semiconjugacies form a
category with ordinary identities and composition. The naturality
condition uses a common clock. A state-dependent time change such
as the positive rescaling in Chapter~\ref{ch:compositional-games}
requires a separate description.
Given \(F\) and a surjective submersion \(q\), a suitable \(G\)
exists exactly when \(Dq_xF(x)\) is constant on every fiber
\(q^{-1}(y)\), as a vector in \(T_yN\). Necessity is immediate
from \eqref{eq:exact-semiconjugacy}. For sufficiency, define the
candidate at \(y\) to be this common value. A local smooth section
of the submersion expresses it as a smooth field on a neighborhood
of \(y\), and the expressions from different sections agree.
Surjectivity also gives uniqueness. For an affine quotient of
simplices by a partition, a global affine section is available.
Choose a probability vector supported in each block and lift
coarse masses to these fixed proportions. Evaluating \(Dq F\)
on that section gives the smooth candidate whenever the
fiber condition holds. The candidate preserves the coarse
simplex whenever the original flow preserves the fine simplex,
by applying the chain rule to a source solution through
a lift of each coarse initial state.
\section{Partition quotients of selection}
Let \(b:S\twoheadrightarrow T\) be a partition map and let
\(B_{\alpha i}\) be \(1\) if \(b(i)=\alpha\), and \(0\)
otherwise. Write \(z=Bp\) for the image of the population
\(p\), so \(z_\alpha=\sum_{b(i)=\alpha}p_i\). Let
\(F_i(p)=p_i(f_i(p)-\bar f(p))\) with
\(\bar f(p)=\sum_i p_i f_i(p)\). For every block with
\(z_\alpha>0\) define
\[
\widehat f_\alpha(p)
=\frac{1}{z_\alpha}\sum_{b(i)=\alpha}p_i f_i(p).
\]
Summing the microscopic equations and regrouping the mean gives
\[
(BF(p))_\alpha
= z_\alpha\left(\widehat f_\alpha(p)
-\sum_{\beta:z_\beta>0}z_\beta\widehat f_\beta(p)\right)
\]
on the active blocks. The other block velocities vanish, since
all of their microscopic masses are zero.
\begin{proposition}
For a proposed coarse payoff field \(g(z)\), the partition
sends the replicator field for \(f\) to the replicator field
for \(g\) if and only if
\(\widehat f_\alpha(p)-g_\alpha(Bp)\) is independent of \(\alpha\)
among the active blocks for every \(p\).
\end{proposition}
\begin{proof}
Subtract the two velocities on each active block and divide
by its positive mass. The resulting equality requires the
difference between the two payoffs to equal the weighted mean
of those differences on the active blocks. Hence all of
the differences are equal there. A common difference cancels
in the centered payoffs and yields equality of the velocities.
Inactive blocks contribute neither mass nor velocity to either
calculation.
\end{proof}
A sufficient condition is
\(f_i(p)=g_{b(i)}(Bp)+c(p)\) with the same scalar \(c(p)\)
for all strategies. For matrix payoffs, the corresponding
form is \(A=B^{\mathsf T}\bar A B+\mathbf 1 c^{\mathsf T}\),
where \(c\) is a constant vector and \(\bar A\) is the coarse
payoff matrix. The proposition is weaker than pointwise
payoff agreement within each block. Microscopic deviations
from the proposed block payoff can cancel in the population
average. This cancellation is tested at every microscopic
population in the domain.
\section{Comparison with linear lumpability}
For a Markov equation \(\dot p=Qp\) with a column-conservative
generator, the same partition is exact for another generator
\(\overline Q\) if \(BQ=\overline Q B\). The column of
\(BQ\) at a microscopic state \(i\) records the total signed
rate into each coarse block. A matrix \(\overline Q\) exists
precisely when these columns agree for all \(i\) in each
source block. To see this, compare the matrices on each
basis vector. If the equalities of columns hold, define the
coarse column using any microscopic representative of the
block. The sums of its entries vanish because the columns
of \(Q\) sum to zero. Its off-diagonal entries are nonnegative
because the corresponding target blocks contain no source state.
Thus the resulting matrix is a Markov generator. Equivalently,
the intertwining holds for the matrix exponentials at every
nonnegative time, by applying it to each power and summing
the exponential series. The implication in the reverse
direction follows by differentiating the exponentials at
zero. For open Markov processes, preservation of interfaces
and fluxes adds constraints to the condition on the generators
\cite{baez-courser-markov}. In the nonlinear selection equation
the fiber test applies to the centered payoff averages, whose
weights are themselves microscopic state variables.
\section{Observed equilibria and stability}
Semiconjugacy carries a rest point of \(F\) to a rest point of
\(G\). Over a target rest point the condition says
\(Dq_xF(x)=0\), allowing motion tangent to a fiber. If \(V\)
is a Lyapunov function for \(G\), then
\(D(V\circ q)_xF(x)=DV_{q(x)}G(q(x))\). Thus the same
monotonicity is inherited by the pullback, with strictness
only where the coarse Lie derivative is strict. These facts
identify the limit of equilibrium and stability information
that can be recovered from aggregated coordinates alone.
\section{Finite-time error under a nonzero defect}
Suppose the target is represented in a Euclidean domain, as
for an affine simplex. The defect of an observation is
\[
r_q(x)=Dq_xF(x)-G(q(x)).
\]
Suppose \(G\) is \(L\)-Lipschitz on a region containing both
the observed source solution \(q(x(t))\) and a target solution
\(z(t)\) over \([0,T]\). If the defect along the source
solution has norm at most \(\epsilon\), then for
\(0\leq t\leq T\),
\[
\|q(x(t))-z(t)\|
\le e^{Lt}\|q(x(0))-z(0)\|
+\begin{cases}
\epsilon(e^{Lt}-1)/L,& L>0,\\
\epsilon t,& L=0.
\end{cases}
\]
The integral equations for the two curves, followed by the
triangle inequality, bound their distance at \(t\) by its
initial value plus the integral of \(L\) times the current
distance and \(\epsilon\). The integral form of Gronwall's
inequality yields the claim. For polynomial fields, formal
approximate differential equivalences give more specific
ways to find and certify reductions \cite{cardelli-lumping}.
If \(h:N\to P\) compares \(G\) to a third field \(J\), the
chain rule gives
\[
r_{h\circ q}(x)=Dh_{q(x)}r_q(x)+r_h(q(x)).
\]
In particular a bound on \(Dh\) controls the magnification
of the first reduction's defect under composition. The error
estimate uses a specified time interval and hypotheses about
the region visited by the comparison trajectories. Long-time
conclusions need additional information. For example, if the
target satisfies the contraction inequality
\[
\langle u-v,G(u)-G(v)\rangle\leq -\lambda\|u-v\|^2
\qquad (\lambda>0)
\]
on the comparison region, the error in Euclidean norm satisfies
\[
\|q(x(t))-z(t)\|
\leq e^{-\lambda t}\|q(x(0))-z(0)\|
+\frac{\epsilon}{\lambda}(1-e^{-\lambda t}).
\]
For positive distance, differentiate its square, apply the
contraction inequality and Cauchy--Schwarz, and divide by
twice the distance. At distance zero the upper right
derivative of the distance is at most \(\epsilon\), since
the difference of velocities is then the defect itself.
Integrating the resulting differential inequality gives the
bound. If the two trajectories remain in the region, the
bound holds on their whole common forward time domain.
\par
\bookpart{V}{Descent, compression, and learning}
\chapter{Fitness systems over varying contexts}
\label{ch:contextual-games}
A population in an ecological or strategic context uses a set
of available types, a state of its interactions, and a rule for
evaluating prospective behavior. When contexts vary, these data
can have nontrivial transition maps. A model can then be a family
of games without having a single global numbering of the types.
This chapter uses sheaves to state exactly what must agree between
presentations.
\section{An internal strategy object}
Let $B$ be a smooth manifold and let $p:S\to B$ be a smooth
locally constant finite covering, with all fibers nonempty.
Its sheets over an open set give types in that context. The
set of types is a sheaf of finite sets over $B$. Think of an
open subset as a stage on which type labels may be observed.
On a trivializing open set $U$, the probability state space
over $U$ is $U\times\Delta(I_U)$, where $I_U$ indexes the
local sheets and
\[
\Delta(I_U)=\left\{x\in\mathbb R^{I_U}:
x_i\geq0,\quad \sum_i x_i=1\right\}.
\]
On an overlap the sheets are related by a locally constant
permutation $\sigma_{UV}(b):I_V\to I_U$. The transition
$(b,x)\mapsto (b,\sigma_{UV}(b)x)$ carries faces to faces
and preserves both the mass and the nonnegativity conditions.
The transitions assemble a simplex bundle over $B$. Equivalently,
the simplex is a sheaf of states, using smooth plots on each
context. The closed fibers have corners. Smoothness of a
fitness map on such a fiber means that its local components
extend smoothly to a neighborhood in the affine span.
A fitness operator is a section of the relative function object
from the simplex bundle to the bundle of real type-indexed
vectors. Concretely it is a collection of functions
$f_U(b,x)\in\mathbb R^{I_U}$ with
\[
f_U(b,\sigma_{UV}(b)x)=\sigma_{UV}(b)f_V(b,x).
\]
This is a descent condition on maps between bundles. It can be
applied equally to restricted subfamilies of fitness operators.
For instance a collection of pairwise payoff matrices
$A_U(b)$ gives the operators $A_U(b)x$ when the matrices satisfy
$A_U(b)\sigma_{UV}(b)=\sigma_{UV}(b)A_V(b)$. A constant affine shift
of all payoffs leaves the replicator field unchanged, as in
Chapter~\ref{ch:payoff-equivalence}. It can be convenient to
take the corresponding quotient of fitness data before imposing
compatibility.
\section{Local generators and their gluing}
For each context $b$ and a local vector $x$, the replicator
operator assigns
\[
R_{f_U}(b,x)_i=x_i\bigl(f_U(b,x)_i-
\langle x,f_U(b,x)\rangle\bigr).
\]
Its values are tangent to the affine hyperplane of mass one.
Every zero coordinate remains zero in that field. These
identities are pointwise in $b$ and are independent of the
trivialization.
\begin{proposition}[Descent of replicator fields]
\label{prop:replicator-descent}
Under the fitness transition law above, the replicator fields
glue to a vertical vector field on the simplex bundle.
The flow on any fiber is compatible with relabeling and with
restriction to context opens. If the fitness fields are smooth
up to the boundary in the specified sense, the flows exist
for all nonnegative times on each simplex fiber.
\end{proposition}
\begin{proof}
A permutation $\sigma$ preserves the Euclidean pairing,
preserves coordinatewise multiplication, and fixes the
all-ones vector. Therefore
\[
R_{f_U}(b,\sigma x)=\sigma R_{f_V}(b,x).
\]
The derivative of the transition in the vertical coordinate
is $\sigma$, so these equations are exactly the gluing
condition for a vertical field. The equality is preserved
under further open restriction.
Because the fitness is smooth in an affine neighborhood,
the local differential equation has unique solutions.
A solution for the $V$-presentation, carried across the
overlap, is a solution in the $U$-presentation with the same
initial condition as the one in that chart. Uniqueness gives
the flow comparison. The normalized mass and support
identities of Chapter~\ref{ch:replicator} keep each trajectory
in its compact simplex. The continuation theorem for smooth
ordinary differential equations then gives existence for all
nonnegative time at every fixed $b$. The mass calculation only
uses the finite set indexing the simplex. It does not require
a distinguished order on that set.
\end{proof}
The commutative square records how a transition acts on the
whole flow, rather than on its rest points alone:
\[
\begin{tikzcd}[column sep=large,row sep=large]
\Delta(I_V) \arrow[r,"\Phi^V_t"]\arrow[d,"\sigma"'] &
\Delta(I_V)\arrow[d,"\sigma"]\\
\Delta(I_U) \arrow[r,"\Phi^U_t"] & \Delta(I_U).
\end{tikzcd}
\]
Here $t$ is elapsed time in a fixed context, not a path in $B$.
A model in which context itself changes with time needs a
rule for that motion. A connection or transition process would
be additional data.
\section{Parametrization and effective descent}
In an $\infty$-topos $\mathcal E$, let $X,Y,P$ be objects over a
base $B$. A family of evaluations from $X$ to $Y$ with parameter
$P$ is a morphism $P\times_B X\to Y$. By the adjunction
for the exponential in $\mathcal E_{/B}$, such a family is
classified by $P\to[Y^X]_B$, where $[Y^X]_B$ denotes the
fiberwise exponential. An inverse-image or presentation change
can preserve evaluation while changing parameter coordinates.
The pullback formula follows by applying the functor
composing structure maps to the exponential adjunction:
for $q:C\to B$ and $T\to C$,
\[
q_!(T\times_C q^*X)\simeq q_!T\times_B X .
\]
Thus $q^*[Y^X]_B\simeq[(q^*Y)^{q^*X}]_C$ by Yoneda.
If $U\to B$ is an effective epimorphism with \v{C}ech nerve
$U_\bullet$, then
\[
\mathcal E_{/B}\simeq
\lim_{[n]\in\boldsymbol\Delta}\mathcal E_{/U_n}.
\]
This is descent for slices in an $\infty$-topos
\cite[\S6.1.3]{lurie-htt}. Objects and morphisms over $B$
can thus be specified locally along with a homotopy-coherent
chain of overlap comparisons. The comparison extends to
evaluation families since their parameter maps land in the
compatible exponential object.
This gives a semantics for local games. A family can be observed
by changing the base, and the result of interpreting that family
is independent of a selected cover. For the concrete smooth
model the relevant gluing equations reduce to equalities; for
spaces with groupoid-valued states a commuting triangle may
instead carry an invertible comparison. One must still specify
which points are probability states and which maps define
fitness. The categorical descent theorem glues these structures
once they are defined in the chosen semantics. It does not
supply a probability law from the base topos.
\section{Monodromy and restricted parameter choices}
A loop in $B$ transports its strategy fiber by a permutation.
The image of the resulting representation of the fundamental
group is a monodromy group $M$. In a connected component,
a flat state is a locally constant probability vector in the
trivializations. It is specified by a vector $x$ at one base
point that is fixed by every permutation in $M$. To prove it,
transport a choice from that point along paths. This is independent
of the path exactly when every based loop fixes the vector.
Flat states therefore have a common value within each orbit of
$M$ on types. If $O$ is such an orbit and $w_O$ is its mass,
each of its coordinates has value $w_O/|O|$. The flat
parameter region is consequently a simplex on monodromy orbits.
A general smooth section of the bundle can vary within a
context and needs only to obey transition laws.
A flat payoff matrix, by the same argument, is fixed under
conjugation by the monodromy permutations. Its entry depends
on an orbit of the diagonal action on ordered pairs of types.
These conditions can force parameter tying, even when local
populations have several distinguishable types. Conversely,
the pair orbits may retain interaction patterns that are
lost if one replaces all types by their orbit labels before
giving the fitness. Context determines which changes of
presentation are legitimate. The next chapter measures a
different failure, when a coarse description is a randomized
channel rather than a permutation.
\par
\chapter{Selection under stochastic compression}
\label{ch:stochastic-compression}
An observer can group strategies by a classification rule.
A deterministic grouping sends each strategy to one class,
whereas a randomized classifier sends it to a distribution of
classes. These two changes of description have different effects
on replicator dynamics. When fitness is pulled back along the
classifier, the difference is given by a conditional covariance.
\section{Columns as conditional distributions}
Let $I,J$ be nonempty finite sets, and let $A$ be a
column-stochastic matrix from $I$ to $J$. Its column $a_i$
is the conditional distribution of the label $j\in J$ given
a strategy $i\in I$. Write $x\in\Delta(I)$ and $z=Ax$.
The pushforward of a distribution is $Ax$. The corresponding
pullback of a payoff vector $\gamma\in\mathbb R^J$ is
$A^\top\gamma$, since
\[
\langle x,A^\top\gamma\rangle=\langle Ax,\gamma\rangle.
\]
This is expectation along a Markov kernel, expressed in
dual coordinates \cite{fritz-markov}. If $g:\Delta(J)\to\mathbb
R^J$ is the observed fitness, consider the fine fitness
$f(x)=A^\top g(Ax)+c(x)\mathbf 1_I$, where $c$ is a
common payoff offset. This assumption controls the dependence
of a fine type's payoff on all its possible labels. It must
be checked in a proposed aggregation.
For a probability vector $w$, set
$M(w)=\operatorname{diag}(w)-ww^\top$. The replicator
field is $R_f(w)=M(w)f(w)$. This form includes the
constraint that shifts by a common payoff have zero field,
since $M(w)\mathbf 1=0$. For a channel $A$ define
\[
C_A(x)=\operatorname{diag}(Ax)-A\operatorname{diag}(x)A^\top .
\]
\begin{theorem}[Covariance of a compressed replicator field]
\label{thm:channel-defect}
Under the payoff hypothesis above,
\[
R_g(Ax)-A R_f(x)=C_A(x)\,g(Ax).
\]
The matrix $C_A(x)$ is positive semidefinite. At a given $x$
the defect vanishes exactly when $g(Ax)$ is constant on the
support of every column $a_i$ with $x_i>0$.
In particular, exact pushforward for all $x$ and all
coarse fitness maps holds precisely when $A$ represents
a deterministic function $I\to J$.
\end{theorem}
\begin{proof}
The common offset disappears in $M(x)f(x)$. Using
$Axx^\top A^\top=(Ax)(Ax)^\top$ gives
\[
M(Ax)-A M(x)A^\top=C_A(x).
\]
Multiplication by $g(Ax)$ gives the claimed difference.
For an arbitrary test vector $u\in\mathbb R^J$,
\begin{align*}
C_A(x)&=\sum_i x_i
\bigl(\operatorname{diag}(a_i)-a_i a_i^\top\bigr),\\
u^\top C_A(x)u&=\sum_i x_i
\left(\sum_j (a_i)_j u_j^2-
\left(\sum_j(a_i)_j u_j\right)^2\right).
\end{align*}
The summands are conditional variances and are nonnegative.
A variance is zero precisely when $u$ is constant on the
support of the corresponding column. For a positive
semidefinite real symmetric matrix $C$, $u^\top C u=0$
is equivalent to $Cu=0$ by diagonalization.
Taking $u=g(Ax)$ proves the equality criterion. Every
deterministic column has zero covariance. Conversely choose
$x$ concentrated on any specified $i$ and allow a constant
target fitness taking an arbitrary vector value. Universal
vanishing then forces every coordinate indicator to have zero
variance under $a_i$, so every coordinate probability is either
zero or one. Their sum is one, proving determinism.
\end{proof}
Deterministic compression groups types without spreading any
type across distinct observed labels. Its exactness is the
coordinate expression of summing replicator derivatives
over the fibers of a function, when the payoffs agree within
fibers. In stochastic compression, equality can still hold
for particular payoffs or particular support faces. It is
then a condition on the conditional distribution and the
payoff together.
\section{Composition of two compressions}
For a second channel $B:J\to K$ the defects obey
\[
C_{BA}(x)=C_B(Ax)+B C_A(x)B^\top.
\]
Indeed, first add and subtract
$B\operatorname{diag}(Ax)B^\top$ in the expression for
$C_{BA}(x)$. The two terms are the covariance from the
second channel and the covariance of its conditional means
induced by the first. This is the law of total covariance
for the original sampling sequence. The formula treats
loss of a selection signal as a two-stage correlation
calculation.
The two summands are positive semidefinite. If the final
fitness is $v\in\mathbb R^K$, exact pushforward through
the composite holds at $x$ precisely when
$C_B(Ax)v=0$ and $C_A(x)B^\top v=0$. To see this,
test the displayed expression in the quadratic form at
$v$. It is a sum of two nonnegative numbers, each with the
zero criterion of Theorem~\ref{thm:channel-defect}.
The two equality conditions correspond to asking that
the final payoff be constant on the occupied conditional
supports of the second stage and that its first-stage
conditional mean be constant on the relevant supports of the
first. This criterion can be applied before composing the
channels.
\[
\begin{tikzcd}[column sep=large,row sep=large]
\Delta(I) \arrow[r,"A"] \arrow[dr,"BA"'] &
\Delta(J) \arrow[d,"B"]\\
&\Delta(K)
\end{tikzcd}
\]
If each step is deterministic, the maps and the exact
fitness pullbacks compose without defect. This yields a
class of morphisms of selection systems. The same category
of stochastic maps can be used for compression with a
nonzero defect. In that case the formula identifies the
specific obstruction to the map being a morphism of
replicator dynamics.
\section{Trajectory error on a bounded time interval}
Write $x(t)$ for the fine solution and $z(t)$ for the
coarse solution starting at $Ax(0)$. Suppose both remain
on their simplices and that $R_g$ is Lipschitz there with
constant $L$ for a fixed norm. Define
$c_t=C_A(x(t))g(Ax(t))$. The projected fine solution
satisfies
\[
\frac{d}{dt}Ax(t)=R_g(Ax(t))-c_t .
\]
Subtracting the coarse equation and applying the
integral form of Gr\"onwall's inequality yields
\[
\|Ax(t)-z(t)\|
\leq\int_0^t e^{L(t-s)}\|c_s\|\,ds .
\]
One derivation is to integrate the difference of the
two derivatives, bound the target field difference by
$L\|Ax-z\|$, and iterate the resulting integral inequality.
Its Volterra series sums to the displayed exponential.
If the defect is identically zero, uniqueness gives exact
semiconjugacy. If the defect is merely small, the estimate
controls finite-time approximation. It does not give a
uniform long-time guarantee when the bound grows with $t$.
For other mechanisms that yield semiconjugacy and
lumpability, see Chapters~\ref{ch:aggregation}
and~\ref{ch:semiconjugacy}.
\section{The tangent and cotangent interpretation}
On the relative interior of a simplex, the metric
\[
\langle v,w\rangle_x=\sum_i \frac{v_iw_i}{x_i},
\qquad \sum_i v_i=\sum_i w_i=0
\]
identifies the centered payoff covector with its
replicator vector. In fact
$\langle R_f(x),v\rangle_x=\sum_i f_i(x)v_i$.
The metric is the familiar Shahshahani metric
\cite{shahshahani}. Its inverse on covectors modulo
constant vectors has matrix $M(x)$. If $Ax$ is on the
boundary of the target simplex, the same construction applies
on the relative interior of its support face, with its inverse
metric represented in the full coordinates by zero extension.
The difference between the target inverse metric and the
pushforward of the source inverse metric is $C_A(x)$. The covariance is thus also the change in the
quadratic form used to convert a payoff into a selection
direction. This observation explains why contraction
of information under sampling need not make evolution
commute with sampling. The statement proved here
uses ordinary finite-dimensional distributions and
an exact payoff pullback, which are strong enough to
identify the defect in closed form.
\par
\chapter{Action groupoids and symmetric selection}
\label{ch:higher-equivariance}
A permutation group can act on a game by relabeling its types.
The resulting symmetry has a fixed locus, an orbit space, and
an action groupoid. Each answers a different question. This
chapter puts these constructions alongside the probability
simplex and uses them to separate symmetry of a state from
the quotient of a family of states.
\section{The fixed simplex and equilibrium}
Let a group $G$ act by permutations of a nonempty finite set $I$, and
suppose $f:\Delta(I)\to\mathbb R^I$ is continuous and equivariant.
The image of $G$ in the permutation group is finite. The fixed
simplex $\Delta(I)^G$ consists of the states with equal
coordinates on each orbit, so it is naturally a simplex on
the orbit set. If $p$ is fixed, $f(p)$ is also fixed. The
replicator field and its flow preserve this fixed simplex by
the descent calculation in Chapter~\ref{ch:contextual-games},
whenever the field satisfies the existence and uniqueness
hypotheses there.
There is an equilibrium in the fixed simplex under the
given continuity hypothesis, even before choosing a
dynamic solution.
\begin{proposition}[An invariant Nash state]
\label{prop:symmetric-nash}
Under these hypotheses there is a $G$-fixed $p$ such
that $\langle y,f(p)\rangle\leq\langle p,f(p)\rangle$
for every $y\in\Delta(I)$.
\end{proposition}
\begin{proof}
For any $x$ put
$a_i(x)=\max\{f_i(x)-\langle x,f(x)\rangle,0\}$ and
$s(x)=\sum_i a_i(x)$. The continuous map
\[
F_i(x)=\frac{x_i+a_i(x)}{1+s(x)}
\]
is equivariant because each operation in its definition
commutes with permutation. It maps the fixed simplex
to itself. By Brouwer's theorem it has a fixed point
$p$ there. The fixed point equation gives
$a_i(p)=s(p)p_i$. If $s(p)>0$, each occupied type has
strictly higher fitness than the mean. Taking the mean
over all occupied types would give a strict inequality
with itself. Hence $s(p)=0$, and the individual
inequalities $f_i(p)\leq\langle p,f(p)\rangle$ give
the result for all $y$ by linearity in $y$.
\end{proof}
This calculation uses the ambient equilibrium inequalities.
One can also compress along the orbit function and work with
orbit masses, provided the fitness assigned to each orbit is
the correct one. A restriction of the dynamics to symmetric
states only tests attraction in the fixed simplex. For smooth
fitness, at an interior fixed point the derivative of the
field is a linear map commuting with the permutation action
on the tangent space. The averaging projector onto the fixed
vectors commutes with that map. The tangent space consequently
splits into the fixed part and the kernel of averaging, both
invariant under the derivative. Stability on the fixed
part leaves the transverse block to be tested separately.
At the boundary, feasible perturbations are governed by the
tangent cone of the ambient simplex and can enter larger faces.
\section{What the quotient records}
For an action on a space $X$, write $X//G$ for the
realization of the action bar object
$[n]\mapsto X\times G^n$. It maps to the delooping $BG$
and the fiber over its distinguished point is $X$.
If $X$ is an ordinary discrete set, this is the groupoid
of elements and symmetry arrows. After choosing orbit
representatives $x$, it is equivalent to the coproduct
of $BG_x$ over those representatives, where $G_x$ is
the subgroup fixing $x$. Every orbit is a connected
groupoid and inclusion of the full subgroupoid at one
object is an equivalence. The set of connected
components is the coarse orbit set. The fuller quotient
retains the automorphism groups of pure states.
The whole probability simplex makes the role of the
acted-on object clear. It has an equivariant contraction
$\lambda x+(1-\lambda)u$ to the uniform state $u$.
Its homotopy quotient over $BG$ is therefore equivalent
to $BG$. The pure strategy set, a rest-point subspace,
or a space of trajectories may have different quotient
types. Equivariant contractibility of the whole simplex gives
its quotient type. Each subspace requires a separate computation.
For a smooth equivariant field, the space of trajectories
on a fixed time interval carries the action by transport
of each trajectory. Initial and terminal evaluation maps
are equivariant. In a well-posed deterministic system the
initial evaluation is an equivariant homeomorphism to
the initial state space, using continuous dependence on
initial conditions. Terminal evaluation is then the flow
map expressed through this identification. For a
set-valued best response or a multivalued update (see
Chapter~\ref{ch:revision}), the corresponding initial
evaluation can have multiple trajectories over a state.
In that situation its fibers encode a choice of resolution
of ties or admissible continuations.
\section{Coherent actions in an \texorpdfstring{$\infty$}{infinity}-topos}
An internal group object in an $\infty$-topos
$\mathcal E$ has a reduced bar groupoid $G_\bullet$ and a
delooping $BG$. Effectivity identifies $G_\bullet$ with
the \v{C}ech nerve of $1\to BG$. An action on $X$ can be
specified by a cartesian transformation into
$G_\bullet$ with degree-zero object $X$. Its realization
is the map $X//G\to BG$. Conversely any object over $BG$
pulls back along the \v{C}ech nerve to a cartesian
transformation. Descent for slices shows that these
assignments are inverse \cite[\S6.1.3]{lurie-htt,nss}.
In particular,
\[
\operatorname{Act}_G(\mathcal E)\simeq \mathcal E_{/BG}.
\]
The corresponding square
\[
\begin{tikzcd}[column sep=large,row sep=large]
X\arrow[r]\arrow[d] & X//G\arrow[d]\\
1\arrow[r] & BG
\end{tikzcd}
\]
is a homotopy pullback. This describes quotient families
while retaining their lifts and all compatibility
conditions for change of context.
An evaluator $P\times X\to Y$ between equivariant objects
is represented in this slice, so its local evaluations
must transform both their parameters and inputs. For a
discrete group acting on spaces, the space of equivariant
maps from $X$ to $Y$ is the homotopy fixed-point space of
$\operatorname{Map}(X,Y)$ with conjugation transport.
For example, the cosimplicial description has in degree
$n$ a family of maps for each string in $G^n$. Transporting
the domain along the string identifies it with the usual
cosimplicial diagram for the homotopy limit of the
conjugation action. Totalization imposes naturality and
its higher coherence. The assertion concerns the space
of possible equivariant maps. The existence of a
natural dynamical invariant, a distinguished equilibrium,
or a numerical update requires its own mathematical
conditions.
In these constructions the context can be an ordinary
parameter space, a classifying groupoid, or an internal
base for locally varying types. Equivariance and descent
describe how locally constructed fitness and revision
laws assemble. They do not determine a revision law
without assumptions on its behavioral or statistical
meaning. Weight updates provide a particularly direct
case where those assumptions and the resulting equations
can be specified together.
\par
\chapter{Learning rules and populations of agents}
\label{ch:agent-learning}
Selection can act on candidate policies or programs as well
as on inherited traits. In a learning system, three choices
enter explicitly: what counts as a strategy, how its score
is obtained, and how that score changes its frequency.
The replicator field is an infinitesimal revision rule.
A discrete-time learner can approximate it or use a
different update with its own statistical guarantee.
\section{An update from relative entropy}
Let $x$ be a probability vector with positive coordinates
on a finite set $I$. Given a payoff vector $a$ and a step
$\eta>0$, exponential weights defines
\[
x_i^+=\frac{x_i e^{\eta a_i}}{\sum_j x_j e^{\eta a_j}}.
\]
A common additive shift of the payoff cancels. The
update uniquely maximizes on the simplex
\[
y\longmapsto
\langle y,a\rangle-\eta^{-1}D(y\Vert x),
\qquad
D(y\Vert x)=\sum_i y_i\log(y_i/x_i),
\]
where a zero summand is interpreted as zero. Indeed,
substitution of $x^+$ writes the objective as a constant
minus $\eta^{-1}D(y\Vert x^+)$. Nonnegativity of relative
entropy and its equality condition follow from Jensen's
inequality for $\log$, with terms of zero mass handled
by continuity. If the initial state has zeros, the same
definition is used on its support face. This is a
multiplicative weights update \cite{freund-schapire,arora-mw}.
Differentiation at $\eta=0$ gives
\[
\frac{\partial x_i^+}{\partial\eta}\bigg|_{\eta=0}
=x_i(a_i-\langle x,a\rangle).
\]
Putting $a=f(x)$ yields the replicator field. If $f$ is
Lipschitz on the compact simplex, it is bounded there
and the local truncation error relative to the Euler
step is uniformly $O(\eta^2)$. Indeed the denominator
is bounded away from zero for small $\eta$ and the
second derivative with respect to $\eta$ is uniformly
bounded. On a bounded time interval, compare with
the Lipschitz ordinary differential equation. The local
flow also differs from its Euler step by $O(\eta^2)$.
The discrete error then satisfies a recursion bounded by
$(1+L\eta)$ times the previous error plus $O(\eta^2)$.
Summing the geometric series gives an $O(\eta)$ error
at grid times for fixed elapsed time. This connects the
update to continuous selection without drawing a
conclusion about arbitrarily long training.
\section{The pathwise regret bound}
Suppose payoffs $a^t$ are observed each round, with
$|a^t_i|\leq M$ for every coordinate and round. Their
choice may depend on previous weights. Let $x^t$ be
the pre-update state and $\eta$ a fixed step size.
\begin{proposition}[External regret of exponential weights]
\label{prop:regret}
For a comparator $p$ supported on the initial support,
after $T$ updates one has
\[
\sum_{t=0}^{T-1}\langle p-x^t,a^t\rangle
\leq \frac{D(p\Vert x^0)}{\eta}
+\frac{\eta M^2 T}{2}.
\]
\end{proposition}
\begin{proof}
Write $Z_t=\sum_i x^t_i e^{\eta a^t_i}$.
The update equation gives
\[
D(p\Vert x^{t+1})-D(p\Vert x^t)
=-\eta\langle p,a^t\rangle+\log Z_t .
\]
For a fixed round, the second derivative of
$\log\sum_i x^t_i e^{s a^t_i}$ is the variance of $a^t$
under the distribution tilted by $s$. It is at most $M^2$
since all of its values belong to $[-M,M]$.
The first derivative at zero is $\langle x^t,a^t\rangle$.
Integrating the second derivative bound gives
$\log Z_t\leq\eta\langle x^t,a^t\rangle+\eta^2M^2/2$.
The claimed inequality follows by summing and dropping the
nonnegative final relative entropy.
\end{proof}
This is a comparison to a fixed behavior in hindsight.
With an appropriate step as a function of the horizon, the
bound divided by $T$ tends to zero. It allows an arbitrary
sequence of bounded revealed scores and does not require
that a single game produce them. For continuous-time
regularized reinforcement learning and its relation to
replicator and other dynamics, see
\cite{mertikopoulos-sandholm}.
\section{Averaged joint play}
The regret theorem yields a statement about a game when
the scores are the actual counterfactual payoffs of that
game. For a finite action product, let $u_i$ be player $i$'s
utility. Each round the players independently use their
mixed strategies conditioned on the history, giving a
product law $\nu_t$. If player $i$ receives the payoff
vector of expected utilities of each of its possible
actions against the other players' current choices of
distributions, its payoff under its own distribution is
$\mathbb E_{\nu_t}u_i(a)$. Its payoff from a fixed unilateral
deviation $b_i$ is
$\mathbb E_{\nu_t}u_i(b_i,a_{-i})$. Set
$\overline\nu_T=T^{-1}\sum_t\nu_t$. Then the coarse
correlated equilibrium inequalities take the form
\[
\mathbb E_{\overline\nu_T}
[u_i(b_i,a_{-i})-u_i(a)]
\leq \frac{\operatorname{Regret}_i(T)}{T}
\]
for every $i,b_i$, with regret defined as the maximum
over the fixed deviations (or a bound on that maximum).
This is obtained by adding the per-round deviations.
When these bounds tend to zero every accumulation point
of the averaged joint laws is a coarse correlated
equilibrium, by continuity of the finitely many linear
inequalities. The conclusion concerns joint laws, which
can carry correlation through their common time index.
A Nash statement on the last product distribution
would require other hypotheses. This distinction is
important in agent populations whose behavior continues
to cycle or react to changing opponents.
\section{Outer selection and the evaluation interface}
A program strategy can be represented by a fixed artifact,
with an evaluation map from its identity and the current
context to a score. The test distribution, budget,
counterfactual comparison class, and timing determine
whether the score estimates the intended fitness.
An inner process can generate or train candidate
artifacts. An outer process can then keep a distribution
over verified candidates and update that distribution using
scores. Exponential weights keeps the distribution in the
simplex and is insensitive to score shifts common to
all candidates in a round. These facts are properties of
the update. They alone say nothing about the behavior
or validity of any candidate.
There is also a compositional concern. If an evaluator
reports the same payoff for every strategy in a coarse
fiber, deterministic aggregation can commute with
selection. If it reports only a randomized label, the
calculation of Chapter~\ref{ch:stochastic-compression}
identifies a covariance defect even when the fine
payoffs are conditional expectations of coarse
scores. This gives a test of a proposed reporting
interface before using it as a summary of population
behavior. Delayed reports, context changes, or
evaluation mismatch introduce separate errors. A
learning guarantee is conditional on specifying
those mechanisms and bounding their effect.
The constructions in the book put equilibrium,
dynamics, stochastic modeling, and presentation
changes into compatible types. A model of interacting
agents adds an interpretation of strategies and a
mechanism for producing scores at the chosen interface.
The resulting learning statements remain relative
to that mechanism.
\par
\appendix
\chapter{Notation}
\label{app:notation}
\small\raggedright
\setlength{\parskip}{0pt}
\begin{description}[leftmargin=2em,style=nextline,itemsep=.31\baselineskip]
\item[$I$, $\Delta(I)$, $x_i$] Type set, its probability simplex,
and mass assigned to type $i$. See Chapter~\ref{ch:population}.
\item[$f(x)$, $\langle x,f(x)\rangle$] The current fitness vector
and its mean under $x$.
\item[$R_f(x)$, $M(x)$] The replicator field and the matrix
$\operatorname{diag}(x)-xx^\top$ that converts a payoff
covector into a selection direction. See Chapters~\ref{ch:replicator}
and~\ref{ch:stochastic-compression}.
\item[$D(p\Vert x)$] Relative entropy of a comparator $p$ from
the state $x$, finite when the support of $p$ is contained in
that of $x$. See Chapters~\ref{ch:lyapunov} and~\ref{ch:agent-learning}.
\item[$\operatorname{supp}(x)$] The types with positive mass in $x$.
The associated support face contains trajectories from $x$ for
the replicator field without mutation.
\item[$P$, $Q$] In stochastic chapters, transition kernels or
generator matrices with conventions specified at their use.
In parametrized games, $P$ can also denote a parameter object.
\item[$A^\top$, $C_A(x)$] The pullback of payoff vectors along
a column-stochastic channel $A$, and its conditional covariance
$\operatorname{diag}(Ax)-A\operatorname{diag}(x)A^\top$.
See Chapter~\ref{ch:stochastic-compression}.
\item[$(X,S)$, $(Y,R)$] Forward and backward types of an open-game
interface. See Chapter~\ref{ch:open-games}.
\item[$\mathcal E_{/B}$, $P\times_B X$]
The slice over a context object $B$ and its fiber product.
See Chapter~\ref{ch:contextual-games}.
\item[$BG$, $X//G$] The delooping of an internal group object and
the homotopy quotient of a $G$-object. See Chapter~\ref{ch:higher-equivariance}.
\end{description}
\chapter{Machine-checked selection identities}
\label{app:formalization}
The accompanying Lean project formalizes the algebra that makes
deterministic aggregation compatible with selection. It uses Lean~4.19.0,
pinned mathlib, and 23 audited theorems. The index types are arbitrary
finite types. The statements apply uniformly to every cardinality,
including reindexing along a map that leaves some target coordinates
empty. See \cite{lean4} for the proof assistant.
\section{Drift and the normalized update}
Let $x,u:I\to R$, where $I$ has a finite type structure. The
formal definitions are
\[
m(x)=\sum_i x_i,\qquad
\mu_x(u)=\sum_i x_i u_i,\qquad
F(x,u)_i=x_i\big(u_i-\mu_x(u)\big).
\]
Over a commutative ring the kernel checks
\[
\sum_i F(x,u)_i=(1-m(x))\mu_x(u).
\]
It follows that the drift is tangent to the mass-one hyperplane.
The zero-coordinate property, invariance under adding a common payoff
on that hyperplane, and linearity under uniform payoff scaling are
also checked. These are identities of algebraic expressions used in
Chapter~\ref{ch:replicator}.
Over a field the project defines the reproductive update
\[
S(x,u)_i=\frac{x_i u_i}{\mu_x(u)}.
\]
It verifies unit mass when the denominator is nonzero. Over a linearly
ordered field with $x$ in the simplex and strictly positive fitness
$u$, Lean derives positivity of the denominator and proves that the
output lies in the simplex. For affine reproductive weights
$w_i=1+\eta u_i$, under $m(x)=1$ and $1+\eta\mu_x(u)\ne0$, it also checks
the exact discrete-to-drift identity
\[
S(x,w)_i-x_i
=\frac{\eta}{1+\eta\mu_x(u)}F(x,u)_i .
\]
No time limit is needed to obtain this equality.
\section{Deterministic aggregation}
For a map $q:I\to J$, the Lean definition of aggregation is
$(q_*x)_j=\sum_{q(i)=j}x_i$. The project proves preservation of mass
and of simplex membership, functoriality under composition, and the
pullback-pushforward pairing
\[
\mu_x(v\circ q)=\mu_{q_*x}(v).
\]
For $v:J\to R$ it proves the two commutation identities
\[
q_*F(x,v\circ q)=F(q_*x,v),\qquad
q_*S(x,v\circ q)=S(q_*x,v).
\]
A state-dependent form and a specialization to pullbacks of payoff
matrices make the first equation applicable to the deterministic
semiconjugacy of Chapter~\ref{ch:semiconjugacy}. Each result uses
finite sums and function extensionality. The second identity is
algebraic as an equality of field terms. Its population interpretation
uses the nonzero normalizer as in the preceding section.
\section{Reproducing the check}
From the extracted Lean source, run \texttt{lake exe cache get},
\texttt{lake build}, and \texttt{lake env lean Audit.lean}.
The manifest pins the mathlib revision. The last command prints
the transitive axioms of the audited declarations. The supplied audit
records standard Lean axioms and no admitted proofs. The appendix
covers the algebraic vector field and reproductive update. The main
text gives the analytical and higher-categorical arguments, including
existence of flows, stochastic limits, and the channel covariance
formula in Chapter~\ref{ch:stochastic-compression}.
\chapter{Subject index}
\label{app:index}
\small
\setlength{\parskip}{0pt}
\noindent References give the chapters or appendices where each term is developed.
\smallskip
\begin{multicols}{2}\raggedright
\begin{description}[leftmargin=0pt,labelindent=0pt,itemsep=.10\baselineskip]
\item[Aggregation] \ref{ch:aggregation}, \ref{ch:semiconjugacy}, \ref{ch:stochastic-compression}.
\item[Assortative interaction] \ref{ch:structured}.
\item[Best response] \ref{ch:revision}.
\item[Coarse correlated equilibrium] \ref{ch:agent-learning}.
\item[Context] \ref{ch:contextual-games}, \ref{ch:higher-equivariance}.
\item[Continuation] \ref{ch:open-games}, \ref{ch:compositional-games}.
\item[Coplay] \ref{ch:open-games}.
\item[Covariance defect] \ref{ch:stochastic-compression}.
\item[Descent] \ref{ch:contextual-games}, \ref{ch:higher-equivariance}.
\item[Equilibria] \ref{ch:ess}, \ref{ch:higher-equivariance}.
\item[Evolutionarily stable strategy] \ref{ch:ess}, \ref{ch:superiority}, \ref{ch:lyapunov}.
\item[Exponential weights] \ref{ch:agent-learning}.
\item[Finite populations] \ref{ch:markov}, \ref{ch:mean-field}.
\item[Fitness] \ref{ch:population}, \ref{ch:payoff-equivalence}.
\item[Formalization] \ref{app:formalization}.
\item[Homotopy quotient] \ref{ch:higher-equivariance}.
\item[Invasion] \ref{ch:ess}, \ref{ch:superiority}.
\item[Lens] \ref{ch:open-games}.
\item[Logit dynamics] \ref{ch:revision}.
\item[Lumpability] \ref{ch:aggregation}.
\item[Lyapunov functions] \ref{ch:lyapunov}.
\item[Markov kernel] \ref{ch:markov}, \ref{ch:aggregation}.
\item[Mass action] \ref{ch:open-dynamics}.
\item[Mean-field limit] \ref{ch:mean-field}.
\item[Monodromy] \ref{ch:contextual-games}.
\item[Multipopulation dynamics] \ref{ch:structured}.
\item[Mutation] \ref{ch:replicator}, \ref{ch:stochastic-stability}.
\item[Open dynamics] \ref{ch:open-dynamics}.
\item[Open game] \ref{ch:open-games}, \ref{ch:compositional-games}.
\item[Payoff shift] \ref{ch:payoff-equivalence}, \ref{ch:agent-learning}.
\item[Pushout gluing] \ref{ch:open-dynamics}.
\item[Regret] \ref{ch:agent-learning}.
\item[Revision protocol] \ref{ch:revision}, \ref{ch:compositional-games}.
\item[Replicator equation] \ref{ch:replicator}, \ref{ch:lyapunov}.
\item[Semiconjugacy] \ref{ch:semiconjugacy}, \ref{ch:stochastic-compression}.
\item[Shahshahani metric] \ref{ch:lyapunov}, \ref{ch:stochastic-compression}.
\item[Stochastic stability] \ref{ch:stochastic-stability}.
\item[Support face] \ref{ch:population}, \ref{ch:replicator}.
\item[Symmetry] \ref{ch:contextual-games}, \ref{ch:higher-equivariance}.
\end{description}
\end{multicols}
\backmatter
\begin{thebibliography}{99}
\addcontentsline{toc}{chapter}{Bibliography}
\small\raggedright
\setlength{\itemsep}{0pt}\setlength{\parsep}{0pt}\setlength{\parskip}{0pt}
\setlength{\baselineskip}{11.5pt}
\bibitem{alexander} J. McKenzie Alexander. \emph{Evolutionary Game Theory}. Elements in Decision Theory and Philosophy, Cambridge University Press, 2023. \href{https://doi.org/10.1017/9781108582063}{doi:10.1017/9781108582063}.
\bibitem{weibull} J. W. Weibull. \emph{Evolutionary Game Theory}. MIT Press, 1995. \href{https://mitpress.mit.edu/9780262231817/evolutionary-game-theory/}{Publisher}.
\bibitem{maynard-price} J. Maynard Smith and G. R. Price. The logic of animal conflict. \emph{Nature} 246 (1973), 15--18. \href{https://doi.org/10.1038/246015a0}{doi:10.1038/246015a0}.
\bibitem{taylor-jonker} P. D. Taylor and L. B. Jonker. Evolutionarily stable strategies and game dynamics. \emph{Mathematical Biosciences} 40 (1978), 145--156. \href{https://doi.org/10.1016/0025-5564(78)90077-9}{doi:10.1016/0025-5564(78)90077-9}.
\bibitem{hofbauer-schuster-sigmund} J. Hofbauer, P. Schuster, and K. Sigmund. A note on evolutionary stable strategies and game dynamics. \emph{Journal of Theoretical Biology} 81 (1979), 609--612. \href{https://doi.org/10.1016/0022-5193(79)90058-4}{doi:10.1016/0022-5193(79)90058-4}.
\bibitem{hofbauer-sigmund} J. Hofbauer and K. Sigmund. \emph{Evolutionary Games and Population Dynamics}. Cambridge University Press, 1998. \href{https://doi.org/10.1017/CBO9781139173179}{doi:10.1017/CBO9781139173179}.
\bibitem{shahshahani} S. Shahshahani. \emph{A New Mathematical Framework for the Study of Linkage and Selection}. Memoirs of the AMS 17, no.~211 (1979). \href{https://doi.org/10.1090/memo/0211}{doi:10.1090/memo/0211}.
\bibitem{hofbauer-sandholm-stable} J. Hofbauer and W. H. Sandholm. Stable games and their dynamics. \emph{Journal of Economic Theory} 144 (2009), 1665--1693.e4. \href{https://doi.org/10.1016/j.jet.2009.01.007}{doi:10.1016/j.jet.2009.01.007}.
\bibitem{sandholm-pairwise} W. H. Sandholm. Pairwise comparison dynamics and evolutionary foundations for Nash equilibrium. \emph{Games} 1 (2010), 3--17. \href{https://doi.org/10.3390/g1010003}{doi:10.3390/g1010003}.
\bibitem{sandholm-population} W. H. Sandholm. \emph{Population Games and Evolutionary Dynamics}. MIT Press, 2010. \href{https://mitpress.mit.edu/9780262195874/population-games-and-evolutionary-dynamics/}{Publisher}.
\bibitem{aubin-cellina} J.-P. Aubin and A. Cellina. \emph{Differential Inclusions: Set-Valued Maps and Viability Theory}. Springer, 1984. \href{https://doi.org/10.1007/978-3-642-69512-4}{doi:10.1007/978-3-642-69512-4}.
\bibitem{hofbauer-sorin} J. Hofbauer and S. Sorin. Best response dynamics for continuous zero-sum games. \emph{Discrete and Continuous Dynamical Systems B} 6 (2006), 215--224. \href{https://doi.org/10.3934/dcdsb.2006.6.215}{doi:10.3934/dcdsb.2006.6.215}.
\bibitem{ohtsuki-nowak} H. Ohtsuki and M. A. Nowak. The replicator equation on graphs. \emph{Journal of Theoretical Biology} 243 (2006), 86--97. \href{https://doi.org/10.1016/j.jtbi.2006.06.004}{doi:10.1016/j.jtbi.2006.06.004}.
\bibitem{moran-random} P. A. P. Moran. Random processes in genetics. \emph{Mathematical Proceedings of the Cambridge Philosophical Society} 54 (1958), 60--71. \href{https://doi.org/10.1017/S0305004100033193}{doi:10.1017/S0305004100033193}.
\bibitem{young-conventions} H. P. Young. The evolution of conventions. \emph{Econometrica} 61 (1993), 57--84. \href{https://doi.org/10.2307/2951783}{doi:10.2307/2951783}.
\bibitem{leighton-rivest} F. T. Leighton and R. L. Rivest. Estimating a probability using finite memory. \emph{IEEE Transactions on Information Theory} 32 (1986), 733--742. \href{https://doi.org/10.1109/TIT.1986.1057250}{doi:10.1109/TIT.1986.1057250}.
\bibitem{kurtz-limit} T. G. Kurtz. Solutions of ordinary differential equations as limits of pure jump Markov processes. \emph{Journal of Applied Probability} 7 (1970), 49--58. \href{https://doi.org/10.2307/3212147}{doi:10.2307/3212147}.
\bibitem{benaim-weibull} M. Bena\"im and J. W. Weibull. Deterministic approximation of stochastic evolution in games. \emph{Econometrica} 71 (2003), 873--903. \href{https://doi.org/10.1111/1468-0262.00429}{doi:10.1111/1468-0262.00429}.
\bibitem{kemeny-snell} J. G. Kemeny and J. L. Snell. \emph{Finite Markov Chains}. Springer, 1976. \href{https://link.springer.com/book/9780387901923}{Publisher}.
\bibitem{fritz-markov} T. Fritz. A synthetic approach to Markov kernels, conditional independence, and theorems on sufficient statistics. \emph{Advances in Mathematics} 370 (2020), 107239. \href{https://doi.org/10.1016/j.aim.2020.107239}{doi:10.1016/j.aim.2020.107239}.
\bibitem{ghani-open-games} N. Ghani, J. Hedges, V. Winschel, and P. Zahn. Compositional game theory. \emph{Proceedings of LICS} (2018), 472--481. \href{https://doi.org/10.1145/3209108.3209165}{doi:10.1145/3209108.3209165}.
\bibitem{hedges-morphisms} J. Hedges. Morphisms of open games. \emph{Electronic Notes in Theoretical Computer Science} 341 (2018), 151--177. \href{https://doi.org/10.1016/j.entcs.2018.11.008}{doi:10.1016/j.entcs.2018.11.008}.
\bibitem{bolt-bayesian} J. Bolt, J. Hedges, and P. Zahn. Bayesian open games. \emph{Compositionality} 5 (2023), no.~9. \href{https://doi.org/10.32408/compositionality-5-9}{doi:10.32408/compositionality-5-9}.
\bibitem{vagner-wiring} D. Vagner, D. I. Spivak, and E. Lerman. Algebras of open dynamical systems on the operad of wiring diagrams. \emph{Theory and Applications of Categories} 30 (2015), 1793--1822. \href{https://tac.mta.ca/tac/volumes/30/51/30-51abs.html}{Journal}.
\bibitem{baez-pollard-reactions} J. C. Baez and B. S. Pollard. A compositional framework for reaction networks. \emph{Reviews in Mathematical Physics} 29 (2017), 1750028. \href{https://doi.org/10.1142/S0129055X17500283}{doi:10.1142/S0129055X17500283}.
\bibitem{baez-courser-markov} J. C. Baez and K. Courser. Coarse-graining open Markov processes. \emph{Theory and Applications of Categories} 33 (2018), 1223--1268. \href{https://tac.mta.ca/tac/volumes/33/39/33-39abs.html}{Journal}.
\bibitem{cardelli-lumping} L. Cardelli, G. Squillace, M. Tribastone, M. Tschaikowski, and A. Vandin. Formal lumping of polynomial differential equations through approximate equivalences. \emph{Journal of Logical and Algebraic Methods in Programming} 134 (2023), 100876. \href{https://doi.org/10.1016/j.jlamp.2023.100876}{doi:10.1016/j.jlamp.2023.100876}.
\bibitem{lurie-htt} J. Lurie. \emph{Higher Topos Theory}. Annals of Mathematics Studies 170, Princeton University Press, 2009. \href{https://www.math.ias.edu/~lurie/papers/HTT.pdf}{Author manuscript}.
\bibitem{nss} T. Nikolaus, U. Schreiber, and D. Stevenson. Principal $\infty$-bundles: general theory. \emph{Journal of Homotopy and Related Structures} 10 (2015), 749--801. \href{https://doi.org/10.1007/s40062-014-0083-6}{doi:10.1007/s40062-014-0083-6}.
\bibitem{freund-schapire} Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. \emph{Journal of Computer and System Sciences} 55 (1997), 119--139. \href{https://doi.org/10.1006/jcss.1997.1504}{doi:10.1006/jcss.1997.1504}.
\bibitem{arora-mw} S. Arora, E. Hazan, and S. Kale. The multiplicative weights update method: a meta-algorithm and applications. \emph{Theory of Computing} 8 (2012), 121--164. \href{https://doi.org/10.4086/toc.2012.v008a006}{doi:10.4086/toc.2012.v008a006}.
\bibitem{mertikopoulos-sandholm} P. Mertikopoulos and W. H. Sandholm. Learning in games via reinforcement and regularization. \emph{Mathematics of Operations Research} 41 (2016), 1297--1324. \href{https://doi.org/10.1287/moor.2016.0778}{doi:10.1287/moor.2016.0778}.
\bibitem{lean4} L. de Moura and S. Ullrich. The Lean 4 theorem prover and programming language. \emph{Automated Deduction -- CADE 28} (2021), 625--635. \href{https://doi.org/10.1007/978-3-030-79876-5_37}{doi:10.1007/978-3-030-79876-5\_37}.
\end{thebibliography}
\end{document}