Independent scholarship · Open accessThe monthly digest
Higher Categorical
Semantics
Submit a paper

Editorially reviewed

Competition and Surplus in Long-Term Iterated Games

A graded category of stochastic reward processes formalizes how actions can preserve a current total yet change future surplus. We characterize finite-horizon invariance by action contrasts on baseline forecasts and discounted infinite invariance by a reward potential, then separate surplus gains from individual and equilibrium incentives.

Anonymous← All research

Scope

This is the concise manuscript body for the journal's online form. The companion PDF, Competition and Surplus in Long-Term Iterated Games, gives the complete treatment, with proofs, extended examples, quantitative bounds, and the full bibliography.

Luca Dellanna's “Long-Term Iterated Games” distinguishes rivalry for the next sale, contract, or bargain from a shared interest in future opportunities. The present model takes actions to affect both the division of current payoffs and the transition to future states. A fixed current total can coexist with a variable future total. A gain in that total, an improvement for every player, and equilibrium support are separate claims.

We assume a finite player set $I$, a common discount $0<\delta<1$, and bounded additive expected payoffs in common units. The strategic results use finite state and action sets and publicly observed states and realized actions. An $N$-stage valuation includes dates $0,\ldots,N-1$ and a zero terminal payoff unless specified otherwise. Comparisons use the same initial state or law. “Constant sum” means that aggregate value is independent of the profile in the stated family. “Aggregate improvement” means a strict gain against a specified baseline. “Strict Pareto improvement” means a strict gain in every modeled coordinate. The aggregation covers the declared player set.

1. A category of discounted reward processes

For a finite or countable state interface $X$ let $V_I(X)=\ell^\infty(X,\mathbb R^I)$. A stochastic kernel $P:X\to Y$ pulls fields back by $(P^*v)(x)=\sum_yP(y\mid x)v(y)$. A grade-$n$ arrow $f:X\to Y$ is $(n,P,r)$ with $r\in V_I(X)$. For $g=(m,Q,u):Y\to Z$ define

$$ g\circ f=(n+m,QP,r+\delta^nP^*u), \qquad f^\sharp(v)=r+\delta^nP^*v. $$

The identities are $(0,\mathrm{id},0)$. For three arrows, both bracketings give the accumulated reward $r+\delta^nP^*u+\delta^{n+m}P^*Q^*w$, since $(QP)^*=P^*Q^*$. Hence the composition is associative and $f\mapsto f^\sharp$ is a contravariant functor to ordered vector spaces with monotone affine maps. In particular, $\left(g\circ f\right)^\sharp=f^\sharp g^\sharp$.

Writing $\Sigma_Xv=\sum_i v_i$, summation gives the scalar functor $A(n,P,r)=(n,P,\Sigma_X r)$. It is natural at the continuation level because $\Sigma_X f^\sharp=(Af)^\sharp\Sigma_Y$. For parallel equal-grade arrows with the same aggregate current reward,

$$ \Sigma_X(f^\sharp(v)-f'^\sharp(v)) =\delta^n(P^*-P'^*)\Sigma_Y(v). $$

The arrows agree in aggregate against every bounded terminal field exactly when their kernels agree. Necessity follows by putting a singleton indicator in one payoff coordinate. For a restricted family of terminal fields, only their aggregate span matters. Thus a change in transition may be invisible to one valuation and visible to another.

Fixing a behavioral profile resolves each stage to a reward arrow on public histories, including histories reached only under deviations. The composite of $N$ stages evaluated at zero gives $V_N^\pi=\mathbb E^\pi\sum_{t<N}\delta^t r(s_t,a_t)$. This follows from iterated conditioning. For a per-coordinate reward bound $M$, the infinite value exists and satisfies $\|V_\infty^\pi-V_N^\pi\|_\infty\le M\delta^N/(1-\delta)$ uniformly in profiles and histories.

2. Finite-horizon invariance

Assume $\sum_i r_i(s,a)=c(s)$ for every action at each state, and let $b$ be any deterministic Markov baseline. Let $P_af(s)=\sum_yP(s,a,y)f(y)$, let $P_b$ be the baseline operator, and write $L_{s,a}f=P_af(s)-P_bf(s)$.

Theorem. Expected aggregate value is independent of every behavioral profile from every state at every finite horizon if and only if every $L_{s,a}$ annihilates $\operatorname{span}\{P_b^jc:j\ge0\}$. If $k$ is the first index with a nonzero contrast on $P_b^kc$, then $k+2$ is the first horizon at which aggregate invariance fails somewhere.

Proof. The baseline field for $n$ stages is $g_n=\sum_{j<n}\delta^jP_b^jc$. The comparison that forces $a$ now and follows $b$ later differs from $b$ at horizon $n+1$ by $\delta L_{s,a}g_n$. Vanishing at every horizon implies the test on each generator by subtracting the equations at consecutive horizons. Conversely, if all contrasts vanish, induction gives the same $n$-stage value $g_n$ at every history with its given current state. Conditioning on the first action at the next induction step gives $c(s)+\delta P_ag_n(s)=g_{n+1}(s)$, independent of the action. The same induction proves invariance through $k+1$ if contrasts on all earlier generators vanish. The first nonzero generator then gives a witness at $k+2$. ∎

A globally constant $c$ satisfies the criterion under any controlled transition. A nonconstant $c$ is insufficient by itself because the available actions might agree on the entire span. With an additional family of terminal fields $u$, the condition adds the generators $P_b^ju$ when the family also includes zero.

3. Infinite-horizon invariance

Allow the aggregate stage reward $c(s,a)$ to depend on actions.

Theorem. All profiles have the same discounted aggregate value from every state if and only if there is a bounded scalar field $\phi$ satisfying

$$ c(s,a)=\phi(s)-\delta P_a\phi(s) \quad\text{for every feasible }(s,a). $$

The function $\phi$ is unique. The finite value of any profile is then $\phi(s)-\delta^N\mathbb E_s^\pi\phi(s_N)$.

Proof. If the common value is $\phi$, forcing a first action and then using the baseline gives the equality by conditioning. Conversely, conditional expectations telescope the discounted sum of the right side, giving the finite formula. The bounded terminal term tends to zero. This also proves uniqueness. ∎

For a grade-$n$ aggregate arrow from $X$ to $Y$, the reward change $r\mapsto r-\phi_X+\delta^nP^*\phi_Y$ respects composition because the two terms at the shared interface cancel. This is the boundary correction behind the discounted potential result. With statewise reward $c$, infinite invariance tests each $L_{s,a}$ on the single field $(\mathrm{id}-\delta P_b)^{-1}c$. Finite invariance tests the entire forecast span. An initial action that produces successive aggregate rewards $1,-1/\delta$ on dates one and two therefore ties a zero path in the infinite valuation but wins at horizon two. If invariance holds for discounts with an accumulation point in $(0,1)$, the analytic functions $\sum_{j\ge0}z^j L_{s,a}P_b^jc$ vanish identically. Their coefficients give finite-horizon invariance.

4. Crossover and incentives

For two fixed profiles $\pi,\rho$ from the same initial condition, let $d_t=\mathbb E^\pi r_t-\mathbb E^\rho r_t$ and $D_N=\sum_{t<N}\delta^td_t$. Write $H_N=\sum_{t<N}\delta^tb_t$ where $b_t$ is a coordinate of $d_t$ or the aggregate. If $H_1\le0$ and $b_t\ge0$ thereafter, a strict crossover occurs at a finite horizon exactly when $\sum_{t\ge1}\delta^t b_t>-b_0$. This follows from monotone convergence of the partial sums. For general signed increments with $|b_t|\le B$, the tail is bounded by $B\delta^N/(1-\delta)$. A margin $H_N>B\delta^N/(1-\delta)$ proves positivity at $N$ and every later horizon. A strictly positive discounted limit also implies eventual positivity.

For an immediate loss $k_i>0$ followed by a constant per-period gain $w_i>0$, set $G_N=\delta(1-\delta^{N-1})/(1-\delta)$. The player first gains when $G_Nw_i>k_i$, and a finite gain exists precisely when $\delta>k_i/(k_i+w_i)$. The aggregate can cross before any particular losing player recovers. If transferable units and unrestricted enforceable transfers are available, a strict aggregate gain can make every player strictly better off. The transfer $t_i=(\sum_j D_{N,j})/|I|-D_{N,i}$ sums to zero and equalizes gains. The transfer's enforceability is a strategic assumption.

For a candidate profile $\pi$ and continuation $W^\pi$, the one-step deviation criterion is that each pure alternative at every feasible public history give player $i$ a value at most $W_i^\pi$. If a deviation gives a current gain $g_i$, the condition is $g_i\le\delta(P_\pi^*-P_{\mathrm{dev}}^*)W_i^\pi$. Necessity follows from the definition of subgame perfection. Sufficiency follows by removing deviations backward in any finite truncation and sending its length to infinity with the uniform tail bound. A repeated prisoner's dilemma shows the difference. The always-cooperate path can strictly Pareto dominate always defecting while the always-cooperate strategy fails the one-step test.

5. Observation and local horizons

A quotient $q:S\to\bar S$ preserves a resolved arrow when the reward is constant on each observation block and the total transition probability into every target block depends only on the source block. These conditions are equivalent to the commuting continuation square $f^\sharp q^*=q^*\bar f^\sharp$ for the induced arrow. Evaluate at zero to recover the reward condition and on target block indicators to recover the transition condition. Commuting squares compose. Thus all finite values with descended terminal fields, and the bounded discounted limits, descend. A scalar reward quotient can preserve aggregate values while concealing transfers among players.

For a space $X$ of contexts with continuous comparisons $C_N=\Sigma D_N$, the positivity opens $U_N=\{x:C_N(x)>0\}$ are subterminal objects in $\operatorname{Sh}(X)$. The natural-number object has locally constant sections. The internal existential “some horizon has a positive comparison” holds over an open $U$ exactly when $U\subseteq\bigcup_N U_N$, because the constant witnesses on $U\cap U_N$ form a cover. A single constant witness on all of $U$ is stronger. If $U_N$ are increasing and cover a compact comparison domain, a finite subcover gives a uniform horizon by taking its largest index. On the discrete contexts $m\ge1$, a gain first paid at date $m$ is positive at every later horizon, yet no single finite horizon works for all contexts. The strict Pareto version intersects the positivity opens for the finitely many players.

6. The three cases

Authorship. The current sale has a fixed total within the author boundary. A program shifts probability $p$ from low to high awareness, yielding later increments $w_i\ge0$, and initially transfers $d$ from author one to author two. The expected difference is $(-d,d)+pG_N(w_1,w_2)$. The aggregate gains from horizon two when $p(w_1+w_2)>0$. The conceding author needs $pG_Nw_1>d$. If education requires both to pay real costs, mutual abstention can remain an equilibrium even when joint contribution would benefit both.

Consulting. A failed project reallocates $g$ of current revenue to a rival but puts the industry in a low-trust state with probability $p$. The comparison of reliability to failure is $(-g,g)+pG_N(\ell_1,\ell_2)$, where $\ell_i$ is the per-period loss in that state. The acquiring firm has a private reason to prefer reliability exactly when $p\ell_1G_N>g$. If low trust persists independently with probability $q$ each date, replace $G_N$ by $\delta(1-(\delta q)^{N-1})/(1-\delta q)$. The scenario comparison does not presume one consultant controls a rival's project.

Negotiation. Each active deal divides $2a$ as $(a,a)$ or $(a+g,a-g)$. The relationship survives with respective probabilities $p_F>p_X$, and stopping pays zero above the outside baseline. The balanced policy raises the expected total at every horizon after the first. The proposer's infinite value favors the balanced policy strictly precisely when $\delta((a+g)p_F-a p_X)>g$. The per-date differences for the proposer change from negative to positive at most once, so this is also the criterion for a strict finite crossover. With one active state and the proposer as sole controller, the weak version is the one-step incentive condition for the balanced policy. If the responder can choose an outside option, that participation constraint must also hold.

7. Relation to prior work

Shapley supplies the controlled stochastic-game framework. The category here resolves policies before composition, while compositional game theory and iterated open games retain strategic responses to continuations. The potential theorem uses discounted telescoping familiar from reward shaping. The finite forecast-span theorem explains which continuation fields matter for aggregate invariance and identifies the first horizon of failure. Folk theorems address equilibrium implementation under additional assumptions. The observation quotient uses the strong lumpability block condition, and the sheaf interpretation uses local existence and gluing to distinguish contextual from uniform horizons. The case studies use reduced-form mechanisms related to value creation, reputation, and relational contracting.

Selected references

Evidence and authorship

The paper supplies mathematical proofs and symbolic illustrations. The examples assign mechanisms to hypothetical parameter values and make no empirical estimate for the markets in the original essay. The byline is anonymous for review. Generative AI assisted with drafting and literature discovery. The responsible human author or research group is to be identified to the journal as required by its workflow.

Reproducibility artifacts

competition-and-surplus-in-long-term-iterated-games.pdf · 373,631 bytes · Download
SHA-256 44987af489a6981bfec3c0b67854435e66d53f198e49cdde00101783343fa749

Discussion

Open a careful, constructive conversation about this note.

Comments appear after editorial approval.