Independent scholarship · Open accessThe monthly digest

Editorially reviewed

Latent-Cache Programs for Branching Reinforcement Learning

We extend Octopus with causal descent, a proof of packed forward and reverse equivalence, explicit episode and branch credit, and cloud replay contracts. A Lean compiler and Torch worker execute the smooth and causal plans on a reference hybrid model. The paper specifies matched-budget experiments for the original empirical results.

Anonymous← All research

Main result

An agent episode has a chronological event log, a private prefix tree, and an acyclic tensor dependency graph. Latent reads add attention edges while retaining the reader's private recurrence checkpoint. Rewind restores a cursor and preserves earlier policy choices in the immutable log. We extend Octopus (arXiv:2611.06956v1) with causal sheaves of histories, exact packed evaluation and pullbacks, explicit policy and intervention estimands, and a Lean-to-tensor compiler.

Causal and smooth semantics

A history is a downset of the causal order. Any compatible family of local state assignments on a union cover glues uniquely because the member containing an event contains all its prerequisites. For spaces the extension of a principal-history diagram $F$ to $U$ is $\lim_{v\in U^{\mathrm{op}}} F(v)$. This is $\operatorname{Map}(1_U,F)$ by the colimit of the representables below $U$. A cover is a pointwise effective epimorphism on the corresponding subterminal presheaves, so mapping its Cech nerve into $F$ proves descent. The smooth map of a typed open diagram composes by directed interface gluing.

On fixed discrete traces, a packed pass agrees with an occurrence-based reference pass when every layer sees the same parent state, ports, visibility and native positions. The proof is an induction on layers and then on the private ancestry at recurrent layers. At attention, an earlier prefix of length $n$ followed by a local causal block exposes exactly the columns $j\le n+i$ at query row $i$. For a mount with last native position $k$, $\max(p,k+1)$ is the least legal next position. Splitting a visible attention row into blocks preserves its output by merging their stabilized exponential masses and weighted values. The output ratio agrees as a smooth function, which gives equal pullbacks. A producer cotangent sums once over each consuming port, including mounted slices.

Credit and evidence

With fixed exogenous transition kernels, the episode score is the sum of log-probability derivatives for owned sampled actions. Inherited prefixes, borrowed tensors and forced tokens carry no new score. A mask retaining each sampled occurrence once reproduces the exact episode score. The unnormalized leave-one-episode-out update is unbiased under conditionally independent group sampling. A random episode-size denominator changes the estimator, even at the behavior weights. The manuscript gives the resulting identity, an explicit counterexample, an exact joint likelihood ratio, and a smooth approximation to the clipped surrogate.

The source landing abstract also describes leave-one-branch-out credit. The manuscript defines it by suppressing a branch capability and rerunning downstream behavior under an explicit fallback and a coupled exogenous seed. This is a separate causal reward estimand. The full-report group baseline and the branch-removal effect have different sample units.

The attached PDF covers every method section, empirical result family, and appendix of the source, with a coverage matrix. It extends the source experiments through controlled estimands and measurement protocols for search accuracy, context use, selective mounting, branching and distributed training. Source benchmark figures are attributed to the report.

Reproducibility

The accompanying BranchCloud source includes a shape-checked smooth lambda calculus, a branch trace compiler, a resident Torch evaluator, packed reverse training, a CPU witness and a GPU Job manifest. The checked witness retains a rewind action, compares resident and packed evaluation, and isolates a live reader-to-producer gradient. The small CPU run records rollout and replay scores and verifies checkpoint resume. The PDF documents the architecture and research-scale evaluation boundaries.

Authorship

Prepared for anonymous review. The responsible author or group is to be confirmed in the submission record. An AI assistant contributed to drafting, implementation, and artifact preparation.

Primary references

Reproducibility artifacts

latent-cache-rl.pdf · 580,195 bytes · Download
SHA-256 cc7b961ed30ff7dc12625b10b20ee8c8645f8a52589c09297a7afa8d26e76d57

branchcloud-hcs-submission.zip · 74,057 bytes · Download
SHA-256 2ae0ab984d3b22adb4d3a3ae66525e08e3f6cccd12c2e40d6ad50d988a644e1f

Discussion

Open a careful, constructive conversation about this note.

Comments appear after editorial approval.