Long-horizon LLM agents often fail not because relevant information is absent, but because semantically relevant memories are epistemically invalid: stale observations, failed tool traces, or claims contradicted by newer verified evidence are retrieved and treated as support. We introduce HELM (Hierarchical Epistemic Learned Memory), a memory framework that makes this reliability problem explicit. HELM exposes memory through logged read/write/consolidate/prune operations and stores each item with provenance and epistemic state. Its three-tier Semantic-HNSW Nested Memory (SHNM) links episodic tool traces to consolidated recalls and thematic indices, enabling coarse-to-fine retrieval that can be audited back to concrete evidence spans. Epistemic governance filters and re-ranks recalls by status, freshness, consistency, recency, and provenance, while a lightweight learned controller decides when memory operations are worth their cost. Across five long-horizon benchmarks, HELM improves success over strong memory baselines on GAIA, WebArena, Mind2Web, StreamBench, and LifelongAgentBench while reducing effective context tokens and epistemic errors. New scaling, training-free, and governance-only ablations show that the main gains persist with a 32B backbone and that epistemic governance alone explains a large share of the reliability improvement.

On-policy distillation (OPD) for long-horizon language agents fails even when the teacher remains sequence-informative: in a failing Qwen3 configuration, the teacher still separates good from bad rollouts (sequence AUROC ≈ 0.75), yet the local token-gradient alignment 𝛼_diag decays from 0.71 to 0.09 and standard OPD recovers only 1.3% of the teacher–student gap. We restrict our claims to two coupled mechanisms, jointly termed CHOP (Counterfactual Hierarchical On-Policy distillation): (i) a per-state reliability gate routing between per-token reverse Kullback–Leibler matching and per-segment sliced-Wasserstein matching, and (ii) a counterfactual total-variation gate activating privileged information (PI) only when the PI changes the teacher’s distribution. The scope is explicit: CHOP targets low-reliability, long-horizon traces where token-local support matching plateaus; in short-horizon or high-reliability regimes Top-K local-support matching (LSM) is already sufficient and CHOP’s segment branch is rarely activated. CHOP recovers 31.7% of the anisotropic-Qwen3 gap (vs. 1.3%/14.5%/24.7% for OPD/Top-K LSM/Li et al. 2026); on SWE-bench Verified under a fixed OpenHands harness, CHOP-deployable prevents the OPD-induced regression (6.5%\!→\!13.2%, matched to off-the-shelf 11.8%, not a significant gain over off-the-shelf) and oracle PI reveals further headroom (14.2%, 95% CI [11.4,17.3]); on AIME 2024 at a 60K-token stress cap (a diagnostic horizon, not a deployment target), CHOP holds 44.9 vs. 42.8 for Top-K LSM (p=0.032), while standard OPD collapses to 28.4. Compute-matched, leakage, and PI-removed-teacher controls attribute the gain to supervision-object routing rather than extra compute or PI contamination.