Calibrated for Which Policy? Conformal Uncertainty for Fixed World Models Under Intervention

Introduction

When a robotic system or any decision‑making agent updates its deployment policy, the underlying world model often remains unchanged. Traditional uncertainty quantification assumes the data‑generating distribution is static, so a shift in policy can invalidate calibrated confidence sets. The recent work on Conformal Uncertainty for Fixed World Models Under Intervention shows how to retain reliable coverage by re‑weighting conformal predictions with an estimated density‑ratio that captures the change from the source to the target policy.

From Policy Shift to Density Ratio

Consider a one‑step prediction problem where the conditional mechanism \(P(Y\mid X)=Q(Y\mid X)\) stays the same across environments, but the marginal distribution of the context \(X\) changes because a new policy alters state visitation and action selection. The intervention can be expressed as a Radon‑Nikodym derivative \[ w(X)=\frac{dQX}{dPX}, \] which decomposes into a product of a state‑occupancy factor and an action‑propensity factor when the dynamics are policy‑controlled. Intuitively, \(w\) tells us how much more (or less) likely a particular context is under the target policy compared to the source.

Weighted Conformal Calibration

Standard conformal prediction builds a set \(C(X)\) that contains the true outcome \(Y\) with probability at least \(1-\alpha\) under the source distribution \(P\). To transport this guarantee to the target law \(Q\), we weight each calibration example by \(w\). The resulting weighted conformal set \(C{\hat w}(X)\) satisfies \[ PQ\{Y \in C{\hat w}(X)\} \ge 1-\alpha\,\text{TV}(Q,\hat Q) \ge 1-\alpha\,EP|w-\hat w|, \] where \(\hat w\) is an estimator of the true ratio. Thus, the coverage loss is bounded by the total‑variation distance between the true and estimated target distributions, which in turn is controlled by the estimation error of \(w\).

Estimating the Ratio via Classification

A practical way to obtain \(\hat w\) is to train a binary classifier that distinguishes source from target samples. Let \(h\) be the clipped posterior of the target class; its population log‑loss regret \(\Delta(h)\) measures how well the classifier separates the two domains. The analysis shows \[ EP|w-\hat wh| \le \frac{1}{\rho\gamma}\sqrt{\frac{\Delta(h)}{2}}, \] where \(\rho\) is the target‑domain prior and \(\gamma\) is a lower bound on the classifier’s odds. This bound improves over generic odds‑map results, reducing the dependence from \(\gamma^{-2}\) to \((\rho\gamma)^{-1}\).

Empirical Illustration

The authors evaluate the method on a DINO‑feature world model paired with real‑world Franka robot trajectories. Under a strong state‑action distribution shift, ordinary 90 % conformal sets cover only about 55 % of the outcomes. After applying the learned‑ratio calibration, coverage rises to roughly 91 %, matching an oracle that knows the exact density ratio. Moreover, the surrogate ratio estimator stays within 0.9 % total variation of the true target law, confirming the theoretical guarantees.

Practical Takeaways

1. Separate Model and Calibration – Keep the predictive model fixed; adjust only the uncertainty calibration when policies change.
2. Use Weighted Conformal Scores – Multiply each calibration residual by the estimated density ratio to transport coverage.
3. Leverage Simple Classifiers – A binary classifier trained on source vs. target data provides a tractable way to estimate \(w\) and bound its error via log‑loss regret.
4. Validate Coverage Empirically – Even under large distribution shifts, weighted conformal sets can recover nominal coverage levels.

Conclusion

Weighted conformal calibration offers a principled, data‑efficient solution for maintaining reliable uncertainty estimates when deployment policies evolve. By framing the policy shift as a change‑of‑measure problem and estimating the associated density ratio through classification, practitioners can preserve coverage guarantees without retraining the underlying world model. This approach bridges the gap between static uncertainty quantification and the dynamic realities of modern robotic and reinforcement‑learning systems.