What a learned policy holds still

A learned robot policy is often studied by what can be read out of it. A probe trained on the policy's internal state shows which quantities are decodable. Decodability shows that information is present. It does not show that the policy uses the quantity^[1], and it shows less still about whether the policy acts to keep that quantity steady when the world pushes back.

Control theory asks a different question of any regulator: what does it resist? The answer names the variables the system is organised around, whatever its builders intended. For an organism that answer is hard to get, because an organism cannot easily be disturbed along any chosen direction, cheaply and repeatedly. A frozen policy running in a simulator can.

The method was developed for organisms^[2]; whether it carries into a trained network is the question.

The theory behind the test is old. Powers's 1973 book presents perceptual control theory, which treats behaviour as the control of perception^[2]. The test for the controlled variable follows from it: apply a disturbance to a candidate variable and see whether the system acts to cancel the effect. The obvious candidate is often the wrong one. In a pursuit-tracking study, the vertical distance between cursor and target looked like the controlled quantity, and a control model pointed to the angle between them instead^[3]. A separate line of work builds algorithms that identify the quantities a system regulates, tested on simulated biological models and benchmark physical systems^[4].

On the machine side, probing is a standard way to ask what a network represents, and its limits are documented. In one language-model study, probe accuracy did not track how much a property mattered to the task^[1]. For robot policies, recent work intervenes on internal activations instead. One study finds directions in the activations of OpenVLA and Pi0, two vision-language-action robot policies, that steer behaviour at inference time, tested in simulation and on a physical arm^[5]. Those interventions act inside the network. The test described here acts on the scene.

Robot evaluation also perturbs conditions. One benchmark pairs capability tests with stress tests for vision-language-action policies and reports weaknesses that success rates hide^[6]. A survival score says whether a policy copes. The controlled-variable test asks which variable the coping serves.

The design has two stages. The second runs only if the first passes.

Stage one uses one policy and one task: OpenVLA^[7] running closed-loop in simulation on a single task from LIBERO, a public suite of simulated robot-manipulation tasks^[8]. Disturbances are applied mid-episode, in graded sizes, along a set of directions in the scene chosen to be independent of one another (orthonormal), not along named variables. Named variables are physically coupled in a scene, so none can be moved alone. For each direction the harness records how strongly the policy's actions oppose the push. Resistance is measured against a matched-effort control, an arm that spends the same effort without targeting the disturbed direction. Control is scored as specificity: opposition in the disturbed direction against opposition in the directions left alone.

A floor on specificity-corrected resistance will be fixed in a dated pre-registration before any data is read. The working example is 0.2; the pre-registration will set the value.

Stage two takes five checkpoints across two simulators and repeats the sweep. The signature, the low-dimensional set of directions a policy defends, is used to predict which of twelve held-out disturbance conditions that policy survives. The comparators are a linear probe decoding the same variables from the same internal state, and, as a simpler baseline, frozen-action deviation alone. Prediction is scored out of sample by area under the ROC curve (AUC): the chance that a randomly chosen condition the policy survives is ranked above one it fails, where 0.5 is chance and 1.0 is perfect.

The first practical test is whether the harness can push one direction without contact, slip or a regrasp altering the physics. If it cannot, there is nothing to measure.

Two checks come before stage one. The harness must push one scene direction mid-episode without contact, slip or a regrasp changing the physics, and the policy must come close, on the LIBERO suite its task is drawn from, to the closed-loop success rate published for its checkpoint, within a tolerance to be set in the pre-registration. If either check fails, the inquiry stops there, recorded as a failure of the set-up, not as a finding about the policy.

Stage one stops the inquiry if specificity-corrected resistance, measured against the matched-effort control, does not separate physically coupled directions by more than the pre-registered floor. The likely reading is that one corrective move restores several linked variables at once, so apparent defence is collateral physics. The floor is not adjusted afterwards.

Stage one passes if the separation exceeds the floor against the matched-effort control. Stage two then begins.

Stage two stops the inquiry if out-of-sample AUC for predicting survival on the twelve held-out conditions is 0.65 or below, or if the signature shows no positive predictive margin over the linear probe, judged by a test to be fixed in the pre-registration. Stage two passes only if neither holds. If it passes but the signature predicts survival no better than frozen-action deviation alone, the result is recorded as a pass with no gain over the simpler baseline, and no claim is made for the signature over it. Failing the probe comparison would mean that what a policy represents predicts its limits as well as what it defends.

Inconclusive. At stage one, the interval on specificity-corrected resistance includes the floor. At stage two, the interval on AUC includes 0.65, or the interval on the margin over the probe includes both no effect and the pre-registered margin. Reported as such, not as a pass or a stop.

Both stages must pass before any claim that the identified variables make a useful training target. A pass is a reason to continue, not a finding about robots in general.

Several things other than control could make a policy look as if it defends a direction. The main one is physical coupling: one corrective move restores several linked variables, so opposition appears in directions that were never pushed. Pushing along orthonormal directions and scoring specificity, opposition in the pushed direction against opposition elsewhere, is aimed at this. A second is general reactivity. A policy that moves harder after any push will oppose every push; the matched-effort control is designed to spend the same effort without targeting the pushed direction, so effort alone does not count. A third is the harness itself. If a push causes contact, slip or a regrasp, the physics decides what happens next, which is why the aim test comes before stage one. At stage two, survival might be predicted by what the policy represents rather than what it defends. The linear probe on the same internal state will test this, and the signature must beat it. The comparison is only as fair as the probe, since how much a probe reads out depends partly on the probe and not only on the representation^[9]; the probe will therefore be fixed in the pre-registration and not changed afterwards. Last, a designer's choice of twelve conditions could favour the signature. Fixing the conditions before any sweep is read, and scoring out of sample, is meant to limit this, though it cannot remove it.

A pass would show, for a few public checkpoints in simulation, that a disturbance sweep recovers low-dimensional structure that predicts survival under twelve chosen shifts better than a probe does. It would not show that a policy holds a goal, a belief or anything like perception in an organism. It would not carry over to other policies, other tasks, a physical robot or contact-rich manipulation. The twelve conditions are chosen by the designer, and survival under them says little about conditions nobody listed. The word “defend” labels opposition measured in the sweep; whether that licenses the language of control stays open. Resemblance between a trained network and a regulating organism remains a hypothesis after the test, whatever its outcome.