Mind the Phase

Reinforcement learning is widely used in legged locomotion, enabling behaviors such as backflips and parkour through massively parallel simulation. Shallow networks remain common under the non-stationarity of PPO training, supported by carefully staged curricula and environments. However, their learned representations remain poorly understood, limiting the signals available during training to assess how a policy may behave on hardware.

We empirically study locomotion policies using the effective rank of the policy Jacobian. Conditioning this measure on gait phase reveals architectural differences that are obscured by global rank. Networks with layer normalization and residual connections have higher effective rank during swing than during stance; this swing-dominant pattern is absent in vanilla MLPs.

We use these representational signatures to guide an architecture and training recipe for smoother sim-to-real transfer. The reduction in joint jitter observed in simulation also holds on a physical Spot. These results suggest that phase-conditioned rank can help assess representation health during training, without treating rank itself as an optimization objective.