Introduction

Reinforcement learning trading agents learn policies (decision rules) from market data. However, policies are opaque: difficult to understand why agents take specific actions. Interpreting policies via decision trees—fitting interpretable models to mimic RL policy decisions—enables understanding of learned trading logic.

Policy Distillation

Generate rollouts of RL agent: observe agent decisions in various market states, record market features and actions taken. Train decision trees on (state, action) pairs to predict what RL agent does in each state. Decision tree provides interpretable proxy for RL policy: specific market conditions → specific actions (e.g., "VIX > 25, momentum negative → reduce position").

Interpretation and Trust

Decision tree shows explicit decision rules; traders can understand and potentially override agent decisions if they seem unsafe. Enables identification of problematic policies (e.g., agent overresponding to noise, ignoring important signals). Builds trader confidence in AI-assisted decisions.

Conclusion

Policy distillation via decision trees enables interpretation of opaque RL agents, improving transparency and trust.