Dynamic POV Algorithm Using Reinforcement Learning

Percentage of Volume (POV) algorithms execute a fixed percentage of market volume passing by, e.g., "execute 20% of all trades in this stock." This adaptively adjusts execution pace to market activity: when trading is active, the algorithm executes more; when quiet, it executes less. However, fixed percentages are rigid. Machine learning can dynamically optimize the POV ratio based on market conditions, improving execution quality.

Traditional POV Algorithms

POV targets a fixed fraction of observed volume. Example logic:

• Observe all trades in the market (aggregate volume)
• Every 100ms, execute X% of observed trades in that interval
• Repeat until order complete

Advantage: automatically adapts execution pace to market activity (busier markets → faster execution, vice versa).

Disadvantage: fixed percentage may be suboptimal. During low-participation periods, executing at fixed POV means slow execution and adverse selection. During high-participation periods, executing at fixed POV might be too passive.

Optimal POV Selection

The optimal POV ratio depends on:

• Information leakage: participating more aggressively reveals the order to the market, inviting adverse selection
• Participation efficiency: in liquid markets, executing more shares per unit time is beneficial
• Inventory management: if our position is tilted, more aggressive execution might be necessary
• Time pressure: if execution deadline is approaching, increase POV ratio

These factors suggest the optimal ratio varies over time and state.

Reinforcement Learning Formulation

State = (remaining quantity, time remaining, current market volume, recent volatility, position drift). Action = POV ratio (continuous value, e.g., 5%-50%). Reward = execution quality (average price - arrival price - information leakage penalty).

The RL agent learns which POV ratios maximize long-term execution quality across different states.

Key Learning Patterns

Over many training episodes, the agent learns intuitive patterns:

• When time is short, increase POV (execute faster before deadline)
• When volatility is high, increase POV (take advantage of potential favorable moves)
• When inventory imbalance is large, increase POV (correct imbalance faster)
• When market volume is low, decrease POV (avoid sending too much to thin market)

Multi-Asset Learning

Some algorithms execute across multiple related securities. A single RL agent can learn optimal POV for each asset, accounting for correlations between assets. The agent learns that executing more aggressively in one asset can be compensated by being more passive in correlated assets.

Handling Information Leakage

More aggressive participation (higher POV) reveals information faster, allowing other traders to front-run. The RL agent must balance the benefit of faster execution against the cost of information leakage.

By learning from realized execution outcomes, the agent discovers where this balance lies for different asset-market-condition combinations.

Convergence and Stability

RL training can be unstable. The agent might discover a policy that works well on historical data but fails on live markets. Common techniques to improve stability:

• Experience replay: mix old and new data to maintain diverse training
• Target networks: maintain a separate, slowly-updated copy of the network to stabilize targets
• Entropy regularization: encourage exploratory behavior to avoid premature convergence

Practical Deployment

In production, the learned POV policy is typically one component of a larger execution system. Other algorithms (smart VWAP, TWAP, list execution order optimization) provide alternatives, and the system selects the best approach for each situation.

Conclusion

Dynamic POV via reinforcement learning optimizes execution pace based on market conditions and state, achieving better results than fixed-ratio policies. The approach exemplifies how RL handles complex, multi-factor tradeoffs in trading.