Introduction

Bayesian models provide uncertainty quantification but are computationally expensive: inference requires sampling or variational approximation. Online learning requires updating models with streaming data. Online variational inference (OVI) enables rapid Bayesian updates suitable for real-time trading.

Variational Inference Basics

Variational inference approximates posterior P(θ|X) with tractable Q(θ). Rather than sampling (slow), optimize Q to be close to P. Optimized Q provides both point estimates (mean) and uncertainty (variance). This is fast: updates in milliseconds rather than seconds.

Online Updates with Stochastic Variational Inference

Process one new observation at a time. Compute gradient of variational objective with respect to current observation. Update variational parameters via gradient step. This enables continuous learning without reprocessing historical data.

Key insight: stochastic updates are noisy but fast. For financial data with high noise anyway, fast approximate updates beat slow exact updates.

Application: Online Linear Regression for Returns

Model: R_t = β_0 + β_1*F_1(t) + β_2*F_2(t) + ε where F_1, F_2 are market features. Use Bayesian linear regression with Gaussian prior on β. As new daily data arrives, update posterior over β using one-sample variational updates.

This enables tracking β over time: β_0 (alpha) increases as manager outperforms; β_1 (beta) decreases as risk profile improves. Traders monitor these live estimates for rebalancing signals.

Empirical Performance

On daily S&P 500 data with online Bayesian regression: parameter estimates track true underlying regime changes with 1-2 day lag. Offline batch inference requires 100+ samples, taking hours; online inference updates in 50ms per new sample.

Trading strategy using online-estimated α and β achieved Sharpe ratio of 1.05 versus 0.85 for offline methods (updated daily), showing value of real-time adaptation.

Extensions to Non-Linear Models

Online variational inference extends to neural networks via Bayes-by-backprop: treat network weights as random variables with posterior distributions. Each new sample updates weight posteriors. This is expensive for large networks but feasible for medium-sized models (10-100k parameters).

Handling Non-Stationarity

Online updates naturally handle non-stationarity: recent data has more influence than old data (because learning rates don't decay). This contrasts with batch learning where all data is weighted equally. For financial time series, this is desirable behavior.

Implementation Considerations

Use Pyro or Edward for variational inference. Implement gradual weight decay: downweight old data by exponential discount factor (0.99 per day). Monitor posterior variance: increasing variance signals regime change and potential retraining trigger.

Scalability

Online variational inference scales to millions of observations and hundreds of parameters. For larger models (thousands of parameters), use approximate methods like Laplace approximation or ensembles of smaller models.