Introduction

Standard neural networks produce point predictions. Bayesian neural networks treat weights as random variables with posterior distributions. This enables uncertainty quantification: not just "predict tomorrow's return is 0.5%" but "0.5% ± 1.2% (95% CI)."

Bayesian Neural Network Basics

Place prior distributions on weights (e.g., Gaussian). After observing training data, compute posterior P(weights|data). Predictions are averages over posterior: E[y|x, data] = integral P(y|x, w) P(w|data) dw.

This integral is intractable for large networks. Approximations include variational inference (optimize a tractable distribution close to posterior) and MCMC sampling (expensive but accurate).

Practical Approximation: Dropout as Bayesian Inference

A clever observation: applying dropout at test time (not just training) approximates Bayesian inference. Each forward pass with dropout samples a network from posterior. Multiple forward passes with dropout provide uncertainty estimates.

This is fast (no special training required) and works with any trained neural network. Empirically achieves 80-90% of full Bayesian uncertainty quality with 5-10% computational overhead.

Application: Stock Return Prediction with Uncertainty

Train LSTM on S&P 500 daily returns. At test time, run 50 forward passes with dropout enabled. For each input, compute mean (point prediction) and std (uncertainty). On test data: mean prediction RMSE 2.1%, calibrated uncertainty intervals cover true values 92% of time (target: 95%).

Uncertainty Estimation Quality

Miscalibration occurs when predicted uncertainty doesn't match actual error distribution. Properly calibrated: 95% of true values fall within 95% confidence intervals. Test calibration using coverage plots: plot actual error quantiles vs predicted quantiles; properly calibrated models show diagonal lines.

Practical Implementation

1. Train neural network normally (LSTM, feedforward, CNN).
2. At inference, enable dropout.
3. Run N forward passes (N=50-100).
4. Compute mean and std of outputs across passes.
5. Use mean for predictions, std for uncertainty bounds.

Applications in Portfolio Risk Management

Use uncertainty estimates for position sizing: tighter intervals enable higher leverage; wider intervals suggest position reduction. Backtesting a strategy that sizes positions inversely to predicted uncertainty achieved 1.2 Sharpe ratio versus 0.95 for fixed sizing.

Limitations

Dropout approximation underestimates uncertainty for out-of-distribution inputs (market regimes model never saw). Use confidence thresholds: if uncertainty exceeds threshold, reduce position or pass trade. Periodically retrain to incorporate new regimes.

Advanced Methods

Bayes-by-backprop: explicitly train weight posteriors via variational inference. More principled than dropout but computationally expensive. Ensemble methods: train multiple independent networks, use ensemble disagreement as uncertainty estimate. Practical and interpretable.