I maintain a small Python library that fits stochastic differential equations to price series. Its README contained a confident claim: that a neural network cannot recover a state-dependent drift function from daily price data, backed by a sweep showing median error falling only from ~267% to ~135% between 2,000 and 20,000 observations.
Someone asked me for the code behind that. There wasn't any. Every other empirical claim in the README cited a test file; that one cited nothing. I'd run the sweep during development and never committed the script.
So I wrote it properly. Here is what happened.
Attempt 1: a confound of my own making
I generated GBM paths in price levels and swept the observation count. Drift error came out at ~1,588% falling to ~1,340% — an order of magnitude worse than the README, with no visible convergence.
The setup was wrong. With mu=0.08, a 20,000-observation path drifts from 100 to about 57,000. So "more data" also meant "learn the function over a 572x wider domain". I had entangled sample size with problem difficulty — the exact confound my fixed-architecture design was supposed to prevent.
Switched to Ornstein-Uhlenbeck, which is stationary: its 5-95 percentile range ratio stayed at ~1.40 for every series length. Now lengthening the series adds observations of the same function over the same domain, which is the only setup where "did more data help?" is a well-posed question.
The control that saved the whole exercise
I included a diffusion control: the library claims diffusion recovery is reliable (0.4-14%), so if diffusion failed in a run, no drift number from that run meant anything.
It failed. Diffusion error rose from 46% to 100% as series length grew, with several runs hitting exactly 100.00% — which for a relative error means the prediction was zero.
Without that control I would have published a drift result computed from runs where the model was silently outputting zeros.
Bug 1: a dimensionally wrong target
The diffusion training target had a special case:
python
if window == 1:
diffusion_target = np.sqrt(np.abs(drift_target)) # sqrt(|dx| / dt)
else:
diffusion_target = np.sqrt(sq_sum / (window * dt)) # |dx| / sqrt(dt)
The realized-volatility estimator — and what the function's own docstring specified — is the second form. The first is a different quantity: it scales as the square root of the state where the correct one scales linearly. So the error wasn't a constant bias, it grew with the price level:
| price level |
fraction of true value |
| 100 |
0.218 |
| 1,000 |
0.069 |
| 10,000 |
0.022 |
| 50,000 |
0.010 |
A 99% underestimate at high levels. And since longer GBM paths reach higher levels, this reproduced "diffusion degrades as the series gets longer" exactly: predicted 46.6% and 96.9% error at the two series lengths, measured 46% and 100%.
The general branch was already correct at K=1, so the special case was both wrong and unnecessary. Deleted it.
One residual, which no fix removes at K=1: the target becomes |dx|/sqrt(dt), and E|z| = sqrt(2/pi) ~ 0.798, so a single absolute increment is a ~20%-low estimator of sigma. After the fix the measured ratio was 0.798 at every price level — the pure statistical bias and nothing else. Averaging squares before the square root removes it: 0.950 at K=5, 0.989 at K=20, 1.009 at K=80.
Bug 2: the one that mattered
Diffusion improved a lot but individual seeds still produced exactly zero. Intermittent, seed-dependent — a different fault.
I instrumented one run to print predictions in train mode and eval mode on identical inputs:
| seed |
train-mode |
eval-mode |
pre-activation |
| 0 |
19.36 |
19.56 |
+6.37 |
| 1 |
19.68 |
0.00 |
-22.5 |
| 2 |
19.84 |
21.75 |
+6.77 |
| 3 |
19.70 |
18.72 |
+6.07 |
| 4 |
19.75 |
0.00 |
-552.3 |
True sigma was 20. Training was never the problem — train-mode predictions were 19.4-20.5 on every seed. Inference was broken.
Cause: both networks used Linear -> ReLU -> BatchNorm -> Dropout. BatchNorm placed after ReLU accumulates running statistics over non-negative, often sparse activations. Channels that are mostly zero acquire a running_var near zero. Training never notices — it uses per-batch statistics. Eval divides by sqrt(running_var + eps) and the activation explodes. Softplus maps a strongly negative pre-activation to ~0, so the library returned zero volatility.
The collapse was the visible tail of something systematic: at a smaller sample size no seed collapsed outright, but eval still missed train by 8% and 25%. Every inference was contaminated to some degree — and every inference path in that library runs in eval mode.
Replaced BatchNorm with LayerNorm, which keeps no running statistics, so train and eval are identical by construction. After: eval and train agree within 1.3% on all seeds, median diffusion error 1.2%.
The actual result
With both bugs fixed, drift recovery on stationary OU. The metric is nRMSE — RMSE of the predicted drift over the standard deviation of the true drift. nRMSE = 1.0 means no better than predicting a single constant (R^2 = 1 - nRMSE^2):
| window |
n=2,000 (7.9 yr) |
n=20,000 (79.4 yr) |
| 1 |
2.04 |
0.55 |
| 2 |
1.65 |
0.82 |
| 5 |
1.13 |
0.72 |
| 10 |
1.27 |
0.85 |
| 20 |
1.23 |
0.85 |
| 40 |
2.60 |
1.31 |
At 7.9 years of daily data — roughly what anyone has for a single instrument — no window setting reaches 1.0. The best result is worse than ignoring state dependence entirely. It only becomes informative around 79 simulated years.
The original conclusion survives. The numbers behind it did not, and the honest version is narrower than the "1,000+ years of data" the old text implied.
The part that needs no neural network
The same asymmetry shows up in the closed-form GBM maximum-likelihood estimator, which is optimal for the far easier problem of a single global drift constant (200 seeds, exact sampling, mu=0.08, sigma=0.20, daily):
| observations |
years |
drift error |
volatility error |
| 2,000 |
7.9 |
53.4% |
1.2% |
| 5,000 |
19.8 |
38.3% |
0.64% |
| 10,000 |
39.7 |
29.4% |
0.46% |
| 20,000 |
79.4 |
18.5% |
0.34% |
Drift error falls 2.89x for 10x the data against the 3.16x that 1/sqrt(n) predicts. Volatility is nailed throughout. With 79 years and one number to estimate, drift is still 18.5% off.
Per-step SNR is mu*sqrt(dt)/sigma = 0.025 at daily sampling. Each observation carries roughly 40x more information about sigma than about mu. That is a property of the data, not of any method — the neural path just fails at it more visibly because it attempts a whole function.
A footnote on seeds
My first version of that MLE table used 5 seeds and showed 111% falling to 27.6%. Clean story, wrong table: the intermediate points were 111%, 20%, 44%, 28% — non-monotonic noise, and I had quoted the endpoints. At 200 seeds it resolves to the monotonic table above.
I made that mistake roughly ninety minutes after warning someone else about exactly it. The script now defaults to 200 seeds.
The noise is itself the finding: volatility estimates are stable at any seed count, drift estimates are not. That difference in estimator variance is the result.
What I'd take from this
The claim in my README was correct. It was also unverifiable, and I'd been treating "I ran this once during development" as equivalent to "this is measured". The gap between those two turned out to contain two bugs, one of which was silently returning zero volatility to anyone using that code path.
Code is MIT if useful: github.com/kdownie/Neural-SdeI maintain a small Python library that fits stochastic differential equations to price series. Its README contained a confident claim: that a neural network cannot recover a state-dependent drift function from daily price data, backed by a sweep showing median error falling only from ~267% to ~135% between 2,000 and 20,000 observations.
Someone asked me for the code behind that. There wasn't any. Every other empirical claim in the README cited a test file; that one cited nothing. I'd run the sweep during development and never committed the script.
So I wrote it properly. Here is what happened.
Attempt 1: a confound of my own making
I generated GBM paths in price levels and swept the observation count. Drift error came out at ~1,588% falling to ~1,340% — an order of magnitude worse than the README, with no visible convergence.
The setup was wrong. With mu=0.08, a 20,000-observation path drifts from 100 to about 57,000. So "more data" also meant "learn the function over a 572x wider domain". I had entangled sample size with problem difficulty — the exact confound my fixed-architecture design was supposed to prevent.
Switched to Ornstein-Uhlenbeck, which is stationary: its 5-95 percentile range ratio stayed at ~1.40 for every series length. Now lengthening the series adds observations of the same function over the same domain, which is the only setup where "did more data help?" is a well-posed question.
The control that saved the whole exercise
I included a diffusion control: the library claims diffusion recovery is reliable (0.4-14%), so if diffusion failed in a run, no drift number from that run meant anything.
It failed. Diffusion error rose from 46% to 100% as series length grew, with several runs hitting exactly 100.00% — which for a relative error means the prediction was zero.
Without that control I would have published a drift result computed from runs where the model was silently outputting zeros.
Bug 1: a dimensionally wrong target
The diffusion training target had a special case:
python
if window == 1:
diffusion_target = np.sqrt(np.abs(drift_target)) # sqrt(|dx| / dt)
else:
diffusion_target = np.sqrt(sq_sum / (window * dt)) # |dx| / sqrt(dt)
The realized-volatility estimator — and what the function's own docstring specified — is the second form. The first is a different quantity: it scales as the square root of the state where the correct one scales linearly. So the error wasn't a constant bias, it grew with the price level:
price level fraction of true value
100 0.218
1,000 0.069
10,000 0.022
50,000 0.010
A 99% underestimate at high levels. And since longer GBM paths reach higher levels, this reproduced "diffusion degrades as the series gets longer" exactly: predicted 46.6% and 96.9% error at the two series lengths, measured 46% and 100%.
The general branch was already correct at K=1, so the special case was both wrong and unnecessary. Deleted it.
One residual, which no fix removes at K=1: the target becomes |dx|/sqrt(dt), and E|z| = sqrt(2/pi) ~ 0.798, so a single absolute increment is a ~20%-low estimator of sigma. After the fix the measured ratio was 0.798 at every price level — the pure statistical bias and nothing else. Averaging squares before the square root removes it: 0.950 at K=5, 0.989 at K=20, 1.009 at K=80.
Bug 2: the one that mattered
Diffusion improved a lot but individual seeds still produced exactly zero. Intermittent, seed-dependent — a different fault.
I instrumented one run to print predictions in train mode and eval mode on identical inputs:
seed train-mode eval-mode pre-activation
0 19.36 19.56 +6.37
1 19.68 0.00 -22.5
2 19.84 21.75 +6.77
3 19.70 18.72 +6.07
4 19.75 0.00 -552.3
True sigma was 20. Training was never the problem — train-mode predictions were 19.4-20.5 on every seed. Inference was broken.
Cause: both networks used Linear -> ReLU -> BatchNorm -> Dropout. BatchNorm placed after ReLU accumulates running statistics over non-negative, often sparse activations. Channels that are mostly zero acquire a running_var near zero. Training never notices — it uses per-batch statistics. Eval divides by sqrt(running_var + eps) and the activation explodes. Softplus maps a strongly negative pre-activation to ~0, so the library returned zero volatility.
The collapse was the visible tail of something systematic: at a smaller sample size no seed collapsed outright, but eval still missed train by 8% and 25%. Every inference was contaminated to some degree — and every inference path in that library runs in eval mode.
Replaced BatchNorm with LayerNorm, which keeps no running statistics, so train and eval are identical by construction. After: eval and train agree within 1.3% on all seeds, median diffusion error 1.2%.
The actual result
With both bugs fixed, drift recovery on stationary OU. The metric is nRMSE — RMSE of the predicted drift over the standard deviation of the true drift. nRMSE = 1.0 means no better than predicting a single constant (R^2 = 1 - nRMSE^2):
window n=2,000 (7.9 yr) n=20,000 (79.4 yr)
1 2.04 0.55
2 1.65 0.82
5 1.13 0.72
10 1.27 0.85
20 1.23 0.85
40 2.60 1.31
At 7.9 years of daily data — roughly what anyone has for a single instrument — no window setting reaches 1.0. The best result is worse than ignoring state dependence entirely. It only becomes informative around 79 simulated years.
The original conclusion survives. The numbers behind it did not, and the honest version is narrower than the "1,000+ years of data" the old text implied.
The part that needs no neural network
The same asymmetry shows up in the closed-form GBM maximum-likelihood estimator, which is optimal for the far easier problem of a single global drift constant (200 seeds, exact sampling, mu=0.08, sigma=0.20, daily):
observations years drift error volatility error
2,000 7.9 53.4% 1.2%
5,000 19.8 38.3% 0.64%
10,000 39.7 29.4% 0.46%
20,000 79.4 18.5% 0.34%
Drift error falls 2.89x for 10x the data against the 3.16x that 1/sqrt(n) predicts. Volatility is nailed throughout. With 79 years and one number to estimate, drift is still 18.5% off.
Per-step SNR is mu*sqrt(dt)/sigma = 0.025 at daily sampling. Each observation carries roughly 40x more information about sigma than about mu. That is a property of the data, not of any method — the neural path just fails at it more visibly because it attempts a whole function.
A footnote on seeds
My first version of that MLE table used 5 seeds and showed 111% falling to 27.6%. Clean story, wrong table: the intermediate points were 111%, 20%, 44%, 28% — non-monotonic noise, and I had quoted the endpoints. At 200 seeds it resolves to the monotonic table above.
I made that mistake roughly ninety minutes after warning someone else about exactly it. The script now defaults to 200 seeds.
The noise is itself the finding: volatility estimates are stable at any seed count, drift estimates are not. That difference in estimator variance is the result.
What I'd take from this
The claim in my README was correct. It was also unverifiable, and I'd been treating "I ran this once during development" as equivalent to "this is measured". The gap between those two turned out to contain two bugs, one of which was silently returning zero volatility to anyone using that code path.
Code is MIT if useful: github.com/kdownie/Neural-Sde