MemCast
MemCast / episode / insight
Financial markets provide a small, noisy data set that makes overfitting especially dangerous
  • Unlike language models with billions of data points, traders have only ~10,000 tradable instruments and relatively short histories.
  • This scarcity means a model can easily capture noise rather than signal.
  • Overfitting occurs when a strategy works on past data but fails when market dynamics change.
  • Nang warns that “there is a small data problem and it keeps changing,” urging constant model validation.
Rishi NangTitans Of Tomorrow00:15:46

Supporting quotes

You also have very little data compared to these other things, right? So, if you're looking at language models... there is no big data problem in our world. There's a small data problem and it keeps changing. Rishi Nang
So, it's a much noisier, much more chaotic system in which to predict. Rishi Nang

From this concept

Overfitting, Underfitting & The Small-Data Problem

Nang explains why traditional machine-learning pitfalls are amplified in finance: limited data, regime shifts, and the temptation to over-customise models for specific assets.

View full episode →

Similar insights