Million‑fold compute increase enables massive spatial models

Modern training clusters can marshal a million‑fold more FLOPs than were available at the start of Fei‑Fei’s PhD.
This scale makes it feasible to train models that ingest and reason over billions of pixels of visual data.
The sheer compute budget also lets researchers experiment with richer loss functions that capture geometry, physics, and semantics simultaneously.
As a result, world‑scale generative models like Marble become practical rather than speculative.

Fei-Fei LiLatent Space00:04:28

Supporting quotes

Quote

“The amount of compute that we can marshal today on a single model is about a millionfold more than we could have even at the start of my PhD.” — Fei-Fei Li

Quote

00:13:02

“Performance per watt from Hopper to Blackwell shows scaling limits; we need new architectures.” — Fei-Fei Li

From this concept

Compute Scaling as the Engine of Progress

The guests argue that every major leap in AI has been driven by orders-of-magnitude increases in compute. The exponential growth in GPU performance and the ability to train on thousands of devices unlocks the data-hungry spatial models that were impossible a decade ago.

View full episode →

Similar insights

“Compute scaling fuels the leap from ImageNet to world models”

Fei-Fei LiLatent Space

“GPU performance per watt is hitting limits, prompting new hardware primitives”

Fei-Fei LiLatent Space

“Public recognition lags far behind actual AI progress”

Dario AmodeiDwarkesh Patel