Skip to content

Welcome to Stochastic Thoughts

The Great Reversal

Comparing small AI models on performance vs cost. Credits: Artificial Analysis.
Comparing small AI models on performance vs cost. Credits: Artificial Analysis.

Something interesting is happening in AI. After years of “bigger is better,” we’re seeing a shift towards smaller, more efficient models. Mistral just released NeMo and OpenAI unveiled GPT40-mini. Google’s in on it too, with Gemini Flash. What’s going on?

It’s simple: we’ve hit diminishing returns with giant models.

Training massive AI models is expensive. Really expensive. We’re talking millions of dollars and enough energy to power a small town. For a while, this seemed worth it. Bigger models meant better performance, and tech giants were happy to foot the bill in the AI arms race.

But here’s the thing: throwing more parameters at the problem only gets you so far. It turns out that data quality matters way more than sheer model size. And high-quality data? That’s getting scarce.

The paradox of AI

I remember back in 1999 I got to try a speech recognition software for the first time. It required a training phase where I had to read paragraphs and paragraphs of text out loud so it could “pattern match” my voice. I was excited enough so I did the chore and waited for the moment of truth. I was about to experience magic, I was about to use “AI” for the first time.

Spoiler alert, it didn’t last for more than 10 minutes, I had a couple of “wow” moments but then ditched it and never looked at it again.

Why? 

Because it was good as something novel but not good enough to be part of my life.