Interactive tool

Billions of parameters. But what is a parameter?

Every AI headline counts parameters like horsepower — but few explain the number. In ten minutes you'll train a real 2-parameter model with your own hands, see why feeding a model more data doesn't give it more parameters, and learn how today's small models beat yesterday's giants. By the end, "70 billion parameters" will be a sentence you can actually picture.

01 — Turn the knobs

A parameter is a knob. Training turns it.

Below is the smallest possible "AI model": it draws a line through data points, and it has exactly two parameters — two adjustable numbers. Drag the sliders and watch the line move. Your goal: make the error bar green. Then press Train automatically and watch the computer do the same thing — nudge each knob a tiny amount in whichever direction reduces the error, over and over.

Prediction error—
Move a slider to start.
That's the whole secret. Training a large language model is exactly what the button just did — measure the error, nudge every knob a tiny amount to reduce it, repeat — except with billions of knobs instead of two, adjusted trillions of times. A "70B model" simply has 70 billion of these adjustable numbers. Nobody sets them by hand; the training process finds their values automatically.
02 — Data vs. size

Does more data give you more parameters? No — and that's the point.

The parameter count is a design decision made before training starts — engineers choose the model's size the way an architect chooses a building's footprint. Data never adds parameters; it fills the ones you chose. Think of the model as a sponge: parameters set the size of the sponge, data is the water. Try mismatching them below — the interesting question isn't "how big?" but "how well fed?"

Model size — chosen up front
Training data — poured in during training: 1T tokens
Ratio: — tokens per parameter
The sponge —
The 2022 "Chinchilla" result made this famous: DeepMind showed that most giant models of the era were huge sponges kept half-dry — for the same compute budget, a model 4× smaller trained on 4× more data (roughly 20 tokens per parameter) gave better answers. Since then the industry has fed models far past that ratio — modern small models drink 100–1,000+ tokens per parameter — which is a big part of why they keep surprising people.
03 — Small but mighty

Can fewer parameters win? Place your bets.

Each round shows two real (or realistic) models. Pick the one that gave better answers. The pattern you'll discover is the most useful thing to know when a vendor leads with a parameter count.

● Interactive challenge
Score: 0 / 0

The five levers: more power per parameter

Parameter count is only one lever, and it's the most expensive one. These five are how labs squeeze more capability out of fewer knobs.

04 — Metaphor toolkit

Four metaphors to keep.

Each metaphor carries a different part of the idea. Steal whichever one lands with your audience — they're all honest about how the technology works.

05 — Quiz

Lock it in.

Five quick questions covering the knobs, the sponge, and the levers.

Total score: 0 / 5
06 — Why it matters

Key takeaways for the boardroom.

You'll never tune a parameter yourself — but you will absolutely sit in meetings where parameter counts get used as sales arguments.

07 — Go deeper

Related tools.

Parameters are one piece of the picture — these modules cover the neighbors.

AI Weights & Capabilities Quiz

Feel what it's like to be a model running out of training data — and why weights and biases exist in the first place.

The Training Pipeline

Where the knob-turning happens: pretraining, fine-tuning, RLHF, and deployment, compared stage by stage.

Next-Token Prediction

What all those tuned parameters are ultimately for: predicting the next word, one token at a time.

Compute & Hardware

Why every extra billion parameters costs real money — the chips and datacenters that hold and update the knobs.

The Building Blocks of AI

Quantization and architecture — the deployment-side tricks for shrinking a model once training is done.

Parameter counts will come up in your next vendor meeting.

"How big is the model?" is the question everyone asks. "What was it trained on, and what does it cost to run per query?" are the ones that decide whether an AI initiative pays off. We help leadership teams ask the second kind.

Talk to us about your AI roadmap