Whether it writes code, sees photos, or holds a conversation, every model is doing the same three things underneath: running a brain made of learned values, turning whatever you give it into numbers, and holding it all in a working memory long enough to answer. Explore each layer below.
The model is its parameters, how precisely those parameters are stored, and how its internal structure is organized. Play with each dial below.
Text, images, video, and audio are all converted into numeric data before a model can touch them — and each modality has its own shortcut for doing that efficiently.
The easiest modality to grasp: each word (or word-piece) is looked up in a vocabulary and replaced with an ID number. The model never sees the words — only the numbers.
An image is a grid of pixels, and each pixel is just three numbers — how much red, green, and blue it has. Click a pixel to see its numbers.
Re-processing every pixel of every frame from scratch would be wasteful — most of the frame doesn't change. Instead, the model mostly tracks what moved between frames.
A microphone measures air pressure thousands of times per second — that's the waveform. Group those measurements by frequency over time and you get a spectrogram, which is usually what the model actually learns from.
Once everything is numbers, the model needs to hold it all at once to make a decision — that's the context window. Add blocks until it's full, then keep adding and watch the oldest one fall out of memory.
Put the three layers together and you get the actual sequence every model runs: the world gets turned into numbers, those numbers get held in working memory, and the brain runs over whatever is currently in that memory to produce an output.
"Which model should we use" is really three smaller questions: is the brain big enough for the task, can it handle the modality you're feeding it, and does its context window actually hold what you need it to remember. That's the kind of model-selection conversation we have with teams before they commit budget.
Talk to us about your AI roadmap