A marketer's guide to context windows
Why your AI tool "forgets", and how to give it what it needs.
Your AI tool does not forget anything, because it never remembered anything. Every answer it gives is built from scratch out of whatever is sitting in front of it at that moment, and your job is deciding what that is.
This matters because almost every complaint I hear about AI output is really a complaint about context. The tool wrote off-brand copy. It contradicted what you told it ten minutes ago. It invented a product feature. Those look like intelligence problems and they are almost always supply problems.
The window is a desk, not a filing cabinet
A context window is the working set for one response. Everything the model can see right now, all at once: your instruction, the transcript so far, whatever you pasted, whatever the tool quietly attached.
It is measured in tokens. In English, a token averages about four characters, so a thousand tokens is roughly 750 words. Useful shorthand when you are trying to work out whether your brand guidelines will fit.
Here is the part that trips people up. The model does not read your latest message. It reads the entire window, from the top, every single time. Nothing persists between turns except by being put back in front of it again.
So when a chat tool “remembers” what you said twenty minutes ago, it is because the whole conversation is being resent with every message. When it starts to lose the thread, it is because the conversation outgrew the window and the earliest part fell out or got compressed.
Nothing broke. There was never any memory.
Bigger is not the fix
Windows have grown fast. Around two thousand tokens in 2020, then eight, then a hundred, then two hundred thousand, and by 2024 a million was on the table. Any advice anchored to a specific number dates within a year, which is why I am not going to give you one.
More to the point, a bigger desk does not help if you do not know which papers to put on it.
Two things degrade as you fill a window. Cost and latency scale with what you put in, because the model reads all of it, every turn, and you pay for all of it every turn. And relevance degrades. Research on long contexts has found models attend better to material at the beginning and the end than to things buried in the middle - the “lost in the middle” effect. Put your most important instruction halfway down a forty-page paste and it competes with everything around it.
[illustrative] An edtech content team pasting a 40-page brand guideline into every single prompt. Paying for all 40 pages on every request, waiting longer for each one, and getting output measurably worse than a one-page distillation produced. The long version had the tone rules on page 31.
What to put in front of it
The useful question is never “how do I get it to remember?” It is “what does this model need to see to do this one task?”
For marketing work that usually means four things. The instruction, stated plainly. The real examples, because models follow examples far better than they follow adjectives. The actual data, not a description of the data. And the constraints that matter, near the end where they hold their weight.
What it does not need is a paragraph telling it that it is a world-class marketing strategist with twenty years of experience. That is theatre. (It does not have twenty years of experience and telling it so does not give it any.)
And if you have ever wondered what retrieval or RAG actually is, underneath the acronym: it is automated context selection. Something goes and finds the relevant few pages out of your thousand and puts those in the window instead of all of it. Same principle, done by a machine.
Where I would push back on myself
Bigger windows genuinely do change what is possible, and I do not want to undersell that. A full quarter of sales call transcripts in one pass, or an entire product catalogue, is not just a convenience. It is a different class of task, and it was not available three years ago.
The other honest objection is that curation costs time. “Ruthlessly distil your brand guidelines” is easy advice to give and it is half a day of work. Sometimes pasting everything in and paying for the tokens is the right call, because your afternoon is worth more than the API bill. Judge it on the size of the job.
Where to start
Take the longest thing you paste into AI tools regularly. Write the one-page version. Run both against the same five tasks and compare the output side by side.
That exercise teaches more about context than any explainer will, including this one.
Editing notes
- Verify: the window-size progression (roughly 2,000 tokens in 2020 to 1,000,000 by 2024). Directionally solid, worth a check on specifics before publishing since this is the piece’s one numeric claim.
- Verify: the “lost in the middle” finding. Real published research, Liu et al., 2023 - check the citation if you want to name it explicitly rather than describing it.
- Verify: four-characters-per-token is a widely used approximation for English. Safe as shorthand, but say “roughly”, which it currently does.
[illustrative]on the edtech brand guideline example. Swap for a real one if you have it.- Deliberately omitted: any claim about AI adoption rates among marketers. Everything circulating on that is vendor-sourced.