Context windows, explained: why your assistant forgets

    You paste a long document into a chat and start asking questions. It goes well. Twenty messages later the assistant contradicts something it said earlier, or behaves as if it never saw your first instruction. Nothing is wrong with it. You have run into the context window.

    The model doesn’t remember. It re-reads.

    A language model doesn’t keep a conversation in its head the way you do. Each time it answers, the app sends it everything it is allowed to see: the instructions, the earlier messages, any documents, and any tool results. That bundle is the context. The window is the largest amount of text the model can take in at once, counted in tokens, which are small pieces of words.

    Figure 1 · What goes into one reply

    InstructionsEarlier messagesDocuments and tool resultsYour new message
    Modelreads all of it, every timeReply
    Everything on the top row is sent again with each reply. If it does not fit in the window, something has to be cut or summarised.

    Why long chats drift

    1. Old material drops out. When a conversation outgrows the window, the app trims or summarises the oldest parts. Your first instruction is usually the first to go.
    2. Detail gets lost in the middle. Even inside the window, a fact buried in the middle of a very long input is easier to miss than one near the start or the end.
    3. Mistakes stay. If an early answer was wrong and you kept going, that wrong answer is still in the context and keeps pulling later replies toward it.

    Habits that help

    • Start a fresh chat for a new task. Carrying an old one along is rarely worth it.
    • Put the instructions that matter at the top, and repeat them when a task runs long.
    • Keep the document and the question close together, and paste only the part that is relevant.
    • When a long chat has reached a good place, ask for a summary of what was decided, then start a new chat with that summary.

    There is also a cost angle. Because the whole context is re-read on every turn, a bloated conversation is slower and, with paid APIs, more expensive. We came at this from the agent side in our note on switching models.

    Want help putting this into practice?

    Book a free consultation and tell us about your team or school.

    Book a free consultation