>>106190522It's how much chat history the model can think about while writing your response.
8k tokens is approximately 8,000 words.
If your chat history (meaning your messages + the AI messages) exceed ~8,000 words, the model will start forgetting things you said earlier, stops following older instructions, and generally just hallucinates the chat history.