Embedded Files
Chat Tokens
Chat Tokens
Tokens are the fundamental units of data the AI uses to process text. You can think of a token as a word, a part of a word, or punctuation. (Rule of thumb: 100 tokens ≈ 75 words).
Whether you are engaging in a standard direct chat or utilising chats integrated with tools, monitoring your tokens ensures efficiency.
To help you manage performance and track costs, the system displays three real-time metrics: Context Window, Billable Input and Billable Output.
Step 1: To view your token usage, click on Chat Tokens from within any ongoing or completed conversation.
Chat's Context Window:
What it is: The active memory of the current conversation, measured in tokens.
What it tracks: The total active tokens the AI can "see" at once. This includes prompts, AI replies, chat history, background instructions, and any uploaded or retrieved content.
Note: Reasoning and cached tokens are currently excluded from this count.
The Limit: Every model has a maximum token capacity. For chats using tools, this is further constrained by tool-level Max Output Tokens configuration (capping tokens per response).
Once these limits are reached, the application may drop older messages, truncate content, or the request may fail entirely.
Billable Input:
What it is: The total cumulative tokens sent to the AI during the session.
How it works: Each time a new message is sent, the AI does not just read that line—it re-reads the new prompt, past chat history, and system instructions to maintain context. All of this counts toward the input total.
Billable Output:
What it is: The total cumulative tokens generated by the AI across all responses in the session.
How it works: These tokens are billed the moment they are created. This count is separate from the context window, meaning earlier responses are still billed even if the AI eventually "forgets" them.
Page updated
Report abuse