Input tokens are what you send
Input tokens include the system prompt, developer instructions, user message, conversation history, retrieved context, and any structured data you place in the request.
Prompt length matters because repeated instructions and large context blocks are billed every time unless the provider supports discounted caching for reused input.
Output tokens are what the model generates
Output tokens are the model's answer. Long explanations, code blocks, summaries, and multi-step reasoning traces can increase output volume quickly.
Output tokens usually cost more than input tokens because generating new text requires more work than reading prompt context.
A simple cost example
Suppose a model costs $1 per 1M input tokens and $4 per 1M output tokens. If one request uses 2,000 input tokens and 500 output tokens, the request costs $0.002 for input plus $0.002 for output, or $0.004 total.
At 100,000 similar requests per month, that workload would be about $400 before any caching, batching, retries, or provider-specific fees.
How to control both sides of the request
- Shorten repeated instructions and remove context the model does not need.
- Set a reasonable maximum output length for each product workflow.
- Use cheaper models for short classification, routing, or extraction tasks.
- Compare provider pages such as OpenAI, Anthropic, and Gemini before choosing a default model.