Skip to content

Budget ​

A cap on model spend or tokens for one scope, such as a project, a user or a client, over a period, which warns or blocks new work when it is reached.

Model calls are priced per tokenToken (LLM): The unit a language model reads and writes: a word, part of a word or a punctuation mark. Context limits, prices and rate limits are all counted in tokens., and agents make many of them: every iteration of the loop sends the whole conversation again. A loop that does not stop, a user who scripts a thousand requests, or a prompt that pulls a huge document into context can turn a normal bill into a large one within hours. OWASP calls this unbounded consumptionUnbounded consumption: OWASP LLM10: an application that lets users or agents run up model calls, tokens or compute without limits, which costs money and can deny service to others. (LLM10) and lists "Denial of Wallet" among its attacks.

The parts of a budget ​

  • Scope: what is counted, such as the whole deployment, a project, a workflow, a user or an API client.
  • Measure and limit: money (estimated from token counts and model prices) or tokens.
  • Period: a day, a week or a month, after which the count starts again.
  • Action: warn sends an alert at a threshold; block refuses new work until the period ends.

Example ​

A project gets a $500 monthly budget that warns at 80% and blocks at 100%. On the 21st, a new agent with a bad prompt loops on a search tool. The warning reaches the team at $400; at $500 new runs are refused instead of the agent running all weekend.

Common confusion ​

A budget is not a rate limit. A rate limitRate limit: A cap on how many requests a caller may make in a period, so one client cannot flood a service or run up its costs. caps how fast calls arrive; a budget caps how much they cost in total. They catch different failures, so they are often used together, along with an iteration cap on each agent run.

In L2M ​

Optional budgets cap USD or tokens per day, week or month (UTC) for the deployment, a project, a workflow, a user or an MCP gateway client. Warn sends an alert; block refuses new runs from any trigger, and gateway calls, until the period ends.

Further reading ​

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings