Per user, per model, per UTC day: metering a free AI chat

LE PEAK gives away access to 21 language models. Ten are free for anyone; eleven need a free account and come with a daily allowance: 10 requests or 10,000 tokens for some models, 40,000 tokens for others, a million input tokens for TypeSafe’s Jev. Every allowance is spent against KTwo’s own provider keys, so the metering has to be right, including when someone opens five tabs and sends from all of them at once.

This post walks through how it works: the rules, the single SQL statement that enforces them, and the details that took longest to get right.

The rules

Per user, per model, per UTC day. Every allowance is its own counter. Using up Claude Haiku has no effect on GPT Luna.
Requests or tokens, whichever runs out first. Some models cap both.
Tokens are counted by the provider. Input plus output for most models; input only for Jev, whose output is free.
At most 20 messages a minute for everyone, signed in or not.
A refusal says exactly why, and when the allowance comes back.

Every request passes the same gates

Before a message reaches a model, the API checks, in this order: that the model exists; that the user is signed in if the model needs it (401 if not); that its provider is configured (503); the 20-a-minute guard (429); that the request is valid; Jev’s context limits (400); and finally the daily quota (429). The cheap checks come first. The quota, which reserves part of the day’s allowance, comes last, so a request that was going to fail anyway never holds any.

Check and reserve in one statement

The naive implementation reads today’s usage, compares it with the limit, sends the request and adds the cost afterwards. Under concurrency that leaks: two requests read the same row, both see room for one more, and both go through.

LE PEAK closes the gap by making the check and the reservation one statement: an upsert whose update applies only if the row still has room, and which returns the row only when it does.

INSERT INTO usage_daily (user_id, model_id, day, requests, reserved_tokens, reserved_at, updated_at)
SELECT :user, :model, :day, 1, :estimate, :now, :now
WHERE (:max_requests IS NULL OR :max_requests >= 1)
  AND (:max_tokens IS NULL OR :estimate <= :max_tokens)
ON CONFLICT (user_id, model_id, day) DO UPDATE SET
  requests        = usage_daily.requests + 1,
  reserved_tokens = live_reserved + excluded.reserved_tokens,
  reserved_at     = excluded.reserved_at,
  updated_at      = excluded.updated_at
WHERE (:max_requests IS NULL OR usage_daily.requests < :max_requests)
  AND (:max_tokens IS NULL
       OR usage_daily.tokens + live_reserved + excluded.reserved_tokens <= :max_tokens)
RETURNING requests, tokens, reserved_tokens;

That is simplified from the production query, which uses typed, numbered parameters; live_reserved is an expression explained below.

The first request of the day is an insert, guarded by the WHERE on the SELECT. Every later one is an update, guarded by the WHERE on the DO UPDATE. If the guard fails, nothing is written and nothing is returned, and the API answers 429. Because it is one statement, the database serializes concurrent requests for the same row: five parallel requests against one remaining slot produce exactly one success.

The same SQL runs on Postgres (Neon, in production), on SQLite locally and in tests, and on libSQL (Turso). Two small adjustments make that possible: numeric parameters are cast explicitly, because Postgres can’t infer the type of a parameter that is only compared with NULL, and CASE stands in for SQLite’s two-argument MAX(), which Postgres doesn’t have.

Reserving room for the reply

Token caps add a twist: a request’s cost isn’t known until its reply has finished streaming. So the reservation holds an estimate.

The prompt is estimated before the request is sent.
For models whose output counts toward the cap, the reservation also holds room for the whole reply: the model’s maximum output length, or whatever is left of the day’s budget if that is less. The reply is then capped to that room with max_tokens.

Without the second part, five replies streaming in parallel could each start within budget and together finish far past it. If a reservation is refused on tokens but some room is left, the API tries again with the reply capped to the room that remains; the cap only ever shrinks, so the loop ends.

When the reply ends, the reservation is settled: the estimate is swapped for the provider’s real token counts. A request that never reached the model is released, which refunds both its request and its tokens.

Requests that die

A serverless function can be stopped mid-request, and then nothing settles its reservation. Left alone, those orphaned tokens would count against the user for the rest of the day.

So reservations carry a timestamp, and one that hasn’t been touched for 15 minutes stops counting. That is the live_reserved expression in the statement above:

CASE WHEN usage_daily.reserved_at < :now_minus_15_minutes THEN 0
     ELSE usage_daily.reserved_tokens END

The function is allowed 300 seconds per request, and the API stops a reply at 285 seconds on purpose, so that even the longest reply ends inside the function’s lifetime and is metered.

A refusal that explains itself

A refused request gets a 429 with a Retry-After header and a body the chat can act on:

{
  "error": "quota_exceeded",
  "reason": "DAILY_TOKEN_LIMIT",
  "modelId": "anthropic/claude-haiku-latest",
  "usedRequests": 7,
  "maxRequests": 10,
  "usedTokens": 9640,
  "maxTokens": 10000,
  "resetsAt": "2026-10-20T00:00:00.000Z",
  "message": "Claude Haiku is out of today's tokens (9,640 / 10,000; this message needs about 1,200). It resets at 00:00 UTC; every other model is still available."
}

The chat uses it to mark that model as exhausted, with a countdown to midnight UTC, while every other model stays selectable.

Rate limits without storing IP addresses

Guests have no account to count against, so the 20-a-minute guard counts per IP address, and the address itself is never stored. The counter’s key is a hash of a server-side secret and the client’s network:

key = SHA-256(secret + ":ip:" + network)

Two details matter. The secret means the stored keys can’t be reversed by simply hashing every IPv4 address, which takes little time on modern hardware. And for IPv6 the network is the address’s /64 prefix, not the full address: providers usually give each connection a whole /64, so keying on the full address would let a guest rotate through a practically endless supply of addresses and never hit the limit.

Signed-in users are counted by their account instead.

What we’d tell anyone building the same thing

Make check-and-reserve one statement. Concurrent clients will find any gap between a read and a write.
Reserve the worst case, settle the truth. Hold room for the reply up front and correct it with the provider’s counts afterwards.
Expect requests to die. Give every reservation a timestamp and an expiry.
Keep counters independent. Per user, per model, per day makes limits easy to explain and hard to game.
Explain refusals. A machine-readable reason and a reset time turn a dead end into a clear next step.

Frequently asked questions

When do LE PEAK’s daily limits reset?

At 00:00 UTC every day, separately for every model.

Does LE PEAK store IP addresses?

No. Rate limits count against a salted SHA-256 hash of the client’s network, which is the full IPv4 address or the /64 prefix of an IPv6 address, never the address itself.

Which databases does the quota SQL run on?

The same statement runs on Postgres (Neon in production), SQLite and libSQL (Turso).

How do you stop parallel requests from overspending a token budget?

Each request reserves its prompt estimate plus room for its whole reply in one atomic statement, and its reply is capped to that room. The reservation is settled with the real token counts afterwards.