cost-accounting.md

docs/cost-accounting.md

1. # How Cortex counts cost
3. Cortex exists to make the price of each reply visible, so the arithmetic is
4. simple and written down here.
6. ## Prices
8. DeepInfra publishes, for every model, a price per million tokens:
9. `input_tokens`, `output_tokens` and, for models with prompt caching,
10. `cache_read_tokens`. They come with the public model list
11. (`GET https://api.deepinfra.com/v1/openai/models`, no key needed). Cortex
12. keeps only models tagged `chat`, refreshes the list at startup when it is more
13. than a day old, and stores it locally.
15. When the list cannot be fetched (offline, or DeepInfra changes the format),
16. Cortex falls back to `crates/cortex-core/data/pricing-snapshot.json`, a dated
17. copy made by `node scripts/snapshot-pricing.mjs`. The model picker then says
18. "Prices as of DATE (offline copy)".
20. A price above $1,000 per million, or below zero, is treated as a parsing
21. mistake and the model is dropped rather than shown with a wrong price.
23. Local servers (Ollama, LM Studio, llama.cpp) cost nothing.
25. ## One reply
27. ```
28. cost = (prompt - cached) x input price
29. + cached x cache price (the input price if the model has none)
30. + completion x output price
31. ```
33. with prices per million tokens. Money is stored as whole **nano-dollars**
34. (10^-9 USD) in a 64-bit integer, so a month of sums never drifts the way
35. floating point does. Displayed values round to four decimal places below a
36. dollar and two above; anything under $0.0001 shows as "<$0.0001".
38. Each reply stores the prices it was billed at. A later price change never
39. rewrites what an old reply cost.
41. ## Where the token counts come from
43. | `usage_source` | When | Shown as |
44. |---|---|---|
45. | `reported` | The provider sent a usage block (Cortex asks for one with `stream_options.include_usage`). | the numbers |
46. | `estimated` | No usage block: most local servers, a stream you stopped, an error part way. | the numbers with an **est.** badge |
47. | `none` | Refused before a single token (bad key, no credit, rate limit). | nothing; costs $0 |
49. The estimate is about four characters per token for Latin text and one per
50. CJK character, plus four tokens per message for the chat template and 765 per
51. image. It is deliberately rough and always labeled.
53. If DeepInfra reports its own cost for a reply (`usage.estimated_cost`), Cortex
54. stores it beside its own figure for comparison, but shows its own arithmetic.
56. ## Month to date and the budget
58. "This month" is the sum since midnight on the first of the current month in
59. your local time zone. The monthly budget warns at 80% and again at 100%, in
60. the chat, the sidebar and the menu bar. It never blocks a send by itself; an
61. optional setting asks for confirmation once the budget is spent.
63. ## Before you send
65. The composer shows roughly what the prompt will cost: the conversation so far
66. plus what you have typed and attached, at the model's input price. The reply's
67. length is unknown until it arrives, so this is the floor, not the total.