# Price a request, not a model name

Understand total cost with measured token usage and a realistic request mix.

Source: https://bitsandbias.com/course/17-costs/

---

## Learning outcomes

- Compute input/output token cost from explicit prices
- Distinguish API spend from total service cost
- Create a cost-quality comparison that accounts for retries and failures




## Cost is a workload property

A model provider normally bills according to usage, such as input and output tokens, possibly with different prices for cached input, batch jobs, or other modalities. Costs change. **Never hard-code live vendor prices into your course business case.** Instead treat prices as parameters with an observation date and source.

For a basic uncached text request:

`cost_USD = (input_tokens * input_USD_per_M + output_tokens * output_USD_per_M) / 1_000_000`

where `P_in` and `P_out` are hypothetical USD per million tokens, and `T_in`,`T_out` are measured token counts. For example, at $1/M input and $4/M output, a request with 2,000 input and 500 output tokens costs $0.004 in model usage before other service costs. That is **four-tenths of one US cent**, not four cents.

## What else costs money?

- Embedding/index creation and storage.
- Retrieval and vector/database hosting.
- Network, application compute, caches and telemetry.
- Retries, timed-out requests, failed validation, and duplicate work.
- Human review and support, if relevant to the cost model.

A cost spreadsheet that contains only the LLM's published input price is not a full business cost model.

## Model routing is a measured decision

Suppose a small model is faster and cheaper but fails on 12% of a reviewed hard-query slice; a larger model passes more often but costs three times as much. You can route only the hard class to the larger model if the classifier is reliable and the added complexity is justified. But routing itself can fail, so evaluate **end-to-end** quality and total cost, not just per-model benchmark scores.

## The denominator matters

Report:

- spend per **request**;
- spend per **successful, supported answer**;
- spend per **1000 user sessions**;
- cost per important query slice.

A cheap model that requires three attempts and fabricates answers may be more expensive per useful outcome than a more expensive reliable model.

## Your experiment

Create a table for three hypothetical models with different input/output prices and quality scores. Evaluate them on the same workload and include retries. Document the limitations of your fake costs; the interactive calculator uses example numbers only, not current vendor quotations.



## Hands-on lab

Implement `estimate_cost_usd(input_tokens, output_tokens, in_per_m, out_per_m)` in `labs/docdesk/student.py`. Use the cost workbench below to verify units. Add a negative-value validation test and a 1,000-request workload estimate.

[Download the lab kit](/course/downloads/docdesk-labs.zip)
