Instrument

Cache economics

A prompt cache is a bet. You pay a premium to write the prefix down, and a fraction of the usual rate every time you read it back. If you read it often enough the bet pays; below that line you have simply bought a more expensive prompt. Where the line sits depends on the provider, and the four here do not price it the same way.

Pick a provider and move the hit rate. The break-even is the number of reads that repays the write premium.

Caching decides what a window costs to refill. The window decides how much of what you refilled the model actually reads — the two multiply, and only one of them appears on the invoice.