Reducing what gets sent addresses one side of an inference bill. The rest is what it takes to hold a model in memory and execute it. That side has been treated as a constant.
Ask what a model costs to run and the answer is a hardware requirement. The requirement is derived from three assumptions that are rarely questioned because they have always been true.
A model is loaded in full and stays loaded. Device memory is sized to the model rather than to the work being done.
Anything not resident has to cross a bus to get there. In practice the path to the device dominates the arithmetic performed once it lands.
A forward pass touches the whole network by default, whether or not the whole network contributes to the token being produced.
The reduction approach applied to execution. Full-precision models run on one consumer desktop with the same answers as the full model. The numbers below are receipted; the method is not published.
The figures above come from captured runs with hashed receipts and a determinism gate, on our reference machine: Ryzen 7 9700X, 32 GB, RTX 4070 Ti Super 16 GB, NVMe. They state what the result is, not how it is achieved.
The how stays in-house. Access to the full capture and the method is available under an evaluation arrangement.
Reduction and context are deployed, billing, and measured against your provider's invoice rather than ours.