Platform / Runtime

The other half of the cost.

Reducing what gets sent addresses one side of an inference bill. The rest is what it takes to hold a model in memory and execute it. That side has been treated as a constant.

Why it hasn't moved

Three constraints, all treated as physics.

Ask what a model costs to run and the answer is a hardware requirement. The requirement is derived from three assumptions that are rarely questioned because they have always been true.

Residency

All of it, all the time

A model is loaded in full and stays loaded. Device memory is sized to the model rather than to the work being done.

Transfer

Weights have to arrive

Anything not resident has to cross a bus to get there. In practice the path to the device dominates the arithmetic performed once it lands.

Execution

Every parameter, every token

A forward pass touches the whole network by default, whether or not the whole network contributes to the token being produced.

Work in progress

What we are working on.

Each of the three constraints above has a corresponding line of work. Status is stated plainly: development means it runs, research means it does not yet.

01
Device memory residency
What has to stay on the card, and for how long.
IN DEVELOPMENT
02
Sequential layer streaming
A storage format built for ordered layer delivery rather than random access.
IN DEVELOPMENT
03
IO path
Removing the host-side bottleneck between storage and the device.
IN DEVELOPMENT
04
Weight representation
How much of a weight tensor is load-bearing, and what the rest costs to keep.
RESEARCH
05
Selective execution
Identifying what actually contributes to a given token before computing it.
RESEARCH
On numbers

We are not publishing figures on this work.

Everything on the reduction side of the platform is measured against a provider's own invoice, which is why we are willing to state it. Runtime results are measured against our own hardware, and we have not finished the work.

Figures will appear here when they are reproducible by someone who is not us. Until then, the status column above is the whole claim.

Elsewhere on the platform

The half that already works.

Reduction and context are deployed, billing, and measured against your provider's invoice rather than ours.