What the product does
When a model’s working memory overflows the accelerator, or a session lands on a different server, the context is thrown away and the next request rebuilds it from scratch. The vendor calls the waste the recompute tax and says it consumes more than half of all GPU cycles in a typical production deployment. Its answer is a store that turns petabytes of standard flash into a shared tier, reachable from any node in the cluster over a direct-memory fabric, so that evicted context is retrieved rather than recomputed.
The vendor has also named the tier: between local flash that strands context on one server and general shared storage that has the capacity but answers in milliseconds. Naming a tier one is first to ship into is an old and effective move, and the accusation in the launch — that every other storage vendor is extending durable-data platforms into a job that needs a different design — is the sort of thing a category creator says. It may also be true.
Read the benchmark for what it is
The headline numbers: time to first token at a long context of 53 seconds without the product and under a second with it; at a longer context, the system without it did not slow down but stopped, out of memory.
Both figures are the vendor’s, and the baseline is the same cluster with the product removed — not a competitor’s product installed. Fifty-three seconds is what an accelerator does when there is nowhere to put evicted context, which is precisely the condition the product exists to eliminate. The comparison measures the size of the problem. It does not measure a lead over anyone else selling a fix.
The money claim is built the same way: a worked example with a chosen cluster size, chosen context lengths, a cache-miss rate “above 50 percent” and an hourly accelerator rate spanning a factor of two, arriving at a recompute tax of “up to” two million dollars a year. It is arithmetic, and it is arithmetic on four inputs the vendor selected. Run it from the bottom of each range and the answer is materially smaller. And the striking figure — a petabyte of context memory for less than the cost of one GPU — prices the drives alone, which are the cheapest part of a tier that also needs fabric, enclosures, processors and software.
None of this makes the product wrong. It makes it unmeasured on anything but its own terms, which is the normal condition of a launch and the reason we keep repeating the six checks: unit, baseline, date, exclusions, who ran it, shipping status.
Why it belongs in the data architecture anyway
Set the marketing aside and look at the shape of the thing. Enterprise inference now has, or will shortly have, a tier that sits between the governed data platform and the model: vector indexes for retrieval, caches of retrieved content, and now persisted model state shared across a cluster. Vendors of storage, databases and inference platforms are all converging on it, and whichever product fills it, the architectural fact will remain.
That tier holds derived copies of enterprise data. Context built from a customer’s contract is a transformation of the contract. A store that lets any node in the cluster retrieve any session’s context is, from a governance standpoint, a shared repository of everything the models have recently read — classified nowhere, retained by a policy nobody wrote, reachable by whatever holds a fabric address.
So the tier belongs on the data platform’s map before the first purchase order. Its contents inherit the classification of their sources. Its retention should be explicit and short. Access to it should be scoped the way access to the warehouse is scoped, with the question “can one tenant’s session reach another’s context?” answered in writing.
Sizing it honestly
The right way to buy into this tier is the same as for any capacity decision in a memory shortage: from measurement. Your context lengths, your session reuse pattern, your actual miss rate on a representative workload — run for a few weeks on a pilot before the vendor’s worked example becomes your business case. The recompute tax may well be real in your estate. It will not be the vendor’s number, in either direction.
And keep the model swappable, the connector governed, and the tier treated as infrastructure that can be replaced — because the category was named this month, and the vendor who named it will not be the last to ship into it.
