A share of a public company has a price. Look up a stock and you'll see it immediately: $42, $180, $900. Tomorrow it'll probably be something else.
That number looks like the market, but it's really just the most visible thing the market produces. Underneath it are a bunch of people expressing what they want: how many shares they want to buy, how much they'll pay, how many someone else is willing to sell, and what it'll take to get them to sell. The price comes out of all of those intentions bumping into each other.
Let's start with a simple example.
At $1, Bob wants the apple, but not badly enough to meet Alice's terms. Alice wants to sell it, but not for what Bob is offering. There's demand and there's supply, but there's no trade.
Move Bob to $2 and their intentions finally overlap. That's all the market needs. Add a crowd and that same simple interaction starts to produce some structure.
Financial markets call this kind of running list of offers an order book. Once the people are sorted by their terms, they start to look like one. Buyers offering the most sit closest to sellers asking the least. When the two sides touch, something trades.
The gap between the highest bid and the lowest ask is called the spread. Watch the little market for a moment and you'll see people tend to gather at favorable existing prices: sellers try to stay high and buyers try to stay low. A trade gets their attention, though. Some move toward the gap, the available offers get consumed, and eventually a new spread starts to form. Quantities matter too, as does how far away the other offers are. We could keep digging into the mechanics, but the important part for this article is simpler: people state their terms, and the market looks for overlap.
Now make it compute
Suppose I want a coding agent to examine a large repository. I figure the job will take about 500 million tokens. I have a budget, I need the result back within a reasonable amount of time, and I need a model that can actually do the work.
That last part needs some way to become a term instead of a vague hope. Maybe I care about a model's score on a particular repository-repair benchmark. I can put a minimum acceptable score on the request, just like I can put a maximum price on it. The benchmark isn't declaring one model universally better than another. It's a capability bar for this job, chosen by the buyer.
Inference usually starts from the other direction. A provider chooses a model, runs the hardware, and publishes a price. I can accept that price or look for another API. That's simple, and it works pretty well.
But what if I could put the job into a market instead? I describe the work and my terms. Providers describe the compute they have available. Then we find out which offers fit.
This is the same idea as the apple market, just with a few more details on the order. Start with the budget. Some offers fit and some don't. Tighten the deadline and cheap overnight capacity may stop being useful. Raise the CodeRepair score and models below that capability bar drop out. Allow another model that still clears the bar and more supply can compete for the same job.
The provider cards aren't rankings. Each one is an offer to do this particular work: a model, its score on the benchmark this request cares about, a completion time, and a total price for the whole 500-million-token job. The labels explain why an offer does or doesn't fit.
That feels like an important difference from a price list. The thing being matched isn't really a generic million tokens. It's a job with terms.
Price still matters, obviously. But a $710 job that finishes overnight is different from a $1,525 job that finishes in eight minutes. Interactive chat, a coding agent, an overnight evaluation, and an incident-response tool can all look at the same available compute and make different choices.
The same supply can serve different jobs
Here's another view of those same provider offers. Each box is one offer. Its position shows price and completion time, while its height shows how much capacity the provider has available.
Pick a workload and the colored boxes are the offers that fit it. The pale boxes are still real supply; they're just a poor match for that job. Switch from a coding agent to an overnight batch and a different part of the market lights up.
Capability doesn't become another spatial axis here. The color simply tells us whether an offer clears the benchmark bar for the selected workload, along with its other terms. There are already more dimensions than a chart can comfortably draw, which is part of the point.
So there may not be one best inference provider. A large cloud may be the right choice when eight minutes matters. A regional datacenter may win work that can wait a few hours. Idle GPUs may be useful overnight. A specialized accelerator may be unusually good at one family of models.
The TensorWorks button makes this a bit more concrete. Suppose better batching and kernels cut its serving cost by 15 percent. Its offer moves toward the cheaper end of the market, and it can now win the coding-agent job it missed before.
The buyer doesn't have to know which kernels changed. They just see another acceptable offer at a better price. The technical improvement has somewhere to show up.
That's the competition I'm interested in. Providers don't all need to look alike or win for the same reason. Better scheduling, caching, quantization, hardware, memory management, or cheaper power can change where one of them is competitive.
The requester doesn't have to be the payer
Money is a big part of a market, but it doesn't have to be the only relationship between the people involved.
Say an open-source maintainer wants a coding agent to investigate a difficult issue. They can still require a model that clears the project's capability bar. The job has a real compute cost, but the project has no budget for it. Commercial providers aren't going to treat $0 as a competitive offer, which is fair enough.
Donated capacity can still make the match happen. The maintainer requested the work, a provider supplied the compute, and somebody else covered its value. Those can be three different participants.
That makes this look less like a store selling tokens and more like a system for allocating computation. Payment is one way to make a match happen. Donation or subsidy can be another.
A real version would need quite a bit more than these cards, of course. It would have to deliver jobs and return results, protect providers from malicious workloads, protect user data, verify eligibility for donated resources, settle payments, and deal with trust. I'm skipping over all of that here.
Why open models matter
There is one pretty large prerequisite for this idea: the model has to be available from more than one provider.
That's where increasingly capable open-weight models get interesting. Open weights means somebody other than the group that released the model can run it. The research team doesn't also have to become the permanent API operator for every request that model will ever handle.
That leaves room for different groups to get good at different parts of the problem:
- researchers can work on better models;
- inference-engine developers can serve them more efficiently;
- hardware companies can build machines optimized for them;
- datacenter operators can compete on infrastructure and power;
- clouds can compete on scale, reliability, security, and availability;
- smaller providers can find niches or sell idle capacity;
- communities can subsidize or donate compute.
The market becomes the place where all of those improvements meet actual jobs.
This isn't an argument that hyperscalers should disappear. They may remain the biggest suppliers and the best choice for plenty of workloads. The change is that owning the API endpoint wouldn't automatically make them the only price setter.
A new hardware company wouldn't have to reproduce an entire cloud business before its advantage mattered. If it could run an open model more efficiently, it could make an offer. The same goes for a specialized inference provider, a regional datacenter, or somebody with spare GPUs at night.
Who sets the price?
So my question isn't really, "What should a million tokens cost?" It's, "Who gets to decide what a million tokens cost?"
Today the answer is usually the company that owns the API endpoint. With open models, it could be the workloads asking for computation and the many suppliers competing to provide it.
That's the part of open models I find most interesting. They make inference contestable. If somebody can produce an acceptable result more efficiently, they have a way to compete, and a market gives that advantage somewhere to show up.
Perhaps inference doesn't need another price list. Perhaps it needs a market.