On paper, two talks at ai-PULSE 2026 had nothing in common. One came from AMD and was about reselling tokens. The other came from Nokia and was about networking inside the racks. Side by side, in my view, they tell the same story.
Serving a model is easy, making a living from it is another matter
Ramine Roane, CVP Product Application Engineering at AMD, opened with a sentence that sums up the topic: “Serving models is easy. Running a profitable business is hard.”
The starting reasoning is simple. You rent a GPU, run a well-known open weight model on it — Mistral, DeepSeek, MiniMax, GLM, Kimi — and resell the tokens it generates. The question is how many you produce per second, and at what cost. To serve the models, he mentioned the usual frameworks for running GPUs, including vLLM.
One equation, five levers
Revenue is the billable input and output tokens. Cost is compute, power, platform and operations. Margin is what funds growth.
Five levers move the outcome: price, utilisation, throughput, latency and SLOs, and the model mix.
Two of them pull against each other. Interactivity is measured in tokens per second per user: it is what the person at the other end perceives. Throughput is measured in tokens per second per GPU: it is what makes the revenue. Pushing the batch size raises throughput and degrades perceived latency. Shrinking the batch does the opposite. The profitable point lies somewhere in between. Ramine Roane placed it at around 60 tokens per second per user, which is already comfortable for reading, for a stated margin of around 50% once the model provider’s royalties are paid.
Hardware in service of the equation
On AMD’s side, the Instinct MI355X is presented as designed for these economics: 288 GB of HBM3E memory, 8 TB/s of peak bandwidth, the MXFP4 and MXFP6 low-precision formats, and the open ROCm software stack.
The demonstration that stings
The most interesting slide measures profit per chip per hour, on MiniMax M3 served at 150 tokens per second per user — the case where users want speed.
MI355X (ATOM): +$2.52. B200 (TRT-LLM): +$1.63. GB300 NVL72: −$0.61. GB200 NVL72: −$0.95. In brackets, the inference engine used: ATOM, AMD’s open source engine, and TensorRT-LLM, Nvidia’s. The slide concludes: “MI355X: +55% profit vs. best Nvidia. Both NVL72 systems run at a loss.” In other words, according to AMD and at this operating point, two of the most expensive systems on the market run at a loss.
You have to read the assumptions. They are in a small-print inset, which I re-read on my photo: SemiAnalysis InferenceX estimates updated on 30 September 2026, rental on a three-year commitment, 60% utilisation, a 20% model licence fee. It is an AMD slide, the figures serve its platform, and everything shifts if any of these three assumptions moves. But the figure to remember is not the profit: it is the utilisation. At 60%, the GPU does nothing 40% of the time. Forty percent of the bill with no revenue to show for it.
The same problem, seen from the network
Nokia came at it from another angle, on the training side this time. The message: the ability to use the available GPUs depends on the speed of the network. The slide title: “The real constraint: GPUs waiting for the network.”
A training step breaks down into compute, communication, then waiting until the synchronisation barrier. A single congested link on one GPU stretches its communication phase. Since the barrier waits for the slowest, all the others sit idle. The slowest flow sets the pace of the entire job.
The metric that matters is MFU, Model FLOPs Utilization: the share of peak FLOPs that is actually useful. Llama 3 405B at Meta, 43%. PaLM 540B at Google, 46%. MegaScale at ByteDance on 12,000 GPUs, 55%. The stated practical ceiling is around 60 to 70%.
Hence the argument that lands: gaining 10 points of MFU, going from 40 to 50% on a 100,000-GPU cluster, is equivalent to adding 25,000 GPUs of output. Without a single euro of additional capex.
Nokia’s answer: fibre everywhere, and fewer stacked protocols. In substance: from the switch silicon to the rack, from the rack to the cabinet and as close as possible to the endpoint, copper no longer keeps up.
What I take away
Occupancy on the inference side, MFU on the training side. The connection is mine; neither speaker made it on stage. But they are two words for the same thing: the chip is bought, powered, counted as in use, and it waits. The cost of a token is not decided on the card, it is decided by everything around it — the network that feeds it, the runtime that schedules it, the demand that fills it.
For those who consume inference rather than sell it, these figures have another use: they give the order of magnitude of the real cost behind the price per million tokens, and the threshold at which hosting your own model starts to be worth discussing.