The News
AMD agreed this month to buy Taalas, whose chips burn a model's weights permanently into transistors. Two weeks earlier, Etched raised $300 million at a $10.3 billion valuation for a processor that runs transformers and nothing else. Qualcomm closed its roughly $4 billion purchase of Modular, whose software exists to loosen CUDA's grip. Three companies, one wager.
The View
The industry has decided that AI has stopped moving, and it is pouring concrete accordingly.
Here is the trade in plain terms. A general-purpose GPU is slower and thirstier than a chip built for one job, and in exchange it will run whatever you throw at it next year. A specialized chip is dramatically faster at the thing it was built for and worthless at everything else. Choosing between them is a forecast about how stable your workload is, dressed up as an engineering decision.
Chip history has run this experiment several times and the results are unambiguous in both directions. Fixed-function 3D accelerators were faster than programmable graphics hardware in the late nineties, right up until developers wanted effects the silicon could not express, and programmable shaders buried them. Bitcoin mining went the other way: SHA-256 never changes, so ASICs annihilated general hardware and never gave the ground back. Fixed function wins when the target holds still. It loses, expensively, when the target moves.
So which is AI? The honest answer is that the transformer has been remarkably durable, roughly nine years now, and inference has become the majority of compute spend, which is exactly the condition that rewards specialization. Taalas is not betting on the transformer, though. It is betting on specific weights, which is a much shorter-lived thing. Models are replaced every few months. A chip with Llama 3.1 cast into it is a beautiful artifact of a particular Tuesday.
What I keep coming back to is that the memory wall is the real driver, not architectural conviction. HBM is scarce, memory pricing just pushed Amazon's capex up by twenty billion dollars and shoved Samsung's chip profit up 250-fold, and every one of these designs is an attempt to stop paying that toll. When the industry starts hardwiring its assumptions into silicon because the alternative has become unaffordable, that is a supply constraint driving architecture, and supply constraints ease. What is left when memory prices normalize is a lot of very fast chips that can only think one thought.
None of this makes the bets wrong. It makes them dated the moment they are cast, which is fine if the payback period is short enough. Just notice that the people building the most inflexible hardware in a generation are doing it during the least predictable stretch the field has had.
Room for Disagreement
The strong counter is that I am pattern-matching to graphics when I should be pattern-matching to Bitcoin. The transformer has survived nine years, every frontier lab still ships one, and the workload underneath inference is converging rather than diverging. If a chip pays for itself in eighteen months and the model lives twenty-four, inflexibility costs nothing and the efficiency is free money. At ten times the tokens per watt, you do not need the bet to last forever. You need it to last one product cycle, and that is a much easier thing to be right about.
Notable
- Taalas claims about 17,000 tokens per second on Llama 3.1 8B at roughly a tenth of an H200's power draw.
- Etched's Sohu chip claims a large inference speedup over Nvidia hardware, with over $1 billion in signed contracts before shipping.
- SK hynix, which makes the memory these designs are trying to avoid reading, is an investor in Etched.
Bottom Line
Specialized silicon is a forecast about stability wearing an engineering costume. Before buying into it, ask how long the model you are casting in stone is expected to live, and whether the memory prices that made the math work will still be there when the chips arrive.
— Hank