Ask HN: What is the black swan event that could bring down GPU costs?

Training is 10-100x more compute than inference. The current rush is all training. When it tips over to mostly inference, a lot of that hardware is just going to sit there.

It's already happening. Cost of capital is increasing, Nvidia and SK Hynix going down. This will follow the patterns of a "real demand but overbuilt" bubble like railway back in the day.

I personally plant to buy GPUs in about 18 months.

Carrington-class (G5) or greater geomagnetic storm melts global power grid and electricity production for years. During this time, GPU prices drop to $0 per unit.

A true black swan event (Also, a black-out event)

Are you actually interested in black swan events or opinions on known or unvalued risks?

There's loads of weird black swan events that might do this... An AI might find a way to dramatically improve matmul algorithms, governments might unilaterally ban frontier AI training, China might make a surprise breakthrough in their lithography tech and start flooding the market with cheap GPUs.

I guess the question here is whether there's any alpha in speculating about these kinda of unlikely events? The only predictions that really matter are ones that are under or over priced, otherwise there's no actionable information.

My view on that would be that when you zoom out AI progress tends to occur in a sort of continuous series of s-curves. Where a breakthrough in an algorithm causes a rapid acceleration in AI progress, before hitting some bottleneck and slowing again until the next algorithmic breakthrough comes.

I think this idea has always been underpriced. It's why predictions about AI progress have always been wrong because people never assume the next breakthrough is coming, just that incremental improvements are coming.

It's hard to say how the next breakthrough might change AI, but what I think is true to say is that LLMs are uniquely inefficient. I mean we have cars with onboard chips that can literally drive themselves around busy urban environments, but getting a model to output some text requires a relatively enormous amount of compute and memory.

My guess is that GPU demand will not drop sharply in the coming years, but if you're asking me whats likely to be undervalued that might cause GPU demand to drop, it would be this. An algorithmic breakthrough is coming in the next few years and all bets are off about how that changes the industry when that happens.

A collapse of demand. That swan looks pretty white to me.

China