General Compute secures $400M loan for AI inference chips

General Compute, an AI inference cloud startup, has secured a $400 million loan from Upper90, a tech investment firm. This deal marks a significant milestone as it may be the first to use inference-specific chips as collateral. These chips are designed to run already-trained AI models quickly and efficiently, as opposed to the more expensive chips used for building the models.

The financing reflects the market's response to concerns over the high cost of AI tools and tokens, shifting towards infrastructure that runs open-source models more affordably than the latest large language models from frontier labs. General Compute, founded by CEO Finn Puklowski, previously raised a $15 million seed round in May to develop an inference neocloud around silicon from SambaNova, an Intel-backed chipmaker. The company's SN50 chips are optimized for inference, offering power efficiency and eliminating the need for expensive water-cooling systems.

This allows for quicker deployment across a wider range of data centers. General Compute claims the new chips will provide 16 times faster inference than GPU-based clouds. Upper90 co-founder and CEO Billy Libby, a former Goldman Sachs quantitative trader, had a strategy for this: In 2021, his firm financed GPU purchases by Crusoe, the energy-focused data center startup, which he believes was the first loan against the value of advanced chips.

Traditional lenders avoided such deals due to risks and uncertainties around GPU depreciation. However, as CoreWeave made chip-backed loans a business model and then the basis of a successful IPO, this type of financing has become more common. Libby noted, 'When we financed Nvidia GPUs as the first group to do that, the market was inefficient.

We could really put together something as an early participant, and kind of get compensated for the risk.' Now, with GPUs comparatively well understood and perhaps over-bought, Upper90 is turning to companies like General Compute to capitalize on the next wave of the AI boom. Libby added, 'We think open source models are going to be important, and we went and looked for a player last year that was in inference. Everyone doesn’t need a supercomputer, but they do need inference and AI.' General Compute’s ability to access chips outside of Nvidia’s ecosystem matters because it offers an alternative.

TensorWave, another AI infrastructure company, is making a similar bet on a partnership with AMD. As more alternatives to Nvidia emerge, compute providers that aren’t locked into Nvidia deals may have an advantage in providing cost-efficient inference. Puklowski said, 'There are a bunch of chips that are starting to scale that have amazing [total cost of ownership], or that can operate much faster than Nvidia, but there’s not too many buyers for them.