The Download: the next big thing in LLMs and how AI academic research is shifting

This is today’s edition of The Download , our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

Nine years after Google researchers introduced the transformer, this family of neural networks has become the engine inside every major large language model. But transformers are starting to show their age.

As LLMs get bigger and better, transformers have become a bottleneck. Their dense attention mechanism becomes increasingly expensive as the amount of text grows, and they’re not great at keeping track of a lot of information at once.

Here are four new ideas for how to solve the transformer problem —innovations that could change LLMs for good, making them faster, far more efficient, and (maybe) even smarter.

This story is from MIT Technology Review’s What’s Next series, which looks across industries, trends, and technologies to give you a first look at the future. You can read the rest of them here .

Last week, I headed to a hotel in Mountain View, California, to join some of the most accomplished, and some of the most promising, AI researchers in the world. I was hosting roundtable interviews and speaking at a media training for a convening of the Schmidt Sciences AI2050 program, an initiative funded by Eric and Wendy Schmidt that supports academics whose work involves AI.

The fellows list is a who’s who of AI luminaries, and though not all of them made it out to the Bay, every time I turned a corner I saw a scientist whom I’d interviewed previously or whose research I admired. It’s a weird time for university AI researchers, who make up most of the AI2050 group. Read Grace’s story to find out why, and what could be coming next .

This story is from The Algorithm, our weekly AI newsletter. Sign up to receive it in your inbox every Monday.

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Nvidia has secured $500 billion from Wall Street for AI infrastructure It’s struck deals with BlackRock, Goldman Sachs, and four others. ( BBC ) + Showing the pull of AI compute for ⁠institutional investors. ( Reuters $) + And that AI infrastructure is becoming a new asset class. ( CNBC ) 2 Mark Zuckerberg’s new manifesto says open-source AI can save the US It presents a utopian vision of personalized “superintelligence.” ( Guardian ) + And arrived the same day as Meta’s new, open-source model. ( NYT $) + Zuckerberg said he plans to launch more of these models. ( WSJ $) + And pit Meta against Chinese open-weight developers. ( SCMP ) 3 Bernie Sanders has called on Silicon Valley to “pause AI development” He noted that AI giants have pledged to do this if necessary for safety. + And warned that lawmakers will step in if no action is taken. ( Guardian ) + House Democrats are already pressing AI leaders over rogue models. ( WP $) + A populist backlash is building against AI. ( MIT Technology Review ) 4 A US court will allow thousands of social media lawsuits to proceed The suits target addictive mechanisms used by Meta, TikTok, Google, and Snapchat. ( Axios ) + They claim the platforms are designed to hook young users. ( Reuters $) + Can we repair the internet? ( MIT Technology Review ) 5 Unitree’s IPO is more than 8,000 times oversubscribed by retail The Chinese humanoid firm raised $900 million ahead of its listing. ( Reuters $) + Its pricing for the Shanghai IPO values the company at $9 billion. ( FT $) 6 Flock’s car-tracking cameras are facing a bipartisan backlash The surveillance network has spread rapidly across the US. ( NYT $) + Flock also plans to chase shoplifters with drones. ( MIT Technology Review ) 7 China is breaking up AI relationships Beijing has introduced new rules for emotionally interactive AI. ( Rest of World ) + It’s surprisingly easy to fall for a chatbot. ( MIT Technology Review ) 8 An AI tool claims to pick the best 1% of scientific papers But researchers doubt that AI can reliably judge scientific quality. ( Nature ) 9 The AI slop backlash is working It’s pushing platforms to restrict AI-generated content. ( Wired $) 10 An 82-year-old rejected $26 million to turn her farm into a data center She criticised the environmental impacts of data centers. ( Fortune )