Turing Award winner Richard Sutton says pure generative AI can't do real science

Richard Sutton, winner of the prestigious Turing Award, has identified a fundamental flaw in traditional generative AI systems: their inability to assess their own outputs. According to Sutton, this limitation renders them incapable of truly scientific discovery, as novel findings are fleeting and cannot be built upon. Sutton points to the constrained nature of current AI models, exemplified by systems like AlphaGo and AlphaProof, which have achieved remarkable success in specific domains.

However, he argues that these systems' creative potential is stifled by their lack of built-in evaluation mechanisms. Without the ability to critically evaluate their own results, AI systems are unable to engage in meaningful scientific inquiry. The issue, as Sutton sees it, lies in the fact that generative AI models are typically designed to produce novel outputs, but lack the capacity to determine whether these outputs are genuinely insightful or merely interesting.

As a result, the scientific community is left to sift through the generated results, searching for meaningful contributions. In contrast, AI systems equipped with built-in evaluation loops, such as those employed in reinforcement learning, can iteratively refine their outputs and genuinely advance scientific knowledge. By integrating evaluation and generation, these systems can sustain creative progress and make lasting contributions to their respective fields.

The implications of Sutton's argument are far-reaching, suggesting that the development of more sophisticated AI systems will require the integration of evaluation and generation capabilities. Only then, he contends, can AI truly be said to be engaging in scientific discovery. Sutton's comments offer a thought-provoking critique of the current state of AI research, highlighting the need for a more nuanced understanding of the complex interplay between generation, evaluation, and creativity in scientific inquiry.