Google's Genie World Model Brings Interactive Street View Simulations to Life

['Imagine dropping the little person icon onto the streets of Paris to see if you booked a hotel in a cool neighborhood, but with a twist: you can adjust the weather, time of day, and even simulate extreme scenarios like a "Day After Tomorrow"-style apocalypse. That\'s now possible thanks to Google\'s latest integration of Street View with its general-purpose world model, Genie. The new feature, launched during the Google I/O developer conference, allows users to simulate real-world environments in a more immersive and interactive way.', 'The integration is a significant development for both robotics and human users.

According to Jack Parker-Holder, a research scientist on DeepMind\'s open-endedness team, "It\'s really powerful for both the agent [and robotics] use case and for humans to play with, and that\'s always been the thesis of Genie." For example, a new robot being deployed in London, which rarely sees the sun, can use Genie to simulate those scarce occasions when the sun glints off the Victorian housing, so the rays don\'t shock the robot when it happens.', 'Google has been collecting Street View data for 20 years, amassing over 280 billion images across 110 countries and seven continents. By combining this rich source of real-world information with Genie\'s ability to simulate worlds, the company aims to create a powerful tool for educational experiences, gaming, and robotics training. Genie 3, the latest version of the world model, has already been used to power one of Waymo\'s simulators to train its self-driving cars on "exceedingly rare events" like tornadoes or casual elephant encounters.', 'The addition of Street View data to Genie could help Waymo prepare to launch in more cities around the globe.

According to Parker-Holder, Genie allows for simulating a world anchored to a real place, but also shifting the point of view to other types of agents, like a human or a robot. Google is launching Street View in Genie to some Ultra users in the United States starting today, with access rolling out at scale over time. Global Ultra users will gain access over the next few weeks.', 'While the technology is still an experiment and not yet photorealistic, the results are impressive and recognizable.

The models are also not yet physics-aware, meaning they don\'t yet understand cause and effect. However, researchers believe that with time, the accuracy and quality of the simulations will improve. "I think for this kind of model, it\'s maybe six to 12 months behind video in terms of the accuracy and quality, so I think it\'s something we will solve," Parker-Holder said.']