Skild AI unveils S1 flagship robot foundation model

Skild AI said S1 enables in-context learning for robotics. | Source: Skild AI

Skild AI recently unveiled S1, the company’s flagship robot foundation model. Since it was founded in 2023, the company has raised nearly $1.7 billion in funding to create a general-purpose robot brain.

With S1, robots can learn new, complex tasks from watching a single video, Skild AI claimed. To do this, the company ‘s model uses in-context learning, Deepak Pathak , Skild AI co-founder and CEO, told The Robot Report .

“You just add a video of a human doing something in the prompt, also called the context of the model, and it can just follow it on the robot,” Pathak said. “The tasks we are showing are extremely complex and long horizon. They are not three-second, four-second tasks, not those tiny, simple tasks.”

Typically, when faced with new tasks, AI models on robots need to be post-trained to handle them. This can be a lengthy process, and it holds robotics back from scaling. According to Pathak, there are four kinds of robot training data, and Skild makes use of all of them.

“If you look at any company out there, they are primarily focusing on one of these sources. But, if we think from first principles, we realize there is no golden path,” Pathak said. “We have to use all of them because the pros of one source compensates for the downside of another source.”

“Human videos are diverse, but far from a robot. You can combine that with glove or teleoperation data,” he continued. “This is how we combine our data sources for pre-training.”

Right now, Skild AI said it isn’t focusing on any one specific task or industry. “Many people on social media have been asking questions, and we have been continuously replying with new tasks, because it’s pretty generalized,” Pathak said.

Importantly, Skild is focusing on longer tasks that can take up to 10 minutes, including things like repotting a plant, making a cup of coffee, or cooking pancakes.

“When the robot flipped the pancake, we were so paranoid that we went back and checked all of our millions of hours of data to look for any flipping example of any kind, and there was none,” Pathak said. “This flipping basically emerged by looking at how the spatula is moving and the robot could just do it.”

The Skild brain is also “omni-bodied.” Pathak said it can work on anything from a quadruped to a humanoid or a static arm . In the future, the company does plan to spend more time improving the model’s performance on humanoids.