LeRobot v0.6.0: Closing the Robot Learning Loop

LeRobot v0.6.0 is the latest release of the open-source robot learning framework, focusing on closing the robot learning loop. This involves policies that imagine the future before acting, reward models that evaluate success, and deployment tools that turn failures into training data. The new release includes world model policies such as VLA-JEPA, FastWAM, and LingBot-VA, which learn to imagine the future as part of their training.

These policies take different approaches to make imagination affordable, such as predicting the future in latent space or anticipating upcoming frames from the model's own actions. LeRobot v0.6.0 also introduces a new reward models API, which includes four reward models: Robometer, TOPReward, HIL-SERL, and SARM. These models enable success detection and progress estimation in the robot learning loop.

Robometer is a pretrained, general-purpose reward model that scores task progress and success from raw video and language instructions. TOPReward is a zero-shot reward model that wraps an off-the-shelf VLM and reads the log-probability of the token 'True' given the trajectory video and task instruction. The release also includes several new simulation benchmarks, unified under the lerobot-eval CLI, which allow for standardized evaluation of robot learning policies.

Additionally, the lerobot-rollout CLI makes deployment a separate workflow, with pluggable strategies and inference backends. Other notable features in LeRobot v0.6.0 include depth sensing, VLM-powered dataset annotation, custom video encoding, and cloud training on HF Jobs. The release also includes hundreds of bug fixes, documentation improvements, and quality-of-life upgrades across the codebase.

Why this matters: The LeRobot v0.6.0 release has significant implications for the robotics industry, as it provides a more comprehensive framework for robot learning and adaptation. The introduction of world model policies, reward models, and deployment tools enables robots to learn from their environment and adapt to new situations more effectively. This could lead to improved performance and efficiency in various robotic applications, from manufacturing to healthcare.

Furthermore, the open-source nature of LeRobot allows developers and researchers to build upon and contribute to the framework, accelerating progress in the field. As robots become increasingly integrated into our daily lives, the ability to learn and adapt will be crucial for their success. The LeRobot v0.6.0 release takes a significant step towards achieving this goal.