Built Robotics, Penn xLAB to develop physical AI for construction

Built will leverage small mobile robots equipped with a sensor suite to scan jobsites and build a dataset to be analyzed by xLAB researchers at the University of Pennsylvania. | Credit: Built Robotics

The University of Pennsylvania’s Safe Autonomous Systems Lab (xLAB) is teaming up with Built Robotics to turn construction sites into a proving ground for “physical AI.” Built plans to use its large construction robotics dataset and a new purpose-built, data-collection robot to develop a world foundation model for how machines and people can safely coexist on the job site.

Built Robotics has been in the field since 2016, developing autonomous controls for large construction equipment . The company entered the utility-scale solar market in 2023 with the announcement of a new product called the RPD 35, or Robotic Pile Driver. Since its inception, Built has amassed more than 50,000 hours of operations, installed more than 3 gigawatts of solar, and is deployed at 40+ sites.

Rahul Mangharam is a professor in electrical and systems engineering and the principal investigator of xLAB at Penn Engineering. Noah Ready-Campbell, founder and CEO of Built Robotics, is a Penn alumn, so this relationship is an obvious one for both parties. Mangharam has an interest in the safety-critical issues with automating outdoor construction equipment, and the partnership will bring real-world data back to the lab.

Built expects to collect diverse personnel and environment data beyond what the work-focused piling and trenching robots naturally see. This includes more edge cases such as odd body poses, occlusions, weird lighting, unexpected human behavior.

“xLAB is committed to building safety-critical autonomous systems for real-world deployment, and construction represents one of the most demanding frontiers for that work,” said Mangharam. “The fundamental challenge is bridging the gap between validation in controlled environments and robust performance under operational conditions. Our collaboration with Built will give us access to active jobsites with high-fidelity mapping data and real-world operational parameters, enabling us to build practical autonomous systems solving a real-world need.”

“What xLAB has built in safety architecture is precisely the kind of rigorous foundation that physical AI demands,” said Ready-Campbell. “Our proprietary edge AI model for personnel detection has been refined across some of the most demanding operational environments in the industry — active construction sites with hundreds of employees stretching over thousands of acres.”

By systematically collecting and labeling edge cases, the collaboration will design AI models capable of detecting humans in atypical conditions and unique construction environments. This advanced training pushes the edge AI model toward “superhuman” perception, enabling it to identify unusual, transient dangers that humans might miss.