Controlling Humanoid Robots with Your Body: A New Job Frontier in Shenzhen

Humanoid Robots with Your Body: A New Job Frontier in Shenzhen">

At IO-AI Tech, a startup about 45 minutes north of downtown Shenzhen, China, workers wearing the company's VR headsets, handheld controllers, and motion-tracking gear remotely control humanoid robots for workplaces like factory floors and convenience stores. The company wants the robots to do useful work, like stocking shelves and picking items out of bins, but it also wants to gather training data that could one day let the bots operate autonomously. To show off the tech, the company invited me to its offices, where I was allowed to control 10 humanoid robotic hands, each from a different company, using a custom motion-tracking glove.

The device instantly transferred my finger movements to all 50 robotic digits. I'm a little embarrassed to say that the first thing I tried with this futuristic gear was getting all 10 hands to flip the bird. After getting this out of my system, I was impressed by how quickly my movements transferred to the robot hands, and how easily the tech went both ways—I was able to feel a ball placed in one of the electronic hands.

The company also let me try a system that's being tested by a Chinese convenience store chain. Using a VR headset and a pair of grippers, I tried picking up boxes of medication from a shelf. It was disorienting at first: I had to adjust to a slight difference between my movements and those of the robot I could see through the headset.

After a little practice, however, I was stacking shelves like a robot-boss. Elsewhere, I watched people wearing virtual reality headsets and body-tracking sensors reminiscent of Ready Player One. In one large room, I saw workers using a range of different systems to control diminutive Unitree humanoids.

One person marched around with a Unitree robot next to them, and the machine mirrored their movements within a mocked-up apartment. The human operator, wearing a headset and viewing the scene through the robot's eye-level cameras, went through the motions needed to remove a shirt from a hanger and fold it. IO-AI develops technology that transfers a person's movements to different robot forms—a useful offering because there are dozens of different humanoids and robot hands on the market in China today.

The startup's algorithms also need to combine human control with some level of autonomy because a person and a robot aren't always going to be the same shape, size, and weight. Without some ability to move independently, the robot may lose its balance. Why this matters: The emergence of humanoid robot control technology has significant implications for industries like manufacturing, logistics, and retail.