Companies hire thousands to train AI robots for real-world tasks

A growing army of data trainers across India and other countries is teaching AI robots how to perform physical tasks by capturing thousands of hours of human movements and demonstrations. Companies are racing to build “physical AI” that can operate in the real world, creating a new industry around human movement data collection that could enable the next generation of autonomous robots in homes and workplaces.

The big picture: Major tech companies including Tesla, Boston Dynamics, Nvidia, and Google are investing heavily in humanoid robotics, with Nvidia projecting the market could reach $38 billion over the next decade.

• Unlike digital AI that learned from internet data, physical AI requires precise human demonstration data showing how to manipulate objects, fold clothes, and navigate real-world environments.
• Companies like Jeff Bezos-backed Physical Intelligence and Figure AI are building foundation models specifically designed for robotic movement and manipulation.

How the training works: Data collection companies employ thousands of workers to perform repetitive physical tasks while being recorded from multiple angles.

• At Objectways in Karur, India, engineers like 28-year-old Naveen Kumar spend their days folding towels hundreds of times while wearing GoPro cameras to capture exact point-of-view footage.
• Each movement must be precisely executed—picking up towels with the right hand, shaking them straight, folding three times, and placing in specific locations—or the entire sequence gets deleted.
• The videos are then annotated by teams who draw boxes around objects, tag movements, and classify each gesture to teach AI systems the nuances of human manipulation.

Key training methods: Companies are using multiple approaches to gather physical movement data for AI training.

• Teleoperation: Humans use controllers to remotely guide robots through tasks like picking up cups or making tea, with AI learning from both successful and failed attempts.
• Human data capture: Companies pay people to wear smart glasses that record everyday actions in countries including Brazil, Argentina, India, and the United States.
• Staged environments: Figure AI partnered with Brookfield, a real estate giant, to capture footage from inside 100,000 homes to teach robots how to move in human spaces.

The challenges: Creating usable training data requires extremely specific conditions and precision.

• Dev Mandal, a 20-year-old entrepreneur in Bengaluru, had to abandon his movement data collection business because clients demanded exact specifications down to table colors and lighting conditions.
• “Everything, down to the color of the table, had to be specified by them,” Mandal explained. “And they said that this has to be the exact color.”
• Objectways regularly deletes 150-200 videos due to small errors in folding or placement techniques.

Scale of operations: The data collection industry is rapidly expanding to meet robotics demand.

• Objectways has over 2,000 employees, with half working on autonomous car and robotics data labeling.
• Meta-backed Scale AI has collected 100,000 hours of training footage for robotics through its San Francisco prototype laboratory.
• Warehouses are being planned in Eastern Europe where teams of operators will use joysticks to guide robots remotely across the world.

What they’re saying: Workers acknowledge the irony of training their potential replacements.

• “Sometimes the robot’s arms throw the clothes and won’t fold properly. Sometimes it scatters the stack,” said Kavin, an Objectways employee. “In five or 10 years, they’ll be able to do all the jobs and there will be none left for us.”
• “Companies are building foundation models fit for the physical world,” explained Ulrik Stig Hansen, co-founder of data management platform Encord. “There’s this huge resurgence in robotics.”

The broader implications: Advanced robotics could reshape labor markets by automating physical tasks currently performed by humans.

• Optimists believe robots will free humans from tedious work and lower labor costs, giving people more time for interesting and important work.
• Critics worry about widespread job displacement and unemployment as robots become capable of performing increasingly complex physical tasks.
• The current generation of teleoperated humanoids faces criticism for being “more sizzle than substance,” impressive when controlled but still far from fully autonomous operation.