UW researchers tackle lack of robotic training data

Training AI models requires a large amount of data. For AI models like ChatGPT, they have access to vast amounts of text, images and videos from the Internet to be trained on – and even then, the question of whether AI is running out of training data has been asked before on Electronic Specifier. 

For the robotics industry, this is a very different situation: acquiring robot-specific data is costly, and there is a lack of readily available data due to the limited number of robots actively operating in diverse, real-world environments like homes.

In response to these challenges, researchers from the University of Washington (UW) have turned to simulation as a training method for robots. However, this approach, which often requires input from graphic designers or engineers, remains both time-consuming and expensive.

Two recent studies from researchers have introduced innovative AI systems that utilise video and photos to generate simulations capable of training robots for real-world applications. This advancement has the potential to dramatically reduce the cost of training robots for complex tasks.

In the first study, a user scans an area with a smartphone to capture its geometry. The system, known as RialTo, then creates a “digital twin” of the space, allowing users to specify how various elements function, such as the opening of a drawer. A robot can then practise these actions within the simulated environment, adjusting its movements slightly to optimise performance. The second study introduces URDFormer, a system that uses internet-sourced images of real environments to rapidly generate physically realistic simulation environments, enabling robots to train in a variety of settings.

These studies were presented at the Robotics Science and Systems conference in Delft, Netherlands, on 16th and 19th July.

“We’re trying to enable systems that cheaply go from the real world to simulation,” said Abhishek Gupta, a UW assistant professor in the Paul G. Allen School of Computer Science & Engineering and co-senior author on both papers. “The systems can then train robots in those simulation scenes, so the robot can function more effectively in a physical space. That’s useful for safety — you can’t have poorly trained robots breaking things and hurting people — and it potentially widens access. If you can get a robot to work in your house just by scanning it with your phone, that democratises the technology.”

While robots are currently well-suited for structured environments such as assembly lines, training them to interact with people in less predictable settings, such as homes, remains a significant challenge.

“In a factory, for example, there’s a ton of repetition,” said Zoey Chen, lead author of the URDFormer study and a UW doctoral student in the Allen School. “The tasks might be hard to do, but once you program a robot, it can keep doing the task over and over and over. Whereas homes are unique and constantly changing. There’s a diversity of objects, of tasks, of floor plans and of people moving through them. This is where AI becomes really useful to roboticists.”

The two systems tackle these challenges differently.

RialTo, developed by Gupta in collaboration with a team from the Massachusetts Institute of Technology, involves recording the geometry and moving parts of an environment, such as a kitchen, via video. The system employs existing AI models, with some quick input from a human using a graphic user interface, to create a simulated version of the recorded environment. 

A virtual robot then learns through trial and error, refining its ability to perform tasks, such as opening a toaster oven, within the simulated space. The experience gained in the simulation is then transferred to the physical environment, where the robot’s performance is almost as accurate as if it had been trained in the real world.

URDFormer, on the other hand, focuses on generating a large number of generic simulations quickly and cost-effectively. It uses images sourced from the internet and combines them with existing models to predict how elements in these environments, such as kitchen drawers and cabinets, might move. This allows for the rapid training of robots across a broad range of environments. However, the accuracy of these simulations is generally lower compared to those created by RialTo.

“The two approaches can complement each other,” Gupta noted. “URDFormer is really useful for pre-training on hundreds of scenarios. RialTo is particularly useful if you’ve already pre-trained a robot, and now you want to deploy it in someone’s home and have it be maybe 95% successful.”

Looking ahead, the RialTo team plans to test its system in real homes, having primarily worked in laboratory settings thus far. Gupta also intends to explore how integrating small amounts of real-world data with simulation data can improve outcomes.

“Hopefully, just a tiny amount of real-world data can fix the failures,” Gupta said. “But we still have to figure out how best to combine data collected directly in the real world, which is expensive, with data collected in simulations, which is cheap, but slightly wrong.”

Original article source:

https://www.electronicspecifier.com/industries/robotics/uw-researchers-tackle-lack-of-robotic-training-data

FAQ

1.Why is training data important for robotics?

Training data is essential for teaching robots how to perform tasks and make decisions. Robots rely on large datasets to learn from experience, improve their performance, and adapt to different environments. Without sufficient training data, it is difficult for robots to effectively navigate real-world scenarios.

2.What challenges are researchers facing with robotic training data?

The primary challenge is the lack of diverse and high-quality datasets that can simulate real-world complexities. Collecting and annotating data for every possible scenario a robot may encounter is time-consuming and expensive. This limits the ability of robots to generalize their learning to new environments.

3.How are UW researchers addressing this data shortage?

University of Washington (UW) researchers are developing new methods to generate synthetic data, use simulation environments, and transfer learning techniques to overcome the scarcity of real-world data. By leveraging these innovative approaches, they can train robots more efficiently without needing large volumes of manually labeled data.

4.What is synthetic data, and how is it used in robotics?

Synthetic data is artificially generated information that mimics real-world data. UW researchers use simulation tools to create synthetic environments where robots can practice tasks and gather training data. This helps them to bypass the limitations of real-world data collection and expand the diversity of training scenarios.

5.What role does simulation play in robotic training?

Simulation environments allow robots to be trained in virtual worlds that replicate real-world conditions. UW researchers are using advanced simulations to create complex, dynamic settings where robots can practice, learn, and adapt. These simulations provide scalable training data without the need for physical setups.

6.What is transfer learning, and why is it important for robotic training?

Transfer learning is a technique where a robot trained in one environment can apply its knowledge to a different, but related, environment. UW researchers are using transfer learning to reduce the amount of new data required when deploying robots in unfamiliar settings, enabling more efficient adaptation.

7.How will these advancements impact the future of robotics?

The approaches developed by UW researchers—such as synthetic data, simulation-based training, and transfer learning—will help accelerate the development of more adaptable, intelligent, and autonomous robots. These advancements will enable robots to be more flexible in real-world applications, from healthcare and logistics to manufacturing and beyond.