Teaching a Robot To Hang a T-shirt Starts With Data
Media Inquiries
麻豆村 sophomore Jasmine Li knew since high school that she wanted to explore robotics while majoring in聽.
Building on her experience volunteering at retirement homes before 麻豆村, Li chose to apply for a聽Summer Undergraduate Research Fellowship(opens in new window) award to combine robotics and teaching technology as a tool for success. She decided to focus on assistive robotics that help people with everyday tasks.
鈥淚 was interested in the side of robotics that helps people who might not be as familiar with technology,鈥� she said. 鈥淚 was thinking about the hardware side of robotics, but I ended up doing a lot more with the data collection and software 鈥� the algorithmic side.鈥�
For her project, she worked with Ph.D. student Zheyuan Hu in the聽 led by assistant professor聽.听
The Summer Undergraduate Research Fellowship (SURF) program awards $4,500 to undergraduates at Carnegie Mellon for 8-10 full-time weeks of summer research on campus in any field of study.
鈥淚 got to explore what robotics looks like behind the scenes, and it demystified some of the ideas I had about the field,鈥� she said. 鈥淩obotics has so many opportunities for innovation and growth through research.鈥�
Li worked with a bimanual robot arm setup 鈥� two multijointed arms clamped on a table 鈥� that can be controlled by a human remotely using a pair of VR joysticks, or operated fully autonomously via a neural network.听
She analyzed the robot's behaviors in both simulations and real-world tasks in order to study how robots fail when imitating complex human activities, such as hanging shirts.
鈥淲e found that when a human tries to insert a hanger, the person will sometimes do minuscule corrections, but we had a theory that the robot might learn better if we corrected the task on a larger scale,鈥� she said.听
So, rather than a tiny twist or adjustment, they guided the robotic arms to return to their original position before attempting to hang the shirt again more accurately.
Using the newfound data collection method, training the robot became more efficient, collecting more data and improving performance with fewer iterations of human teaching, Li said.
The team also聽 using other trials that tasked the robot with packing a burger into a takeout box and sealing an airtight container lid.
Introducing RaC:
A data collection protocol that boosts data efficiency by 10x compared to some of the best imitation results.
Key idea: scale recovery & correction data systematically => policies can reset+retry when acting (consistent self-correct) => better performance.
馃У0/狈鈥� Zheyuan Hu (@real_ZheyuanHu)
Hu applauded Li鈥檚 work designing the task and conducting the experiments, which were聽 accepted by the Making Sense of Data in Robotics: Composition, Curation, and Interpretability at Scale workshop at the 2025 Conference on Robot Learning (CoRL).听
鈥淛asmine did excellent work this summer, helping me design a challenging task and conduct experiments in the lab," Hu said. "Her hard work expedited both the ideation and iteration processes."
Li also enjoyed participating in聽Speak Up!(opens in new window) research presentations for undergraduate students. Inspired by her SURF project, Li is taking Introduction to Human-Robot Interaction this semester.
鈥淚t鈥檚 hard to train a robot to be able to complete multiple different tasks, what we call generalization,鈥� she said. 鈥淩obotics research, for now, focuses on training robots for specific tasks, but, eventually, everyone contributing to the research will help us get there.鈥�
