Three researchers from the University of Maryland and a researcher from the National Information Communications Technology Australia (NICTA) used funding from DARPA’s Mathematics of Sensing, Exploitation and Execution (MSEE) program to develop a system that helps robots learn manipulation actions by watching “unconstrained” Internet videos – in this case, how-to cooking videos on YouTube.
The robots in this research endeavor were able to “recognize, grab and manipulate the correct kitchen utensil or object and perform the demonstrated task with high accuracy – without additional human input or programming,” according to DARPA’s press release.
The research paper, “Robot Learning Manipulation Action Plans by ‘Watching’ Unconstrained Videos from the World Wide Web,” notes two levels to this system: a lower level, which helps robots recognize hand grasp type and objects; and a higher level, “where a robot processes visual cues through a manipulation action-grammar module and translates them into actions,” according to Reza Ghanadan, program manager in DARPA’s Defense Sciences Offices.
This system also enables robots to continuously build on their previous learning, which opens up the door not only to robots that can learn faster but also to robots sharing their knowledge with other robots. “This learning-based approach is a significant step towards developing technologies that could have benefits in areas such as military repair and logistics,” Ghanadan said.
It’s only a matter of time before we see robots vs. humans cooking shows on TV. Let’s just hope the robots don’t learn their manners from YouTube comments.
