HomeRoboticsInterview with Tao Chen, Jie Xu and Pulkit Agrawal: CoRL 2021 greatest...

Interview with Tao Chen, Jie Xu and Pulkit Agrawal: CoRL 2021 greatest paper award winners


Congratulations to Tao Chen, Jie Xu and Pulkit Agrawal who’ve gained the CoRL 2021 greatest paper award!

Their work, A system for basic in-hand object re-orientation, was extremely praised by the judging committee who commented that “the sheer scope and variation throughout objects examined with this technique, and the vary of various coverage architectures and approaches examined makes this paper extraordinarily thorough in its evaluation of this reorientation activity”.

Under, the authors inform us extra about their work, the methodology, and what they’re planning subsequent.

What’s the matter of the analysis in your paper?

We current a system for reorienting novel objects utilizing an anthropomorphic robotic hand with any configuration, with the hand dealing with each upwards and downwards. We reveal the aptitude of reorienting over 2000 geometrically totally different objects in each circumstances. The realized controller can even reorient novel unseen objects.

May you inform us concerning the implications of your analysis and why it’s an attention-grabbing space for examine?

Our realized talent (in-hand object reorientation) can allow quick pick-and-place of objects in desired orientations and areas. For instance, in logistics and manufacturing, it’s a widespread demand to pack objects into slots for kitting. At present, that is normally achieved through a two-stage course of involving re-grasping. Our system will be capable to obtain it in a single step, which may considerably enhance the packing velocity and enhance the manufacturing effectivity.

One other software is enabling robots to function a greater diversity of instruments. The most typical end-effector in industrial robots is a parallel-jaw gripper, partially attributable to its simplicity in management. Nonetheless, such an end-effector is bodily unable to deal with many instruments we see in our each day life. For instance, even utilizing pliers is tough for such a gripper because it can’t dexterously transfer one deal with forwards and backwards. Our system will enable a multi-fingered hand to dexterously manipulate such instruments, which opens up a brand new space for robotics functions.

May you clarify your methodology?

We use a model-free reinforcement studying algorithm to coach the controller for reorienting objects. In-hand object reorientation is a difficult contact-rich activity. It requires an amazing quantity of coaching. To hurry up the training course of, we first practice the coverage with privileged state data resembling object velocities. Utilizing the privileged state data drastically improves the training velocity. Apart from this, we additionally discovered that offering an excellent initialization on the hand and object pose is crucial for coaching the controller to reorient objects when the hand faces downward. As well as, we develop a method to facilitate the coaching by constructing a curriculum on gravitational acceleration. We name this system “gravity curriculum”.

With these strategies, we’re capable of practice a controller that may reorient many objects even with a downward-facing hand. Nonetheless, a sensible concern of the realized controller is that it makes use of privileged state data, which could be nontrivial to get in the true world. For instance, it’s arduous to measure the item’s velocity in the true world. To make sure that we are able to deploy a controller reliably in the true world, we use teacher-student coaching. We use the controller skilled with the privileged state data because the instructor. Then we practice a second controller (pupil) that doesn’t depend on any privileged state data and therefore has the potential to be deployed reliably in the true world. This pupil controller is skilled to mimic the instructor controller utilizing imitation studying. The coaching of the coed controller turns into a supervised studying drawback and is subsequently sample-efficient. Within the deployment time, we solely want the coed controller.

What had been your foremost findings?

We developed a basic system that can be utilized to coach controllers that may reorient objects with both the robotic hand dealing with upward or downward. The identical system can be used to coach controllers that use exterior assist resembling a supporting floor for object re-orientation. Such controllers realized in our system are sturdy and can even reorient unseen novel objects. We additionally recognized a number of strategies which are essential for coaching a controller to reorient objects with a downward-facing hand.

A priori one may imagine that it will be important for the robotic to learn about object form with the intention to manipulate new shapes. Surprisingly, we discover that the robotic can manipulate new objects with out realizing their form. It means that sturdy management methods mitigate the necessity for complicated perceptual processing. In different phrases, we would want a lot easier perceptual processing methods than beforehand thought for complicated manipulation duties.

What additional work are you planning on this space?

Our rapid subsequent step is to attain such manipulation expertise on an actual robotic hand. To realize this, we might want to sort out many challenges. We are going to examine overcoming the sim-to-real hole such that the simulation outcomes could be transferred to the true world. We additionally plan to design new robotic hand {hardware} by way of collaboration such that your complete robotic system could be dexterous and low-cost.


In regards to the authors

Tao ChenTao Chen is a Ph.D. pupil within the Inconceivable AI Lab at MIT CSAIL, suggested by Professor Pulkit Agrawal. His analysis pursuits revolve across the intersection of robotic studying, manipulation, locomotion, and navigation. Extra just lately, he has been specializing in dexterous manipulation. His analysis papers have been revealed in prime AI and robotics conferences. He obtained his grasp’s diploma, suggested by Professor Abhinav Gupta, from the Robotics Institute at CMU, and his bachelor’s diploma from Shanghai Jiao Tong College.

Jie XuJie Xu is a Ph.D. pupil at MIT CSAIL, suggested by Professor Wojciech Matusik within the Computational Design and Fabrication Group (CDFG). He obtained a bachelor’s diploma from Division of Laptop Science and Expertise at Tsinghua College with honours in 2016. Throughout his undergraduate interval, he labored with Professor Shi-Min Hu within the Tsinghua Graphics & Geometric Computing Group. His analysis primarily focuses on the intersection of Robotics, Simulation, and Machine Studying. Particularly, he’s within the following matters: robotics management, reinforcement studying, differentiable physics-based simulation, robotics management and design co-optimization, and sim-to-real.

Pulkit AgrawalDr Pulkit Agrawal is the Steven and Renee Finn Chair Professor within the Division of Electrical Engineering and Laptop Science at MIT. He earned his Ph.D. from UC Berkeley and co-founded SafelyYou Inc. His analysis pursuits span robotics, deep studying, pc imaginative and prescient and reinforcement studying. Pulkit accomplished his bachelor’s at IIT Kanpur and was awarded the Director’s Gold Medal. He’s a recipient of the Sony College Analysis Award, Salesforce Analysis Award, Amazon Machine Studying Analysis Award, Signatures Fellow Award, Fulbright Science and Expertise Award, Goldman Sachs International Management Award, OPJEMS, and Sridhar Memorial Prize, amongst others.


Discover out extra

  • Learn the paper on arXiv.
  • The movies of the realized insurance policies can be found right here, as is a video of the authors’ presentation at CoRL.
  • Learn extra concerning the successful and shortlisted papers for the CoRL awards right here.

tags: ,


Lucy Smith
is Managing Editor for AIhub.

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments