Since its inception, robotics has made significant strides, with robots now widely used in many industries, including home monitoring and electronics, nanotechnology, aerospace, and many others. These robots are able to process complex, high-dimensional data and decide the best possible actions to take. They do this by constructing abstractions, i.e. compact summaries of what they see and what actions they can take, which helps them generalize their tasks. Researchers have mainly focused on learning these abstractions or summaries from data rather than constructing them by hand.

In this work from Microsoft, the research team has focused on temporal action abstractions, e.g. in breaking down complex policies into low-level tasks such as item collection, walking, etc. They believe that this technique has great potential for action representation learning. They have introduced a new method called Primitive Sequence Encoding (PRISE), which helps teach robots multistep skills. The results show that PRIZE allows the robot to learn faster and perform better than if it had been trained on all action codes at once.

PRIZE has been inspired by NLP ideas. The method takes the robot’s continuous actions and converts them into a set of discrete codes. The researchers pre-train a vector quantization module for this task and then apply the Byte Pair Encoding (BPE) technique (which is used in NLP for text compression) to identify these small routines within the action codes. Behavior Cloning is used to test the robot, in which PRIZE leverages these small routines or skills instead of the full set of instructions, making the process faster and more efficient than other methods.

The researchers also evaluated the effectiveness of the skill features of PRISE. They first pre-trained them using large-scale offline multi-task datasets and then evaluated them in two offline imitation learning (IL) scenarios – learning a general multi-task policy and adapting a few downloads to unseen tasks.

For the first task, the researchers evaluated the average success rate across 90 tasks in the LIBERO-90 data set. They observed that the use of discrete skills in PRIZE leads to significant performance improvement compared to other existing algorithms. The researchers also evaluated PRIZE’s 5-shot IL performance in five MetaWorld unseen tasks. They found that PRIZE outperforms all other baselines by a wide margin, highlighting its effectiveness in adapting to invisible downstream tasks.

In conclusion, this research paper presents PRISE, a new method that allows the robot to learn faster and perform better than if it was trained simultaneously on all action codes. They leverage an NLP methodology called the Byte-Pair Coding Algorithm that enables efficient multi-task policy learning and adaptation of few downloads to non-occurring tasks. Experiments also demonstrate the superiority of the method over other algorithms, and this work has the potential to further improve the performance of robots in different tasks.

check it Paper and Work. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us Twitter. Join us Telegram channel, Discord Channeland LinkedIn Groops.

If you like our work, you will love our work newsletter..

Don’t forget to join us 38k+ ML SubReddits