Researchers from Google DeepMind worked with Mila and McGill University to define appropriate reward functions to address the challenge of effectively training reinforcement learning (RL) agents. The reinforcement learning method uses a reward system to achieve desirable behaviors and punish undesirable ones. Therefore, designing efficient reward functions is crucial for RL agents to learn efficiently, but often requires considerable effort from environment designers. The paper proposes leveraging Vision-Language Models (VLM) to automate the process of generating reward functions.

Existing models defining the reward function for RL agents have been a manual and labor-intensive process, often requiring domain expertise. The paper introduces a framework called Code as Reward (VLM-CaR), which uses pre-trained VLMs to automatically generate dense reward functions for RL agents. Unlike VLMs’ direct query for rewards, which is computationally expensive and unreliable, VLM-CaR generates reward functions through code generation, greatly reducing the computational burden. With this framework, the researchers aimed to provide precise rewards that are interpretable and can be derived from visual inputs.

VLM-CaR works in three stages: program generation, verification programs, and RL training. In the first stage, pre-trained VLMs are asked to describe tasks and sub-tasks based on the initial and target images of an environment. The generated descriptions are then used to produce executable computer programs for each subtask. The generated programs are verified to ensure correctness using special and random trajectories. After the verification step, the programs act as reward functions to train RL agents. Using the generated reward function, VLM-CaR is trained for RL policies and enables efficient training even in environments with sparse or unavailable rewards.

In conclusion, the proposed method addresses the problem of manually defining reward functions by providing a systematic framework for generating interpretable rewards from visual observations. VLM-CaR demonstrates the potential to significantly improve the training performance and performance of RL agents in various environments.

check it Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us Twitter and Google news. Participation Our 38k+ ML SubReddit, 41k+ Facebook Community, Discord Channeland LinkedIn Groops.

If you like our work, you will love our work newsletter..

Don’t forget to join us Telegram channel

You might also like ours FREE AI Courses….