Summary
Animals can imitate behaviors after only a few observations, whereas current machine learning methods require orders of magnitude more data. In this work, we directly train MPAIL in the real world by combining adversarial imitation learning with model predictive control (MPC). This approach learns robust, sample-efficient policies from only a few expert demonstrations, greatly lowering data collection costs and eliminating Sim-to-Real transfer issues.
Problem
1. Prevailing imitation learning methods, such as Behavior Cloning, generally require large-scale, high-quality state–action demonstrations to achieve robust performance, which substantially increases the cost of data collection and expert supervision.
2. For tasks in complex and dynamic environments, designing realistic and task-relevant simulation environments remains challenging, and policies trained purely in simulation often face significant obstacles in Sim-to-Real transfer.
Solution: Real-World Model Predictive Adversarial Imitation Learning
We propose the Real-World Model Predictive Adversarial Imitation Learning (MPAIL) framework, which integrates Adversarial Imitation Learning (AIL) with Model Predictive Control (MPC) for direct deployment on real robots. In this framework, a discriminator learns state-transition costs, and a value function provides terminal cost-to-go estimates; both are used by a Model Predictive Path Integral (MPPI) planner to optimize trajectories. Unlike policy-based imitation methods that rely on simulation and Sim-to-Real transfer, our framework learns directly from a small number of state-only demonstrations and real-world interactions, achieving sample-efficient, interpretable, and robust policies with improved generalization to out-of-distribution scenarios.
Tasks
More tasks are being tested