R250 Imitation Learning

Imitation learning was initially proposed in robotics as a way to better robots (Schaal, 1999). The connecting theme is to combine the reward function in the end of the action sequence with demonstrations of the task in hand by an expert. Since then it has been applied to a number of tasks which can be modelled as a sequence of actions taken by an agent. These include the video game agents, moving cameras to track players and structured prediction in various tasks in natural language processing.

Over the years there has been a number of algorithms proposed, in the literature but without necessarily making the connections between the various approaches clear. The initial lecture will set the criteria to be used to examine the algorithms with.

The papers presented in the 2020 version of the topic were:

• Learning to search better than your teacher Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daumé III and John Langford International Conference on Machine Learning (ICML), 2015

• Sequence Level Training with Recurrent Neural Networks Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, Wojciech Zaremba International Conference on Machine Learning (ICLR), 2016

• Neural Modular Control for Embodied Question Answering Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh and Dhruv Batra 2nd Conference on Robot Learning (CoRL 2018)

• Imitation Learning with Concurrent Actions in 3D Games Jack Harmer, Linus Gisslen, Jorge del Val, Henrik Holst, Joakim Bergdahl, Tom Olsson, Kristoffer Sjoo, Magnus Nordin 2018 IEEE Conference on Computational Intelligence and Games (CIG)