Anthill Inside 2017

Most of the people would have heard of Deepmind’s AI getiing really good at playing Atari games.

In this talk, we’ll focus on developing a similar but simpler agent that uses Reinforcement Learning algorithms to learn to play such games on its own. The session will progress from a gentle introduction to the Reinforcement Learning (RL) concepts, steps involved gradually in identifying and modelling a RL problem. The complexity of RL algorithm described in the session will escalate from simple table based Q-learning algorithm, to simple neural network based RL and then finally to highly efficient Deep Q networks.

For demonstration, we’ll be using Open AI gym to simulate the game environment and ipython notebooks to document and display the code.

After this beginner-friendly session, the audience will be able to understand how Reinforcement Learning algorithms work at a broader level and develop their own game agent.

Outline

First 10 minutes:

Initially, the audience is introduced to Reinforcement Learning (RL) and some of the standard terms and concepts like Agents, state, policy, etc.

Next 10 minutes:

We will walk through the implementation of Q-Learning (an RL technique) to develop an Agent that learns to adapt to the game environment provided by Open AI and gets smarter with every move by learning ‘policies’ and ‘strategies.’

Next 15 minutes:

We’ll talk through the design of simple single-layered Neural network algorithm and finally, go through the implementation of Deep-Q network for the same game environment.

Final 10 minutes:

The session will conclude with some general discussion regarding scope and limitations of these Reinforcement learning techniques and the current state of the Reinforcement Learning in terms of applications and availability of open source libraries.

Requirements

Basic programming knowledge, rough knowledge of how Neural Networks work will also be helpful.

Speaker bio

A Junior pursuing Bachelors in Software Engineering. Also, a Freelance Software Developer having experience in Machine Learning and Web Development.

I actively contribute to some Open Source projects. In my leisure time, I like to learn about new technologies like AI, Decentralized Applications, AR/VR, explore the trends, and work on ideas involving these technologies.

I usually conduct sessions on Python and Android development as a part of DTU: Open Source Software Development group. I’ve gained all the knowledge this talk requires from some quality MOOCs, popular articles on the web, and Udacity’s Machine Learning Nanodegree that I was enrolled in.

I regularly participate in Hackathons and community meetups and have won several hackathons over the past year. And lastly, I like all things Python!

Links

• I’ve worked on a couple of RL projects in the past including a Nokia-style classic snake game and a self-driving car simulation. Learning snake - https://github.com/satwikkansal/snakepy
• Self-driving car simulation - https://github.com/satwikkansal/smartcab
• I’ve been selected as a speaker for EuroPython 2017, Rimini for the workshop on a similar topic “Developing self-learning snake game using Reinforcement Learning.” https://ep2017.europython.eu/conference/talks/developing-a-self-learning-snake-game-using-reinforcement-learning
• My Home page: http://www.satwikkansal.xyz
• Code samples: https://github.com/satwikkansal/RL_agents

Slides

https://docs.google.com/presentation/d/1vBW8-ML5PMoQIE5oF1zhYUTqSQIL75ubMRG-LHtQuvw/edit?usp=sharing