Anthill Inside Miniconf – Pune

Deep Reinforcement Learning has been becoming very popular since the dawn of DeepMind’s AlphaGo and DQN. Algorithms that learn to solve a game (sometimes better than) humans seems very complex from a distance, and we shall unravel the mathematical workings of such models through simple processes. This workshops aims to provide a simple insight about Reinforcement Learning and going to Deep RL.

Outline

This session aims to give a gentle introduction to Reinforcement Learning for beginners and moving from simple Dynamic Programming based approaches to Deep RL methods leveraging a variety of Deep Learning methods including but not limited to Convolutions, Recurrent architectures, Attention mechanisms etc. All demonstrations will make use of Keras (with Tensorflow backend) running on a Jupyter Notebook. Some advanced experiments involve the use of OpenAI gym (library for simulating game environments) and/or PyGame. The workshop will be divided into the following sections:

• Introduction (30 mins)
  • Introduction to Reinforcement Learning and problem description.
  • Intuition about observation-reward based learning and policy evaluation.
  • Markov Decision Process (MDP)
• Q-Learning (45 mins)
  • Discussion about Value learning and Q-learning.
  • Understanding Q-learning with a grid world (Toy problem)
  • Learning about playing games from visual input.
• Deep Q-learning (45 mins)
  • Improvement over simple Q-learning (Approximate Q-learning)
  • SARSA with Deep learning, A simple example on a toy game (Tic-Tac-Toe or Pong)
  • DQN for Atari games, a brief discussion
• Final Project (~40 mins)
  • Implement the above algorithms on a few games.
  • Games including Flappy bird, 2048 etc.
  • Discussion about what to do next and how the field is progressing.

After this workshop, you should be able to build your own reinforcement models for solving semi-supervised tasks and catch-up with the recent research in Deep Reinforcement Learning. A concluding discussion about how to progress further and doubts.

Requirements

Mathematics:
Basic knowledge of calculus and linear algebra (matrix oeprations, vectors etc.)

Programming:
Familiarity with atleast one programming language (Python preferable)
Basic programming concepts such as Functions, loops, conditions, classes (OOP concepts) etc.

Computing:
A Laptop with Python IDE.
The following libraries are required for the final project and running the demo:

• Numpy
• Scipy
• Matplotlib
• keras
• tensorflow
• OpenAI gym
• Pygame
• OpenCV (optional)

code shall be provided in the form of repository at the beginning of the session so that participants mey interact with the demos while adding their code snippets.

Speaker bio

Shubham Dokania is currently a Research Intern at Ambient.ai (https://ambient.ai/). Recently he was at CVIT, IIIT-H working under the supervision of Dr. C.V. Jawahar. Previously, a Machine Learning Instructor at Coding Blocks, while parallely working as a research intern at IIIT Delhi, supervised by Dr. Ganesh Bagler. Completed his B.Tech in Mathematics and Computing from Delhi Technological University (Formerly DCE) in 2017. At Coding Blocks, he taught undergrad/grad students and industry professionals about the techniques of Machine Learning with an inclination towards the research background of the methods discussed. Shubham has also spoken at PyData Delhi 2017 and local meetups including PyData Delhi Meetup, DTU workshop sessions and Bootcamps across New Delhi.

Links

• github: https://github.com/shubham1810
• Previous Deep RL workshop: https://github.com/shubham1810/PyDataDelhi_RL_workshop

Slides

http://slides.com/shubhamdokania/drl-pydata17#/