Anthill Inside 2017

PyTorch entered into the realm of DL framework with the promise of being “Numpy on GPU”. The obvious failures of static graph implementation for certain use cases is increasing industry wide adoption of PyTorch. Dynamic Computation Graph being the backbone of PyTorch, comes with some perks.

• Super intuitive
• Shallow learning curve
• Amazing community and discussion forum
• Easy debugging

It puts you explicitly in control of your compute, there’s no compiler trying to be clever and “help you” or speed your code up, but in reality mostly leading to a massive amount of debugging headaches. PyTorch should be the go-to framework for the problem with “dynamicity” in the core, especially for RNNs and highly dynamic Reinforcement Learning algorithms.
My talk would be to convince the audience about the advantages of PyTorch. For what problems pytorch would be the best solution. I will have benchmarks with popular DL frameworks that review speed, performance, memory, and easiness. I’ll try to build on top of a comparison foundation with TF and NumPy so that the audience could correlate with their day to day research/development projects. The demo would be to portray the best cases where pytorch really shines over other DL frameworks.

Note: Code I developed for the presentation would be available under MIT licence

Outline

• Computational Graphs [5 minutes]
• Static and Dynamic graphs [5 minutes]
• Introduction to PyTorch [10 minutes]
  • Variables
  • Functional
  • nn Module
  • Optim
  • AutoGrad
• Examples [10 minutes]
  • Simple Neural Networks
  • Encoder Decoder Network
  • Stack-augmented Parser-Interpreter Neural Network
• PyTorch and Caffe2 [2 minutes]
• How to visualize [5 minutes]
• Benchmarking [1 minute]
• QnA

Requirements

• Basic knowledge about how Neural Networks work
• Knowledge on Numpy operations is recommended
• It would help (but not mandatory) if the audience could go through this 60 minutes pytorch tutorial

Speaker bio

I am working as an AI developer in CoWrks. Here we work on a broad spectrum of use-cases in NLP and CV. I am currently focusing on NLP, particularly word representation using image, speech and predefined lexical structure. In internet, I go by hhsecond.

Links

• https://github.com/hhsecond/PyTorchDemystified
• https://github.com/hhsecond/100neuralnetworks/
• https://www.linkedin.com/in/hhsecond/

Slides

https://docs.google.com/presentation/d/17PNe19jaJNPCeer0SAmfGjS6Pk310iVc_2nm_HDSjzQ/edit?usp=sharing