Highway Networks and ResNet : A deeper approach towards Deep Learning .
Submitted by Vasudev Singh (@vasu-dev) on Saturday, 10 June 2017
Section: Full talk Technical level: Intermediate
Deep Learning though termed so but as the network becomes deeper the neural networks are more difficult to train and their preformance also start to degrade. Residual learning framework(ResNet and Highway Networks) is an Newer kind of architecture which ease the training of networks that are substantially deeper than those used previously, helps overcome the degradation problem and lets the network decide how deep it needs to be and also give significant boost to the performance of task in hand. The layers are reformulated as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. The Architecture won the 2015 ImageNet Challenge along with ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
What is deep Learning ??
Various Deep Learning Architectures
Residual Learning ??
- What is residual Learning ?
- Problem with deeper architectures
- Benefits over simple deep architectures.
- Highway Networks
- Other variants of highway Network
- What are Resnets ?
- Why Resnets are better.
- Some maths behind resnets.
Implementation of ResNet using keras and TF
Participants will need a laptop with VirtualBox installed (if not running directly on your platform), with the following libraries installed:
Tensorflow or Theano
Numpy and Scipy stack
Participants should have understanding of basic deep learning architecture, introductory calculus and some hands on experience on using Keras or Tensorflow or any other deep learning library.
Vasudev Singh is currently working as a research intern at IIIT Delhi, supervised by Dr. Anubha Gupta and is currently pursuing his B.Tech in Computer Science from Delhi Technological University (Formerly DCE).