From RNN to Attention
Submitted by Sarath R Nair (@s4sarath) on Monday, 12 June 2017
The motivation and ideas behind RNN to LSTM to Attention Mechanism and wind up with latest trends , along with mathematical ideas of Backpropogation .
The idea of backpropogation remains a mystery to many people and especially to those who wants to gain some intuition about , deriving it by themselves . This talk , focuss on the basic ideas of BackProp and how one can easily extend this to RNN and LSTM . This will be followed by Attention Mechanism in LSTM and current state of the art techniques in Tensorflow .
Basic Knowledge of Deep learning , and Natural Language Processing
I am Sarath R Nair , working as a Data Scientist from past 2 years. I started exploring Deep Learning by 2014, and now I am mainly focussing on the maths and core concepts behind DL, along with practical experience in DL for NLP , basic Data Modelling and hands on experience in BigData Stack .