Neural Stack: Augmenting Recurrent Neural Networks with Memory
Recurrent neural networks (RNNs) offer a compelling tool for processing natural language input in a straightforward sequential manner. Though they suffer from various limitations which do not allow them to easily model even the simple transduction tasks. In this talk, we will discuss new memory-based recurrent networks that implement continuously differentiable analogues of traditional data structures such as Stacks, Queues, and DeQues.
- Language modelling
- Machine Translation
Introduction to RNN [5 mins]
- RNN architecture
Limitations of a simple RNN model[8 mins]
RNN Equivalence to Finite State Machine[10 mins]
- Stack API
Neural stack architechture[10 mins]
- Neural stack architecture
Closing remarks [3 mins]
- To faclitate better understanding of the talk, I will be giving a github repo as a take away so that the audiance can go back, download the code and play with it.
Satyam Saxena is a ML researcher at Freshdesk. An IIT alumnus, his interest lie in NLP, Machine Learning, Deep Learning. Prior to this, he was a part of ML group Cisco. He was a visiting researcher at Vision Labs in IIIT Hyd where he used computer vision and deep learning to build applications to assisting visually impaired people. He presented some of this work at ICAT 2014, Turkey. https://www.linkedin.com/in/sam-iitj/
- Code associated with this talk: https://github.com/sam-iitj/NeuralStack