Anthill Inside 2019

A conference on AI and Deep Learning

Up next

Attention based sequence to sequence models for natural language processing

MG

Madhu Gopinathan

@mg123

Workshop details including schedule, venue, date and tickets are published here: https://hasgeek.com/anthillinside/sequence-to-sequence-models-workshop/

Ilya Sutskever and others introduced sequence to sequence learning with neural networks. Subsequently, Bahdanau and others introduced “attention”, similar to the human ability to focus with high resolution on a certain part, to improve the performance of sequence to sequence models in machine translation. Later, Vaswani and others introduced the transformer model which is built entirely on the idea of “self-attention”. These ideas have proved to be very useful in practice for building powerful natural language processing models (https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html). In this hands on workshop using PyTorch, we will learn to build natural language processing models using these concepts.

Outline

  1. Introduction to sequence models
  2. Why sequence to sequence models? Build a sequence to sequence model on sample data.
  3. What is attention? Enhance the model and understand the value of attention
  4. Transformer architecture: sequence to sequence modeling using self-attention
  5. Build a transformer model on sample data

Requirements

Laptop

Speaker bio

Madhu Gopinathan is currently Vice President, Data Science at MakeMyTrip (MMT), India’s leading online travel company. At MakeMyTrip, he led the development of natural language processing models for Myra, MMT’s task bot for customer service (https://economictimes.indiatimes.com/jobs/rise-of-the-machines-when-bots-take-over-the-workplace/articleshow/66930068.cms).
Madhu holds a PhD in computer science from Indian Institute of Science, on mathematical modelling of software systems,and an MS in computer science from the University of Florida, Gainesville, USA.. He has collaborated with researchers at Microsoft Research, General Motors and Indian Institute of Science leading to publications in prominent computer science conferences.
He has extensive experience developing large scale systems using machine learning & natural language processing and has been granted multiple US patents.

Links

Comments

Up next

Accountable Behavioural Change Detection (VEDAR) using Machine Learning

SA

Srinivasa Rao Aravilli

With exponential increase in the availability of telemetry / streaming / real-time data, understanding contextual behavior changes is a vital functionality in order to deliver unrivalled customer experience and build high performance and high availability systems. Real-time behavior change detection finds a use case in number of domains such as social networks, network traffic monitoring, ad exchange metrics etc. In streaming data, behavior change is an implausible observation that does not fit in with the distribution of rest of the data. A timely and precise revelation of such behavior changes can give us substantial information about the system in critical situations which can be a driving factor for vital decisions. Detecting behavior changes in streaming fashion is a difficult task as the system needs to process high speed real-time data and continuously learn from data along with detecting anomalies in a single pass of data. In this talk, we introduce a novel algorithm called Accountable Behavior Change Detection (VEDAR) which can detect and elucidate the behavior changes in real-time and operates in a fashion similar to human perception. We have bench marked our algorithm on open source anomaly detection datasets. We have bench marked our algorithm by comparing its performance on open source anomaly datasets against industry standard algorithms like Numenta HTM and Twitter AdVec (SH-ESD). Our algorithm outperforms above mentioned algorithms for behaviour change detection, efficacy
  • Apr 16, 2019
  • 3 minutes

Read more