The Fifth Elephant 2018

King - Man + Woman = Queen

The most famous example of word vectors paint an optimistic picture where computers can represent word into vectors which can be used to infer similarity. But can we extend it to sentences or to documents? How did word vectors come into existence? What are its utilities?

Though most of the people use Mikolov et al’s Word2Vec as a blackbox and train by:

import gensim
sentences = [[‘content’, ‘of’, ‘first’, ‘sentence’], [‘content’, ‘of’, ‘second’, ‘sentence’], ... , [‘content’, ‘of’, ‘nth’, ‘sentence’]]
model = gensim.models.Word2Vec(sentences)

And never know what is cooking inside the hood. This talk would cover a very basic implementation of Word2Vec, a small tutorial of how to use Gensim to train your own word vectors. Building on this we would build vector representation of sentences. We would meanwhile learn about the novelty of classical and deep learning techniques.
After learning all this we would explore the application of Word Movers’ Distance for Information retrieval.
Though these terms sound new, but this talk would build from the very basics(arrays as vectors) and myself being a programmer, along with Deep Learning enthusiast, would focus more on a progammer’s perspective.

Outline

Coverage of the workshop:

Introduction to NLP -> 10 minutes
Tokenization, Stemming and Lemmatization -> 25 minutes (10-15 minutes of Hands on session)
Brief intro of POS and NER -> 10 minutes
Word Embeddings (Theory and Motivation) -> 10 minutes
Word Embeddings (Hands on)
Basic Implementation -> 15 minutes
Gensim based Implementation (Meanwhile explaining the possible use cases) -> 15 minutes
Introduction to Deep Learning and exciting stuff for future (10 minutes)
Small intoduction to PyTorch/Keras (Hands on) (10 minutes)
Basics about one-hot encoding, and explaining the hello world of neural networks (10 minutes)
Learning word embeddings and a glimpse of Transfer Learning (Hands on) (20 minutes)
Introduction to Sentence embedding (10 minutes)
Word Movers’ Distance for Information Retrieval (15 minutes)
Remaining as buffer time

Speaker bio

The speaker is a fourth year undergraduate student at IIT Kharagpur. A robotics and deep learning enthusiast, he spends his time writing blogs about Artificial Intelligence where he was a top writer till June 2017 or teaching humanoid robots to walk and kick at the KRSSG Lab otherwise maintaining the college wiki.

He was a GSoC mentor for SymEngine/SymPy in 2017 where he was a GSoC student in 2016. Furthermore, he has worked on Cross Lingual Word embeddings at UFAL Prague, generating pattern of birds’ songs at ETH Zurich and hierarchical embeddings at Stony Brooks NYC.