The Fifth Elephant 2016

I have a firm belief that,

• there’s elegant and understandable theory behind neural networks.
• the best way to learn about any ML algorithm is to implement one.

The objectives of the workshop are based on this belief, as follows,

• Help the audience understand the basics of neural networks in intuitive fashion.
• Separate fundamental principles from gory details.
• Understand taxonomy of deep learning (explained below in mind map)
• Learn about neural networks by building them from scratch.

The audience can expect to take away the following after attending the workshop,

• Enough understanding of neural network to make scary looking books or papers
  in this area less scary.
• Enough understanding of neural network to call BS on over-hyped headlines and
  dissect grandiose claims on their utility.
• Detailed understanding of few deep learning algorithms and architectures by
  implementing them from scratch.
• For studying further, which areas to focus on.

The question of whether neural nets are the answer to the question of how brains work, the best known
way of doing Artificial Intelligence or just the current fad to be exploited by the cynical as a new form of
intellectual snake oil, merits serious investigation. The writers tend to be partisan and the evidence
confusing. We shall investigate the need for a mid-life crisis in this chapter.
--- page 251, Michael Alder, An Introduction to Pattern Recognition

Outline

History and Philosophical Stuff

Rather than thinking of neural networks as a document handed down from top of a
mountain by gods, we look at the history of development of neural networks. We will
go through questions like what problems were they meant to solve,
critical milestones in the development etc. This hopefully will reveal some connections
(pun intended!) with other models in machine learning.

Tour of Basics

The audience is expected to know certain basics of machine learning (supervised vs.
unsupervised models, gradient descent algorithm, cost functions). We will do a very
fast revision of these topics.

Neural Network Implementation

We will implement simple single hidden layer neural network for classification in this
part. The implementation will be done using with using simple for loops and basic maths
as far as possible.

Calling Apples from Oranges

When I started reading about neural network, jargon or terminologies always overwhelmed me.
I couldn’t understand the difference between convolutional neural networks and layer wise
pre-training, nor could I decide if it even made sense to talk about the two together.
In this part of workshop, we will dissect neural networks (and jargon) from various angles: by
architectures, by optimization strategies, by application domains.

Implementation Part 2

In this section, depending upon time we will implement 1 or 2 ideas from the so called deep learning
neural networks. Again, the idea will be to build everything from scratch and shun the use of libraries.
We could cover few of the following,

• convolutional neural nets
• recurrent nets or long short term memory networks
• word vector embeddings / word2vec
• layer wise pre-training / unsupervised pre-training, optimizations

Closing Remarks

This will be open interaction and question answer section.

Requirements

Knowledge Requirements

This is not a beginner workshop. I expect the participants to know the
following at the least. More the better!

• Basics of machine learning (supervised Vs. un-supervised)
• Understand gradient descent algorithm, cost functions.
• Quick review of high school calculus.

Programming Knowledge Requirements

• You should know basic programming
  • Reading and Writing files
  • Flow controls (if-else)
  • Looping constructs like for loop, while
  • Variable assignment
• In other words, you should have programmed at least few hundred lines
  in any mainstream programming language.
• The implementation choice for this workshop will be Python.

Computing Requirements

• Laptop (operating system of your choice), charged battery + charger.
• Python installed on the laptop + IDE of your choice including console.
  • No hard choice between python 2 Vs python 3.
• Numpy and scipy stack installed for python.
  • Make sure the scipy stack for appropriate python version is installed.
• It won’t be possible to provide installation support at the time of workshop.
  So all requirements should be pre-installed.

Installation instructions

• Python
  • It comes pre-installed on most Linux machines.
  • Overall procedure is documented on python website.
• Scipy Stack
  • Scipy installation is covered in detail here

Data Sets

The links to the datasets to be used during the workshop will be posted here about 2-3 weeks before workshop.

Speaker bio

I am heading the data science and analytics team at sokrati.com, an advertising technology startup based out of Pune. I’ve spent almost 6 years in applying machine learning to various domains (advertising, banking, telecom). I have conducted a number of hands on workshops and talks on various topics in machine learning for audience of data science beginners. I am proficient in use of R and Python for the data science domain and have some hands on experience with Clojure. I also serve as a data science mentor at Sprinboard.com, an education startup, primarily focussed on data science education.

Links

• Link to the mind map of this proposed workshop is here.(http://www.slideshare.net/HarshadSaykhedkar/neural-networks-mind-map)
• Link to my previous workshop on machine learning basics is here.(https://hasgeek.tv/fifthelephant/2014-workshops/959-real-world-machine-learning)