The Fifth Elephant 2014

Intent is to highlight benefits gained by using Clojure, a functional language which works on JVM and Cascalog data processing library for Hadoop. The participants will be exposed to,

• Rationale behind Cascalog

• Basic abstractions of this library

• Use cases where a high level functional language and a declarative programming library helped immensely

• Short code snippets showing extraction of interesting customer behavioural patterns using Cascalog and Clojure

• Highlight how functional abstractions might help bridge the gap between lab culture of data science and so called production environments

Outline

Machine learning practitioners often spend 70%-80% of their time on cleaning, transforming and scrubbing data. Quick iterations and ability to create large number of features out of raw data are pre-requisites for any serious machine learning activity. There is also a large gap between a skill sets and objectives of data scientists and programmers. Data scientists need to slice and dice through real world big data. Implementing all the low level plumbing of multiple, dependent Hadoop jobs is a serious impediment to this goal. On the other hand, companies cannot afford to have lab tools running, as they have large impedance mismatch with production environments.
We have seen Cascalog library and functional abstractions of Clojure providing a sweet spot. It lets data scientists focus on their job of understanding data without worrying about complicated class hierarchies or un-intuitive domain specific languages. On the other hand, Cascalog and Clojure seamlessly integrate with JVM based stack. The result is a fun and productive way to process data at scale!

Speaker bio

Harshad Saykhedkar is senior data scientist at Sokrati, a digital advertising startup based out of Pune. He has experience of applying machine learning to problems in advertising, banking and telecom sectors for past four years. He has used multiple tools (R, Python, SAS) in this journey and lately fallen in love with the Clojure ecosystem, hacking his way to create data processing tools at Sokrati. Harshad holds a master’s degree in Operations Research from Indian Institute of Technology, Mumbai.

Links

• Linkedin profile (http://in.linkedin.com/pub/harshad-saykhedkar/17/569/b99)
• Link to machine learning workshop, conducted as a part of run up event for Fifth Elephant in Mumbai (https://hasgeek.tv/fifthelephant/the-fifth-elephant-2014-run-up-mumbai/943-introduction-to-machine-learning-workshop)