The Fifth Elephant 2017

With AI research and machine learning systems growing at great speed, companies require significant effort to keep up or risk losing their relevance in this brave new world. The new tide also brings with it numerous tools to tackle previously intractable problems. However, there does seem to exist a gulf between appreciating these developments and subsequently deploying them. Despite the global push to democratize machine learning, the steps prescribed don’t align with the fuzzier problems that need solving.

As a startup focused on organizing the world’s ecommerce data, Semantics3 has faced its fair share of challenges. To tackle numerous problems covering categorization, feature extraction, cross-domain product matching and price tracking, we have had to incorporate multiple modern techniques into our workflows.

Going over our experiences, I would like to share the broader questions (not whether you need a CNN, GAN or TROL) that need to be considered. Datasets, frameworks and deployment practices - are just some of the topics I wish to touch upon. The talk is almost a recollection of our journey, when moving machine learning from practice to production in an ecommerce-centric environment.

Outline

Introduction [2-3 mins]

A short introduction to the topics that are to be covered. Additional context about the machine learning problems in the domain of ecommerce data will be presented. This section sets the stage for discussing the machine learning “pipeline” to be built for solving the various problems.

Let there be data [10 mins]

Having high-fidelity datasets is crucial when starting on any machine learning problem. Balanced classes, representative coverage of ground truth, and adversarial examples, all need to be considered before jumping in with the modelling.

Topics covered:

• Starting from Day 0
• Garbage In, Garbage Out
• Organizing datasets
• Preprocessing playbooks

Your first model [15 mins]

Once the datasets have been prepared, the actual “fun” can begin - experimentation is the name of the game. This section will be present an overview of the landscape together with processes that help weigh available options.

Topics covered:

• Problem solving by pattern recognition
• Simplifying and starting simple
• Model calibration
• Launch then iterate

Integration [10 mins]

The 99.8% accurate model is not of much use, unless it can be integrated into systems meant to solve the actual problem. By constantly keeping these integration goals within sight, the aim is to convey the point that early and good enough is often better than late and perfect.

Topics covered:

• Plumbing pipelines
• Versioning & Testing
• Production Skew
• Monitoring

Conclusion [5 mins]

While pushing the state-of-the-art in machine learning might sound interesting, it is still important to maintain focus on its applicability within desired domains. Together with an overview of the points covered, a few concluding remarks will be presented.

Requirements

Aimed at machine learning enthusiasts who wish to get started with applying machine learning techniques across various domains.

Speaker bio

I am a member of the data science team at Semantics3 - building data-powered software for ecommerce-focused companies. Over the years, I have had the chance to dabble in various fields covering data processing, pipeline setup, database management and data science. When not picking locks, or scuba diving, I usually blog about my technical adventures at our team’s engineering blog.

Links

• This talk is based on an earlier blogpost (https://engineering.semantics3.com/moving-machine-learning-from-practice-to-production-9c462eeef9fa) and follows discussion started from the front page of HN: https://news.ycombinator.com/item?id=12954825
• My talk on “Adventures in Postgres management”, Rootconf 2017 - https://rootconf.talkfunnel.com/2017/1-adventures-in-postgres-management
• My other technical articles - https://engineering.semantics3.com/@ramananb
• LinkedIn - https://www.linkedin.com/in/ramananbalakrishnan
• Related article by Martin Zinkevich, “Rules of Machine Learning, Best Practices for Machine Learning” - http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
• Related article by Sculley et al., “What’s your ML Test Score? A rubric for ML production systems” - https://www.eecs.tufts.edu/~dsculley/papers/ml_test_score.pdf

Slides

https://docs.google.com/presentation/d/17AQ0boRinVt1HhTUiufYVz7SE_Q1SRu5AUW6EI_bwbs/pub