The Fifth Elephant winter edition 2019
Winter edition of India's most renowned conference on big data and data science
The Fifth Elephant is rated as India’s best conference on big data, data science and application of data to real-life use cases.
It is a conference for practitioners, by practitioners. The Fifth Elephant completed its seventh edition in Bangalore, on 26 and 27 July 2018. The Bangalore edition caters to data and ML engineers, architects, technologists, data scientists, product managers, researchers and business decision-makers.
Hosted by
The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more
Venkata Pingali
@pingali
Reducing Cost of Production AI: Feature Engineering Case Study
Submitted Oct 1, 2018
Section:
Crisp talk
Technical level:
Intermediate
The number and complexity of datasets, usecases, and models are rapidly growing. However, the number of ML/AI applications in production are growing much more slowly. AI in production is suffering from a multiple challenges that vary by domain. We focus on a common activity - machine learning feature engineering involving textual data. It accounts for 40-80% of time and contributes significantly to the cost of ML applications using business data. We do not address other aspects of AI.
We identify the cost dimensions in feature engineering for business data and share ways to reduce the cost of this step.
The methods have been tested every day for more than a year. We enabled customer modeling in production across multiple deployments that covered 2M+ people, consumed 800GB of data, and computed upto 500 features for each person.
This approach draws upon the experience with building a less flexible and more expensive solution approach in 2016 using Hive and Pandas. It serves as an imperfect baseline, and our current approach is conservatively 3x improvement over the baseline.
Outline
- Feature Engineering Overview
- Typical Feature Engineering Cycle
- Trends
- Detailed Cost Drivers
- Examples: Reconciliation & auditing, change management
- Indicative Quantitative Improvement
- Detailed discussion of each driver
Requirements
Have an ML system in production or plan to have one.
Speaker bio
Dr. Venkata Pingali is Co-Founder and CEO of Scribble Data, an ML Engineering company based in Bangalore and Denver. Scribble’s flagship enterprise product, Enrich, accelerates ML product development in enterprises. Before starting Scribble Data, Dr. Pingali was VP of Analytics at a political data consulting firm. He has a BTech from IIT Mumbai and a PhD from USC in Computer Science.
Links
Slides
https://drive.google.com/open?id=1aKZMWMn_t9A5RSv8wkU-zOgcs2tpIsX4
Comments
Hosted by
The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}