arrow_back Building Spark as Service in Cloud using YARN
On building a cloud-based black-box predictive modeling system
Submitted by Bargava Subramanian (@barsubra) on Thursday, 21 May 2015
Data Analytics platforms, with predictive models at their core, are the buzzword in Enterprise Analytics. Having been on both sides - a consultant providing analytics and a consumer of analytics, I’ve realized that there are few, if any, runaway winners. Rightly so. It is one of the hottest growth areas. This talk would go over some of the ingredients to building a successful data analytics platform
UI/UX is definitely a centerpiece to a successful analytics platform. But what needs to get in there? (There is no single correct answer to this) What are the data science components? What is the impact on the design/architecture of the data science components when data scales?
It is easy to build a machine learning model. But what does it take to build a state-of-art, or even a reasonably good, model ? Is there a secret sauce? When data scales, what are the trade-offs to consider? How far can one go when expert domain knowledge is not available in-house ?
The talk would try to answer those above questions, along with the constraints various choices impose when creating the platform.
On the modeling front, there will be emphasis on the following: Feature engineering, modeling selection, emsembling, importance of bias-variance and generalization.
An interest in Analytics.
Just to draw some distinction between full-fledged apps(Eg: e-commerce apps) and data platforms : Data platforms are primarily meant for data analysts/data scientists/business owners to recommend/make better decisions. Mostly, they solve just one business problem(and well!). The goal is to plug-and-play the enterprise data into them and get insights/recommendations. Some other jargons for them include: APIs for machine intelligence, Productizing Analytics
Bargava Subramanian is a Senior Statistician(Data Scientist) at Cisco Systems, India. He has a Masters from University of Maryland, College Park, USA.