MLOps Conference

MLOps Conference

On DataOps, productionizing ML models, and running experiments at scale.

Venkata Pingali

Venkata Pingali

@pingali

Past and Future of Feature Stores

Submitted May 11, 2021

Audience Level: Intermediate
Nature: Conceptual

https://drive.google.com/file/d/1WpKUXeC0i93f72vC8yXtUR6unPEaB4ma/view?usp=sharing

Scribble has built and operate feature stores for companies for the
past few years. This is a perspective talk on why feature stores came
about, what is being built today, and what we foresee over the next
few years.

  1. Feature store introduction and history

  2. Understanding existing feature stores
    (a) Architecture: Integrated/standalone
    (b) Scale: Peta/Tera
    (c) Core abstraction: SQL-like/program
    (d) Application scope: ML/Non-ML
    (e) Programming Iterface: Tight/Open
    (f) Data classes: Streaming/Timeseries, document, transactions

  3. Classes of decisions
    (a) What, Shallow why, Deep why, Why not
    (b) How these are addressed today & gaps

  4. Feature Stores 1.0: Passive, robust, scalable
    (a) Focused on ML usecases
    (b) Focus on scale & abstractions
    (c) Passive but robust

  5. Feature Stores 2.0: Intelligent, trusted, end-to-end
    (a) Context-aware - Integrates with upstream and downstream
    About data, nature of processing, risks involved
    Changes operations, resources, observation levels
    (b) Knowledge management - Help ip creation
    Better and efficient processes
    (c) Risk management - Trust and safety as first class goal
    Reduce risks from insecure, poor/changing code & data
    Change handling, impact assessment
    (d) Proactive - Actively observes and recommends
    Suggests features, impact assessment
    (e) Scope - Expanded classes of decisions and users
    All classes of advanced data needs (shallow why…)
    (f) Distributed - Handle constraints (time, volume etc)
    Data cannot/should not flow to centralized
    Distributed discovery

  6. Some niche contexts where new classes of
    stores might emerge:
    (a) Constrained devices (handhelds)
    (b) Classes of data (geospatial)
    (c) Computational complexity (1000s of models)

  7. Key Takeaways
    (a) Feature stores are now a standard component
    (b) Understanding the journey will help future-proof your implementation
    (c) Feature stores 2.0 will be different from 1.0

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}