Rootconf Hyderabad edition

On SRE, systems engineering and distributed systems

Tickets Propose a session

Absynthe: Artificial Behaviour Synthesiser

Submitted by N. CHATURV3DI (@chaturv3di) on Tuesday, 18 June 2019

Section: Crisp talk Technical level: Intermediate Session type: Demo

View proposal in schedule

Abstract

Absynthe came about in response to the need for test data for analysizing the performance and accuracy of log analysis algorithms. Even though plenty of real life logs are available, e.g. /var/log/ in unix-based laptops, they do not serve the purpose of test data. For that, we need to understand the core application logic that is generating these logs.

A more interesting situation arises while trying to test log analytic (and anomaly detection) solutions for distributed applications where multiple sources or modules emit their respective log messages in a single log queue or stream. This means that consecutive log lines could have originated from different, unrelated application components. Absynthe provides ground truth models to simulate such situations.

You need Absynthe if you wish test data to evaluate algorithms that model the behaviours of any well defined process – whether it’s a computer application or a business process flow.

Outline

  1. Motivation
  2. Overview of absynthe package
  3. Hands-on examples
  4. What’s next

Requirements

  1. Basic understanding of Python
  2. Basic understanding of log analysis

Speaker bio

Namit Chaturvedi is a computer science researcher, currently working at LinkedIn. He obtained his PhD in logic and automata theory in 2015 and transitioned to the world of machine learning and AI. He has previously worked on diverse projects, from load balancing on distributed systems to applying automata theory for physical access control. He has 10 publications in peer-reviewed conferences and journals; and jointly holds 2 technology patents.

His interests include outdoor sports, history of science, effect of science and technology on societies, and beer.

Links

Comments

  • Zainab Bawa (@zainabbawa) Reviewer a month ago

    Namit, this is very interesting. I have couple of questions:

    1. What is the adoption of Absynthe in the industry?
    2. Who should be the audience for this talk?
    3. Is this talk extensible to an audience that wants to learn about performance and algorithms?
  • N. CHATURV3DI (@chaturv3di) Proposer a month ago

    Hi Zainab,

    This is a new library and, as of now, I do not know of any adoption. However, the motivation behind developing this library comes from my own needs and those of the SRE teams that I’ve seen at different employers.

    The audience of this talk would be someone who is interested in applying ML/DS techniques for log analysis. There have been multiple efforts in this direction, but the problem has been that there are no easily accessible sources of labelled data.

    I didn’t quite understand your third question. What kind of performance and what kind of algorithms do you mean?

    -Namit

  • Zainab Bawa (@zainabbawa) Reviewer a month ago

    Hello Namit,

    Thanks for the detailed response. Ignore question 3, because I got what I wanted to hear in your response on who should be the audience for this talk.

    I have moved your proposal for Rootconf Hyderabad and am confirming it for this edition, given the suitability of the audience in Hyd for your talk.

    I will also send you a separate note about participating in The Fifth Elephant BOFs where your work can be very valuable.

    Looking forward to this.

Login with Twitter or Google to leave a comment