Grafana Loki: like Prometheus, but for logs
Submitted by Goutham Veeramachaneni (@gouthamve) on Thursday, 14 February 2019
Technical level: Beginner
Loki is a horizontally-scalable, highly-available log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather labels for each log stream.
Loki initially targets Kubernetes logging, using Prometheus service discovery to gather labels and metadata about log streams. By using the same index and labels as Prometheus, Loki enables you to easily switch between metrics and logs, enhancing observability and streamlining the incident response process – a workflow we have built into the latest version of Grafana.
In this talk we will discuss the motivation behind Loki, its design and architecture, and what the future holds. Its early days, but so far the response to the project has been overwhelming, with more than 5k GitHub stars and over 12hrs at the top spot on Hacker News.
We first start with the overview of logging and how current solutions don’t scale/work as expected. Then we move to the motivations that led to the creation of Loki and how it works. We discuss the current state of Loki and the roadmap and how you can start using it in your infrastructure.
We finally conclude with a demo of how metrics (Prometheus) and logs (Loki) can come together to help you debug your production issues quickly.
Goutham is a developer from India who started his journey as an infra intern at large company where he worked on deploying Prometheus. After that initial encounter, he started contributing to Prometheus and interned with CoreOS, working on Prometheus’ new storage engine. He is now a maintainer for TSDB, the engine behind Prometheus 2.0. He now works at Grafana Labs on open-source observability tools. When not hacking away, he is either on his bike, or is binge watching GCN!