Rootconf Mini 2024

Abstract

Monitoring and observability tools are a cornerstone in debugging processing for any large organization.

The current state of log query languages and interfaces in popular log analysis tools presents significant usability challenges that hinder productivity, such as a lack of standardized query languages, steep learning curves, and high complexity.

Large Language Models, while promising for log analysis, parsing, and summarization, haven’t quite been explored for log search.

This talk covers our study of creating the first text-to-logql dataset, fine-tuning LLMs for LogQL, Grafana’s Log Query Language; and evaluating various approaches to using LLMs to write LogQL queries.

We have been working on a paper on the same topic and are excited to share our learnings with fellow systems nerds!

Key Takeaways

• Various challenges to using LLMs to solve log search problems.
• Evaluation and performance of various fine-tuned models and approaches for LogQL.
• How to build a dataset for a domain-specific language from scratch!
• How this fits in with a bigger goal of making a semantic search engine for observability data.

Audience

• Anybody who has written a LogQL/DatadogQL/SPL query recently and used or thought about using LLMs for it.
• Systems & Infra engineers interested in LLMs for observability.