The Fifth Elephant
The Fifth Elephant

The Fifth Elephant 2025 Winter Edition Call for Proposals

On AI disrupting analytics - make a submission

All submissions

Mayur Madnani

@mayurmadnani

Fine-Tuning SLM models with LoRA: Building Specialized On-Device Story Generators

Submitted Nov 10, 2025

Type of submission: Hands-on workshop (1.5 hrs to 3 hrs)

Context and Background

Generative AI has opened up new creative possibilities, but most applications rely on cloud-based models, which can introduce latency, privacy, and cost challenges. In this session, we explore how to shift from server-side story generation to efficient on-device personalization. By using LLM and SLM models, we demonstrate a hybrid approach that maintains quality while optimizing for responsiveness, adaptability, and user control.

Abstract:

Join us as we journey from the cloud to your pocket. We’ll start with cloud-based story creation and then shift to running models directly on-device, showcasing how to enhance outputs with tuning techniques. By the end, you’ll see how to measure and compare the results of each stage.

What the Session Covers:

This session dives deep into the practical use of language models for story generation, moving from a LLM model to on-device SLM models and refining them for richer output. We’ll explore the transition to on-device models that can be progressively tuned.

  • Starting with a LLM to generate initial stories.
  • Transitioning to on-device models for offline and personalized story generation.
  • Demonstrating three stages of on-device model use: the base model, a prompt-tuned version, and a fine-tuned version using adapter tuning.
  • Evaluation methods to compare and measure the improvements in story quality across these different models.

Session Highlights:

  • Practical steps for moving from API-based generation to on-device models.
  • A detailed look at how prompt tuning and fine-tuning can enhance on-device model outputs.
  • Simple evaluation metrics to assess the quality of generated stories.

Key Takeaways:

Attendees will gain insight into how to evolve from cloud-based to on-device models, how to refine model outputs through tuning, and how to evaluate the improvements effectively.

Mayur is a seasoned engineer specializing in 𝐀𝐈, 𝐝𝐚𝐭𝐚, 𝐚𝐧𝐝 𝐛𝐚𝐜𝐤𝐞𝐧𝐝 𝐬𝐲𝐬𝐭𝐞𝐦𝐬, with a proven track record of building scalable, high-performance platforms for leading organizations including 𝐉𝐢𝐨𝐇𝐨𝐭𝐬𝐭𝐚𝐫, 𝐈𝐧𝐭𝐮𝐢𝐭, 𝐖𝐚𝐥𝐦𝐚𝐫𝐭 𝐚𝐧𝐝 𝐒𝐀𝐏.

He has taken several webinars and undertaken speaking sessions. He is active on https://www.linkedin.com/in/mayurmadnani/

All submissions

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

The Fifth Elephant
The Fifth Elephant
Jumpstart better data engineering and AI futures