The Fifth Elephant 2026 Annual Conference

The Fifth Elephant 2026 Annual Conference

Built for humans. Now rebuilding for agents.

Tickets

Loading…

Vikram Nayak

Vikram Nayak

@vikramnayak

When There's No Unit Test for "Good": A Maker-Checker Loop for Subjective AI Output

Submitted Jun 25, 2026

BRIEF DESCRIPTION:

The problem: When an AI agent writes code, you can test whether the code works. But when an agent makes a chart, how do you test whether it’s any good? A chart can be technically correct and still fail to get its point across. “Good” depends on the audience and the decision they need to make - there’s no test that returns true or false.

This talk is about a different approach: two AI agents working as a pair. One makes the chart, the other reviews it, and they go back and forth - make, critique, revise - until it’s good enough. Both share the same idea of what makes a chart good; what differs is the angle they come at it from. I’ll use charts as the running example because they have a rare advantage - the audience can look at the screen and judge for themselves, live, whether the agents got it right.

The key insight: a creator agent can write code to produce a chart, but it never sees the chart its code produced. It works in code; the chart is a picture. So it misses things you’d only catch by looking - heavy gridlines fighting the data, labels printed to three decimal places, a cluttered layout, an important number left unhighlighted. It’s blind a second way too: it already knows what it meant to say, so it can’t read the result with fresh eyes the way a real audience would.

The solution: The reviewer agent is built to defeat both blind spots - it renders the chart and looks, and it reviews without being told what the maker intended, so it reacts like a real viewer. That difference in what each agent can see and know is what stops the two from just agreeing with each other.

I’ll close by demoing a maker and a reviewer agent working a real chart end to end - including a moment where one of them gets it wrong - and show how the back-and-forth becomes a signal for improving the system over time.

KEY TAKEAWAYS:

  1. A practical way to think about quality for AI outputs that have no right answer - starting with the difference between problems you can only see by looking and problems in whether the audience will understand.
  2. How to design a maker-reviewer agent pair so that giving each agent different things to see and know keeps them from rubber-stamping each other’s work.

WHO IT’S FOR:

  • Engineers and teams building AI that produces work judged by taste rather than correctness - charts, slides, documents, designs, writing.
  • Anyone wrestling with how to measure quality when “better” is partly subjective.
  • Anyone who wants to learn how to design closed loops for AI capability improvement
  • (No data-visualization background needed; charts are just the example)

BIO:

Hi, I’m Vikram Nayak, founder of ChartBoss. We help compabies benchmark and improve the capabilities of their AI systems at chart and slide generation.

Website: https://www.chartboss.ai
LinkedIn: https://www.linkedin.com/in/vikramnayak85
X: https://x.com/vikramnayak85

I’ve spent 18 years in BI and Analytics, with the last 6 of those dedicated to data visualizationn and data products.

I’ve helped companies like Trendlyne (a stock-market analytics platform with 1M+ users) and Stylumia (AI fashion intelligence), and have run data-visualization workshops at Delhivery, Stylumia, and NSRCEL @ IIM Bangalore. This talk is the engineering behind the product, not a product walkthrough.

Slides: https://docs.google.com/presentation/d/1Z0VXolOvT1JqAwwIF5BZqvk859iNGcZZ/

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Get your hybrid access ticket

Hosted by

Jumpstart better data engineering and AI futures

Supported by

Platinum Sponsor

Atlassian unleashes the potential of every team. Our agile & DevOps, IT service management and work management software helps teams organize, discuss, and compl

Platinum Sponsor

Sahaj is an artisanal technology services company crafting purpose-built AI and data-led solutions for businesses.

Gold Sponsor

Skyflow secures the flow of data across datastores, models, and agents. Enterprises turn to Skyflow as their runtime AI data control layer to protect sensitive

Bronze Sponsor

Internet infrastructure APIs for IP geolocation and more

Bronze Sponsor

Open Source Analytical Database for the AI era.