Vikram Nayak

@vikramnayak

When There's No Unit Test for "Good": A Maker-Checker Loop for Subjective AI Output

Submitted Jun 25, 2026

BRIEF DESCRIPTION:

The problem: When an AI agent writes code, you can test whether the code works. But when an agent makes a chart, how do you test whether it’s any good? A chart can be technically correct and still fail to get its point across. “Good” depends on the audience and the decision they need to make - there’s no test that returns true or false.

This talk is about a different approach: two AI agents working as a pair. One makes the chart, the other reviews it, and they go back and forth - make, critique, revise - until it’s good enough. Both share the same idea of what makes a chart good; what differs is the angle they come at it from. I’ll use charts as the running example because they have a rare advantage - the audience can look at the screen and judge for themselves, live, whether the agents got it right.

The key insight: a creator agent can write code to produce a chart, but it never sees the chart its code produced. It works in code; the chart is a picture. So it misses things you’d only catch by looking - heavy gridlines fighting the data, labels printed to three decimal places, a cluttered layout, an important number left unhighlighted. It’s blind a second way too: it already knows what it meant to say, so it can’t read the result with fresh eyes the way a real audience would.

The solution: The reviewer agent is built to defeat both blind spots - it renders the chart and looks, and it reviews without being told what the maker intended, so it reacts like a real viewer. That difference in what each agent can see and know is what stops the two from just agreeing with each other.

I’ll close by demoing a maker and a reviewer agent working a real chart end to end - including a moment where one of them gets it wrong - and show how the back-and-forth becomes a signal for improving the system over time.

KEY TAKEAWAYS:

  1. A practical way to think about quality for AI outputs that have no right answer - starting with the difference between problems you can only see by looking and problems in whether the audience will understand.
  2. How to design a maker-reviewer agent pair so that giving each agent different things to see and know keeps them from rubber-stamping each other’s work.

WHO IT’S FOR:

  • Engineers and teams building AI that produces work judged by taste rather than correctness - charts, slides, documents, designs, writing.
  • Anyone wrestling with how to measure quality when “better” is partly subjective.
  • Anyone who wants to learn how to design closed loops for AI capability improvement
  • (No data-visualization background needed; charts are just the example)

BIO:

Hi, I’m Vikram Nayak, founder of ChartBoss. We build AI systems for visual analytical communication. I’ve spent 18 years in BI and Analytics, with the last 6 of those dedicated to data visualizationn and data products.

I’ve helped companies like Trendlyne (a stock-market analytics platform with 1M+ users) and Stylumia (AI fashion intelligence), and have run data-visualization workshops at Delhivery, Stylumia, and NSRCEL @ IIM Bangalore. This talk is the engineering behind the product, not a product walkthrough.

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

Jumpstart better data engineering and AI futures