The Fifth Elephant
The Fifth Elephant

The Fifth Elephant 2025 Winter Edition Call for Proposals

On AI disrupting analytics - make a submission

All submissions

Manish Malik

How are we building AI Agents for Fraud Detection?

Submitted Nov 10, 2025

Type of submission: 30 mins talk

At Razorpay, we process billions of transactions. This scale brings a massive, evolving threat from fraud—from phishing scams to money laundering. In this talk, we’ll start by showcasing the ML capabilities Razorpay uses to prevent platform abuse by bad actors posing as legitimate merchants. When our models flag a merchant, the real work begins: conducting a thorough risk review. Traditionally, this has been a manual, time-intensive process. We’ll also explore how we’re leveraging Generative AI, specifically AI agents, to automate risk reviews and audits.
We’ll cover

  • An overview of supervised and unsupervised ML models that continuously assess merchants across demographic, social media, behavioral, transactional, and other metadata dimensions.

  • Our learnings from building AI agents to carry out risk investigations, starting with a ReAct architecture that showed limited success, and why we pivoted to a custom-built agent. We’ll cover the technical challenges of this, from fighting hallucinations to the difficulty of teaching an AI to follow complex, multi-step SOPs.

You’ll walk away with:

  • A Blueprint for a Modern Risk System: We’ll show how we utilize diverse data points and ML techniques to continuously assess risk on Razorpay platform.
  • Architecting Agents for Complex Work: A technical walkthrough of our journey from ReAct to a custom architecture, and the lessons learned that led to better accuracy.
  • The Hard-Won Lessons: How we’re tackling agent hallucinations, managing overconfidence, and the surprisingly difficult challenge of making an AI follow a ~100-page SOP.
  • The Human-in-the-Loop: A practical look at where AI agents excel in automation and where human expertise remains critical for risk management.

Speaker:
Manish Malik is a Staff Analytics Specialist at Razorpay. He has nearly 10 years of experience in Data Science and Machine Learning. Manish is responsible for building multiple risk and fraud detection models at Razorpay.

All submissions

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

The Fifth Elephant
The Fifth Elephant
Jumpstart better data engineering and AI futures