The Fifth Elephant 2025 Annual Conference CfP

Abstract:
Real Life photography is controlled by light, a messed up lighting can ruin a high quality Camera photograph. The situation is similar with AI generated Images, specially when we are doing background swap or inpainting, where foreground is coming from a different light source and background is having a different one, the Images look pasty which affects realism.
In real life photography we fix lighting by manipulating camera angles while shooting, but in AI Images it’s all about post processing, which boils down to Photoshop based approaches. But for the business who are building end to end AI Image delivery platforms using Background Swaps, Inpainting, Photoshop is an unusable step, so it boils down to an automated process that fixes the light !
In this talk we will talk about a custom algorithm which we developed at Caimera to fix the light of an Image without a big latency and no loss to any details !

Agenda:

• What is Re-Lighting ? and why is there a need ?
  • We will first talk about the impact of lighting in AI Images by showing some AI generated Images with a comparison between correct and messed up lighting.
  • Discussion on how lighting impacts specially on in-painted images in terms of realism and over all likeness. This will prove the impact of correct lighting in AI generated Images.
• Current Approaches a comparative study - Noise is the enemy
  • We will compare results of Caimera developed custom relighting algorithms against current community and closed source approaches like IC-Light, LBM Lighting.
  • Demo on comparative analysis on results
  • Discussion on how diffusion based approaches adds noise to re-create lighting in an Image, and how it impacts and changes details inside an Image.
• High Level overview of Caimera’s Relighting Algorithm:
  • Discussion on our non-noise based relighting algorithm that detects lighting information from the background of any Image by using Cromptons and Bling Phong Calculation.
  • How we are able to detect Shadow information, Light Direction, Color Information and other Light features from the Image - Mathematical overview.
• Impact of Light Information maps on Relighting:
  • We use Depth, Normal, Specular, Displacement and Color Albedo maps to represent light information.
  • Discussion on how this map works and how its representation matters.
  • Mathematical overview of how we used this map in calculation of light features from above point.
  • Small Demo on how each of these maps impact.
• Light Overlaying process and Final hypothesis:
  • We follow principles of Physics, a mixture principle of wave optics and Newton’s Corpuscular theory where Light is considered as a Beta particle. We will talk about how using the extracted maps and features are able to create a Light mask that shows how the light will be travelling across the object which we are trying to relight.
  • Then we will discuss how using that light mask which is coming from considering light as a beta particle will interact on the surface. Here we take principles from wave optics and using the calculated shadow values and other features we get a clear picture of light’s behaviour of reflection, refraction etc.
  • Final overlay (small code demonstration)
  • Comparative Analysis on results
• Discussion on how to improve this model for better representation of light features and overlay - Light on training
• Conclusion
• QnA

Takeaways for audience:

• Understanding of how to create realistic Images and how lighting and other features impacts
• Understanding a new custom algorithm , inspiration to build something new and similar

Target Audience :

• Machine Learning Engineers, ML Practitioners
• Businesses, startup founders
• Product Managers
• Students learning about Artificial Intelligence.

About the Speaker:

MLE at Caimera AI, Former MLE at Newton School, Dark Horse, Shell, WRI, Metvy. Contributed to Google Tensorflow(GSOC), Samsung(Prism), IIT Patna (Projects). Founded MBK Health tech backed by supreme ventures to apply AI for early detection of cardiac diseases and create a hyper local support network for patients using wearables. Holding 4 patents on medical Imaging automations using AI algorithms,holding multiple research papers and Indian young Achievers award winner for contributions in artificial intelligence towards nation. Spoken at Py-Bangalore, Belgium-Py conference, Keras Community Day 23, Girlscript India Summit, MIT TECH X , HPAIR(delegate) etc and multiple meetups, hackathons and events.
(Linkedin: https://www.linkedin.com/in/anustupmukherjee/)