The Fifth Elephant 2023 Monsoon

Background

In 2015, I sold the intellectual property (IP) of my Silicon Valley company, Perceptive Code LLC, to Mercedes Benz. Subsequently, I was tasked with meeting certain milestones as part of the handover process. I chose to complete this task in India, where I aways wanted to be. I successfully downsized our research model, initially consuming 6GB of GPU memory, to a mere 300KB of weights. I then implemented this model on a 2W FPGA with 112 DSP blocks. This model is now included as part of the MBUX option in all Mercedes cars. Using cameras installed in the car, it enables gesture control for various functions such as turning on passenger side lamps when a hand is extended to retrieve something, or choosing which rearview mirror to adjust simply by looking at it, among other features.

What will this talk cover?

This talk will be a personal reflection on the past decade of my life. I’ll discuss my journey from leaving my job at Yahoo! in Bangalore and moving to Italy, then Germany for my PhD. I will share how I didn’t secure the jobs I initially aimed for post-PhD and subsequently “settled” for a post-doctoral position with Chris Bregler at the Courant Institute at NYU. Intriguingly, I unintentionally found myself working with Yann LeCun, which led to the publication of four papers alongside this Turing Award winner.

I’ll touch upon the pre-AlexNet era and provide insights into the academic environment at NYU during that time. I’ll also delve into my experience with Apple’s self-driving team and discuss my decision to leave Apple for a riskier venture with Mercedes, striving to meet milestones and deliver a product. I’ll discuss how, in hindsight, this all unfolded.

Additionally, I plan to share my experiences in academia. Despite having offers and spending time at prestigious institutions like IIT Bombay and IISc, I made the choice not to pursue academia full-time. Instead, I continued to explore new ventures such as UAVIO Labs and Fastcode AI.

Talk Outline

• My PhD journey: Machine Learning for Computer Graphics, spending time at Weta, earning credits for Adventures of Tintin, 2011
• The jobs I desired but didn’t secure after my PhD
• My tenure at NYU: people at NYU at that time, working with Yann, working on Theano, and projects I undertook
• How and when Jonathan and I started Perceptive Code LLC
• Life after my post-doc: why I chose Apple
• My conversations with Mercedes and the ensuing deal, coming back to India
• The efforts behind building the product: technology, papers, and patents produced
• My current aspirations and what I look forward to