The Fifth Elephant 2012

Finding the elephant in the data.

Siddhartha Reddy


Build Your Own Search Engine

Submitted Jun 15, 2012

No, this is not another tutorial on using Solr/ElasticSearch/Sphinx/Lucene. Imagine that none of these existed and you need a search engine for your shiny new eCommerce startup. What would you do? Build your own search engine, of course. I’ll go over what it would take to do that.


I plan to cover:

  1. The basic idea of a Search Engine
    — 1.1 How is it different from a Database?
    — 1.2 Inverted Index

  2. Techniques for constructing an Inverted Index
    — 2.1 Distributed Indexing
    — 2.2 Dynamic Indexing
    — 2.3 Index Compression

  3. Scoring and Ranking

  4. Scaling Search

Speaker bio

Siddhartha leads the Search & Product Discovery Team at Flipkart — India’s largest catalog of online inventory. Prior to this he was at a startup called Ziva Software, working on Mobile Search and automatic Information Extraction from the Web.

Siddhartha has been working on Information Retrieval and in related areas for almost seven years. He has a deep love for all things Search — everything ranging from User Experience to Relevance Ranking to Scaling Search to Search as a Platform.

Siddhartha loves to work with large scale, highly available systems and tons of data. He takes particular interest in application of Machine Learning techniques in all kinds of applications.


{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

All about data science and machine learning