Building our very own Data Highway — Dream11’s In-House Analytics Vs Multiple Analytics Platforms

Published on

The Dream11 app hosts over 150 million sports fans, with 308 million RPM (about five million requests per second), and 10.56 million user concurrency. At this scale, it’s all about crunching data to get insights and offer a personalised experience. Dream11 collects over billions of events and Terabytes of data per day

In today’s world when one has access to multiple platforms to solve data-related business requirements, is it possible for a company to solely rely on its in-house analytics tool?

Many would quip ‘It’s unimaginable!’. With each tool fighting to corroborate its unique functionality and stay relevant, companies usually lean towards outsourcing data tools.  However, there is a flip side.

At Dream11, our cultural tenets defined as ‘DO-PUT’ guide us and are the foundation of the company. Read Data-first, Ownership, Perseverance, User-First and Transparency.

In this article, you will see all of these tenets come alive. Be it putting in extra hours to ensure our data is protected, taking ownership to build our very own in-house analytics tool, or dealing with challenges that have come our way during this process. At Dream11, we go the extra mile to ensure the data security and efficiency of our systems. Read more below.

So let’s start with the question that got the ball rolling:  Why do we need to build our very own Inhouse Analytics tool?  

Every start-up reaches a certain inflection point when your data multiples by the second and it is extremely difficult to manage such magnitudes of data inflow. Dream11 went through this phase too; hence, we returned to the drawing board and reflected on the drawbacks of operating company data across multiple tools. That’s how the idea to build our very own ‘Inhouse Analytics’ system was born.

The genesis of ‘Inhouse Analytics’ was based on three factors.

  1. More control over our data
  2. More flexibility to meet our internal requirements at a faster pace
  3. Use the data to get intelligent insights at our discretion, without relying on outsourced tools

The implementation of ‘Inhouse Analytics’ comes with its own set of advantages and challenges.

Initially, we used third party tools to capture interaction data and Redshift to store transaction-led data. Inhouse Analytics enables gathering user interaction events, as well as transaction data on a single platform. This helps us map user actions to a specific transaction, revise the definition of metrics, build user interaction-based audience profiles to send out a promotion, and the list goes on!

Dream11 Data Platform Architecture