Scalability

In a record-breaking feat, Dream11's app has taken fan engagement to new heights this year, managing a concurrency of 10.56 million users during the Indian Premier League (IPL). This time of the year, the Dream11 Stadium is nothing short of stepping onto the cricket field itself, and it's all thanks to the extraordinary efforts by our super talented teams of engineers, data scientists, product developers, designers, customer experience and ops experts. But the real magic happens when we dive into the minds of our #Dreamsters. Join us as we unveil their gameplan and insights leading upto India’s biggest sporting event in Indian cricket – the TATA IPL 2023. Get ready to go #BehindTheDream!

The blog covers the importance of real-time data processing in gaining a competitive advantage in various industries. It introduces Streamverse, Dream11's in-house real-time data processing platform and its core primitives: Streams and Operators, and provides a detailed overview of the platform's architecture. It also gives examples of how real-time data processing can improve user engagement, personalisation and real-time analytics, empowering a product to take business critical decisions.

With Dream11 hosting around 10,000+ matches every year on its platform, have you ever wondered what all goes behind hosting these matches? It starts from deciding the match to host, generating the credit of players (keeping the user’s perspective in mind) and taking the match live - the whole picture is much bigger. In this blog we will take you through the whole process of assigning credit to players for individual matches, what data goes behind it, what were the considerations for automating the process and how through data driven intelligence this automation was achieved. We will also discuss the benefits of doing this automation from operational and business POV.

Our platform in 2022 managed a user concurrency of 7 million. This year, concurrency reached 10.56 million during the opening match between Gujarat Titans and Chennai Super Kings—almost a 50% increase since last year. We could seamlessly execute over 308 Requests Per Minute (about five million requests per second).

Amazon Web Services (AWS) announced their next generation general-purpose Solid State Drives (SSD) volumes in December 2020, aka GP3. In the previous version GP2, high throughput and Input/Output Operations Per Second (IOPS) were required for better performance, and a large storage or IOPS volume was needed to be provisioned. This usually led to the wastage of storage resources at much higher costs.

At Dream11, we have experienced tremendous growth from just 300,000 users in 2015 to over 110 million users at present. To grow at this blazingly fast pace, we moved to a microservice architecture for developing backend systems.

For the 100 million Dream11 users, the thrill and excitement of playing fantasy sports on our platform is unparalleled. They enjoy creating their own teams and competing with fellow fans and friends! However, from a backend perspective, there are various challenges we face in terms of variation in traffic and engagement on Dream11 majorly before the match start time.

At a time when we are collectively homebound, thanks to the pandemic, the Dream11 Indian Premier League (IPL) 2020 came like a breath of fresh air for cricket fans everywhere. Not only did the **Dream11 IPL 2020** fill an otherwise gaping void in the realm of sports this year, but it also kept our passion for cricket, a thread that binds us together, burning as brightly as ever.

Scale to serve over 500,000+ events per second, 5 TB+ ingestion capacity and provide logging, search for micro services, security analytics along with anomaly detection 'Elasticsearch as a Service' for our microservices , security, data services logging and analytics needs in the face of high events frequency and data ingestion requirements_

Our Leaderboard has already served 1 million requests per minute, serving 250k concurrent users at its peak. As our user base grows, we expected the request pattern to grow by a much larger magnitude. So, we needed to design a system that would linearly scale as our traffic increases. It should be able to crunch gigabytes of data using distributed sorting within a SLA (Service Level Agreement) of under a minute, while maintaining strong consistency of different user views across multiple platforms. The persistent systems used should support millions of input/output operations per second, while maintaining throughput and latency under the strict SLA desired.This is done to ensure that our users have a seamless experience across platforms and devices at all times.