Unleashing Efficiency: The Power of Mobile App Automation at Dream11

INTRODUCTION

As the world’s largest fantasy sports company, we have been at the forefront of experimentation, pioneering several industry-first models. Among several innovations, our mobile app automation framework is particularly noteworthy. With a user-first approach, this framework enables faster implementation of innovations, is cost-efficient and has made infrastructure redundancy and shorter regression cycles possible. At Dream11, the model has been modified to enable  implementation of server mocking, allowing seamless testing of various scenarios. Additionally, the framework enables auto-scale automation, providing the flexibility to scale infrastructure as needed, reducing the testing time from days to just an hour. Dream11 has also revolutionized backdoor testing, ensuring comprehensive testing coverage across the app.

Given this context, let’s dive deep into the world of automation at Dream11.

DREAM11’S PLATFORMS

People love high-quality apps. But imagine running an app seamlessly across millions of mobile devices, ranging from a simple Android smartphone supported by a 2G or 4G network, to the latest iOS device.

Currently, the Dream11 app runs on two platforms i.e. Android and iOS. These platforms are powered by React-Native and Native Frameworks.

CODEBASE AND CONTRIBUTIONS

As a user-first company, every team at Dream11 collaborates with the common objective of delivering world-class experiences. To achieve this, we use GitHub as a source code management tool. All committed code goes through meticulous runtime evaluations and pull request checks before production.

Huge volumes of commits and PR’s are merged daily on the main branch responsible for running checks at regular intervals.

THE CHALLENGES: TIME-CONSUMING, UNSCALABLE AND EXTENDED REGRESSION CYCLES

Implementing even minor modifications to the app required extensive testing efforts across both Android and iOS platforms. Regression testing involved developers from every team coming together to perform comprehensive tests over a period of multiple days. There is an additional complexity in managing experiments and feature exposure flags.

THE NEED FOR AUTOMATION

To expedite application delivery, minimize manual efforts, and enhance overall quality, we embarked on developing our automation framework. During this process, we used tools like Appium with Java, TestNG as the test runner, and Maven as the build tool.

Our initial automation tool was primarily focussed on automating critical test cases and user journeys within test environments. However, as we scaled our automation suite, we came across  multiple challenges in our existing framework. The common hurdles we faced were the current team structures, issues with data setup for automation, environment & its maintenance, and in-sprint delivery and automation tasks.

Due to the above-mentioned constraints, regression automation was always a “catch-up” game. Initially, our setup involved deploying Appium with Java for our automation framework. We ran the regression suite on a third-party cloud service provider with parallel threads. However, this setup posed several challenges.

• Expensive and difficult to scale the automation framework as our team grew and new features were added at speed.
• Several limitations to horizontal scale infrastructure provided by cloud services. Running our automation suite was also challenging because of application and architecture changes.
• Finally, running a subset of regression test cases (around 1/5th of the total cases) took approximately a day on a single platform, causing delays in the feedback cycle for critical tests and slowing down our regression phase. We revisited our entire process and identified ways to remove several operational, technical and infrastructure dependencies.
• Mocking Edge Service: We implemented an approach to mock our edge service that powers both our platforms. Following this, we rolled out a service that served mock responses and routed endpoints to production if the data was unavailable in the mock responses. This reduced our dependency on the edge service and allowed teams to write automation for high and medium-priority test cases, increasing our automation test coverage to nearly 100%
• Built in-house service execution infrastructure: Instead of relying on limited parallel execution infrastructure provided by a third-party cloud service, we created our in-house service. We spawned multiple dedicated hosts for each team, with emulators and simulators to execute tests in parallel
• Test Case Automation: We automated all test cases of priority types using mocks and the pseudoservice endpoints. This helped us understand any new feature and its corresponding test cases, resulting in faster feedback and increased test coverage
• Massive Regression Effort Reduction: By automating a majority of the regression tests and running them, we were able to significantly reduce the regression effort to just an hour

Overall, these changes have improved the efficiency and effectiveness of our automation framework, allowing us to increase team efficiency, manage increased test coverage, and drastically reduce the time required for regression testing.

FUTURE PLANS

Moving forward, we will continue to enhance our automation framework to accommodate various aspects of mobile automation. We will focus on implementing user and clickstream event validations for frontend clients. Also, proactively monitoring and detecting app crashes before they impact production is something we’re actively looking into. And finally, capturing crucial interaction performance metrics. These initiatives will strengthen the overall effectiveness and reliability of our mobile automation efforts.