Don’t let that consistent launch time slip away!

Introduction

Every day on the Dream11 app, we host thousands of contests so our 110+ million fantasy sports fans can engage with their favorite sports in the palm of their hands. Since the team lineups are usually announced around 30 minutes before the real-life match begins, many of our users choose to join contests moments before the match start time. This is where the need for an optimum user experience becomes crucial for our success. For any app, every aspect of its performance is a significant factor that could boost user experience and it’s the same for the Dream11 app too! And, one important factor affecting the app’s performance and ultimately the user experience is the launch time.

Why is launch time crucial?

The first impression of an app can be a deciding factor for the user. And the very first impression that the user forms about an app, is the time it takes to launch. A high launch time exponentially decreases the user’s happiness quotient. Besides, the system may kill the app for being unresponsive for too long. Hence with an increase in the launch time, we may lose the valued user in the process and that can affect the overall business metrics.

Why is it critical for the Dream11 app?

For us at Dream11, it is no surprise that the fill rate for most of the contests on the app increases during the last 30 minutes prior to the deadline. At such a crucial time, the delayed launch of the app would result in failure to meet this deadline. If the contests are left with unfilled spots, it could lead to not only a poor experience for users but also a loss in revenue. Hence, an optimized launch time is very critical to avoid this.

Consistent Launch Time

Maintaining consistent launch time

While developing a feature, we might think that a small increase in the launch time would not affect the overall user experience. When we actually compile a release with multiple features, the collective increase in the launch time can be significantly higher. Users have a varied number of devices with different operating systems and with different network bandwidths. A small increase in launch time for one device with a given configuration might be significantly higher in a lower-end device with a lower configuration. Also, we have continuous weekly releases with new features, some of which are even developed by new developers. All of this could make maintaining a consistent launch time of the app difficult.

The Challenges

Earlier, we saw how a cumulative increase in launch time becomes a barrier to the user from joining the contests, especially near to the deadline, resulting in unfilled contests. This situation could become even more severe when the launch time is not maintained and a sudden spike occurs in a crucial release just before a big tournament (God forbid, during the IPL🤞)! Hence, we knew we had to keep an eye on the launch time and maintain its consistency.

Our trials and errors

To maintain the consistent launch time, we initially used Firebase performance monitoring (Android) and MetricKit (iOS) to capture the app’s launch time post the app release. The metrics could be accessed using the Firebase dashboard and MetricsKit in XCode.

Why did we need an upgrade over these tools?

• Firebase and MetricKit took hours to reflect the latest data
• The data was only available after the app release.

As our team size continued to grow, sometimes, a typical app release would have a bunch of new features going live. This impacted the launch time drastically, which could or could not be optimized before going live or until the next app release. Hence, there was a need for a tool that could provide launch time insights before the app was released.

Our Solution

While we tried to figure out how to tackle this, we thought about creating an automated process that could help us keep a check on the app launch time on a regular basis.

The basic workflow we came up with goes like this:

• Build the app and launch it multiple times on a simulator/emulator
• Log the launch time and derive a conclusion based on the cumulative launch time data

The Context

Measuring The App Launch Time

Let’s understand how we can measure launch time on both Android and iOS platforms.

Android launch time

When a user clicks on an app icon launcher, the app process makes an inter-process communication (IPC) call with the ActivityTaskManagerService.startActivity(). Then, the system server process sends a request to the Zygote process, to fork a new process for the app. Before creating a new process for the app, the system server process creates a window that will be visible to the user until the app draws its first frame. This duration is called the ‘app launch time’.

We can learn more about the Android launch time from the official docs here.

iOS launch time

There are two phases of launch time in iOS — the pre-main and the post-main.

The pre-main

The dynamic linker (DYLD) statistics can really help in capturing the pre-main time. This is a time-measurement that starts when the app’s main method is called, till it gets control from the operating system (OS).

We can get the pre-main time from Xcode. The DYLD has a built-in environment variable to get this measurement.

Once the variable is set, we get the pre-main time after running the app in Xcode’s console. The log looks something like the below image:

The post-main

Post-main is the time taken by an app from UIApplicationMain to the first screen presented to the user. In short, it is the time taken from AppDelegate/Main to the first frame that the app renders.

The actual launch time is the sum of the pre-main and the post-main.

The Implementation

Now that we know how we could measure the app launch time, let’s talk about the solution we built to create the above described automated process of measuring the launch time.

Tooling & Instrumentation

Let us take a look at each step of the automated process and understand the usage of various tools we used to achieve this:

Automation

We used fastlane and Jenkins to automate the entire process:

• We created a job on Jenkins that includes various configurations i.e. app launch count, git branch, device configurations, etc.
• We configured the job to trigger periodically at regular intervals
• We used fastlane to run our custom scripts for the workflow by creating separate lanes.

This automated trigger would kickstart the process of building the app once and launching it multiple times on configured devices to collect statistically significant data.

Collecting Data

Once the job was triggered, Fastlane would run our scripts that are specific to both platforms. Let’s talk about these separately.

Android

We integrated Firebase performance monitoring tools in our Android app which gave us the data we needed to track the launch time.

Let us go step-by-step on how we retrieved this data:

• We used a Kotlin script that installed the app on the emulator, launched it multiple times, and captured the logs exposed by Firebase SDK

• We stored these logs in a given log path directory.

iOS

We wrote a similar script for launching the simulator, installing and running the app on it. Once the app was launched, we wrote the time taken until our first view loads into a temporary file. Again, we did this multiple times until we collected a statistically significant amount of data.

Let us take a close look at the process, step-by-step:

• Launched the iOS simulator from the Xcode directory
• Built the app using fastlane’s xcodebuild command
• Installed the app in the simulator using xcrun

• The following code would calculate the time the CPU took to launch the app and present the first screen to the user

• To get the best results, we ran this code on the first call of `applicationDidBecomeActive(_: UIApplication)`, and stored this time in a separate file.

Data Analysis

To collect significant data we used these scripts to launch the app multiple times and gather statistical data. This raw data needed to be processed further to help us derive a conclusion.

To do so, we derived the following data points from the collective data:

• Minimum
• Maximum
• Average
• Median

We analyzed these data points for each job result and made sure the launch time was maintained properly.

Maintaining Historical Metrics

To make sure that the launch time was in the acceptable range, we stored all the launch time data on a remote database and queried it via Kibana.

Additionally, we posted all these data points to a Slack channel, to get instant alerts when the job was finished.

Process changes

We changed our release process to only merge the features that passed the acceptable threshold of the launch time.

Since the releases were very frequent, it was important to reveal the launch time effects of a feature we built well before the release. Hence, any improvements if necessary could be made without affecting the timelines.

The Impact

The major impact of our solution was that the launch time was consistent and stayed within the acceptable range which was decided prior to the release.

This was only possible because the launch time data was available to us even before the release. This also helped us in identifying any anomalies and rectify quickly while keeping both user experience and timelines intact.

For a data-obsessed team like us, data is everything. Having the launch time data available prior to release increased our release confidence and decreased the overall turnaround time of measuring the launch time for a particular release.

Our Next Steps

We have definitely solved our initial problem of maintaining launch time before the release. However, there are some areas which could be further improved.

Current challenges

The major problem with our current system is that we manually look at the available data and decide if the launch time falls under the acceptable range. This introduces a chance of human error. We want this to be an automated process and remove the human component as much as possible.

Proposed solutions

Since we have a dedicated Pull Request for each feature, it would make sense to create an alert system for each PR which would get triggered on each commit and run the job. If the launch time for the PR is out of the acceptable range, it should give us an alert, preferably on Slack.

These improvements should reduce the turnaround time significantly and help us automate most of the tasks.

Conclusion

Building the launch-time automation tool has helped us to smoothen the development experience while making our own process much more efficient. This also helped us get the app launch time insights early on so we can work on the optimizations if needed. The best part of this process is that these insights are available to us before releasing the app! We are able to provide the best user experience every time for every user, which helps us to gain more user adoption on Dream11!

Want to help us solve our current challenges with faster, easier and better solutions? Join us at Dream11 by applying here!