Crash-Free Session Benchmarks for Mobile Apps

When it comes to mobile app crashes, it's always important to look at how often they aren't happening. Unlike bugs, crashes completely end the user experience and a study shows that 80% of users will only attempt to use a problematic app three times or less before uninstalling. Users nowadays don't have much patience given the huge flood of applications available to them.

This is why knowing how often your app isn't crashing (crash-free sessions) is essential to know how users are interacting with your app and if your app needs more work. If a user downloads an app that is constantly crashing, not only will they delete it, but they're also likely to actively discourage others from using it, leaving negative reviews and app store ratings. An internal study by Google's Play Store team found that 50% of one-star reviews mentioned mobile app crashes.

You probably already know how problematic crashes are, and how difficult they are to prevent. The important question is how many crashes count as okay, good, or bad? Ideally, all apps would love to have no crashes whatsoever, but the larger the number of users an application has, the more impossible that undertaking becomes. So what crash-free session rate should you realistically target?

‍

‍

Before we can get to what's a good crash-free rate, you need to figure out what’s the best way to represent your crash-free rate. The most straightforward way to calculate a crash-free rate is:

‍

Depending on your needs, however, you can twist that formula a bit. It doesn’t make a lot of sense to factor in all the crash occurrences that happened when your old release still had some issues that you've already resolved. Depending on how often your app is being, used the formula can be changed to:

‍

Of course, the period you're looking into is up to you to set and it depends on the number of sessions your app generates. If you have less than 100 daily users, measuring your daily crash-free rate wouldn’t get you very useful results. Also, sessions are not the only variable that you should take into consideration.

Swapping out sessions for users determines the percentage of users that haven't experienced a crash. It makes sense that the user crash-free rate would be lower than the occurrences crash-free rate since the total number of sessions is much higher than the total number of users. The equation would instead be:

‍

‍

Now that you’ve found the best way to determine your app’s crash-free rate, how will it hold up compared to industry standards? Crash-free session rates have improved over the years with easier testing tools and progressively better development techniques. Our app stability data for 2023 shows that apps scoring below 99.9% crash-free sessions are much more likely to be rated below three stars on app stores and the median crash-free session rate for apps rated above three stars hovers around 99.95%.

Let’s look into what these percentages actually mean. Our equation for calculating crash-free sessions would get you the crash-free session percentage. This is the most general use case because it highlights the whole app performance in a simple calculation. How many times your app has been launched, compared to how many times your app hasn't crashed after launch.

The crash-free user percentage gives you a better indication of the user experience your app delivers. Focusing on user crashes highlights different environments for your app's performance. A certain device could be a common factor in crashes or a random occurrence with a particular user,, for example. And since a single user can have many sessions it follows that they are more likely to experience a crash.

Looking at both kinds of crash-free rates will paint a more holistic picture of your users’ app experience when it comes to troublesome crashes. And catching an uptick in crashes early on can help you identify and solve problems in your app before users react negatively.

‍

Can crash-free rates be misleading?

Crash-free rates can tell you a lot about how your app is performing, but they don't always give you the whole picture. One crash might not upset a power user since they're more likely to assume that it's a random crash and an unusual event. A new user, on the other hand, won't be as forgiving. The bigger problem occurs when a single user experiences multiple crashes, especially in a row. The frequency makes it impossible to ignore.

Additionally, crash-free rates do not take into account other fatal stability issues like Out of Memory (OOM) and Application Not Responding (ANR) errors, or less fatal ones like app hangs and forced restarts. Without a holistic view of your app's stability, you can reach your crash-free targets but still deliver an unstable and frustrating experience to your users.

So, you should always aim to have crashes very unlikely to occur--but how exactly can you do that?

‍

‍

Continuous testing

It’s essential to perform thorough testing of the entire app experience. It's easy to get lost in the development cycle and start testing feature by feature as they’re developed. However, testing the whole app experience daily is much more effective. Understanding what the end-user will be experiencing gives you important insights and exposes you to issues that won’t be seen from the developer's side.

‍

Crash reporting

More importantly, when a crash occurs, you need to detect and fix it before hearing they drag down your app's store rating. Using the right tool to track your app’s crashes goes a long way in helping you detect crashes in your app and quickly reproduce and debug them to release a fix.

‍

Release management

Having a well-structured release management and monitoring process allows you to track crashes introduced by new versions and quickly take action to prevent them from affecting more users. Using best practices like phased releases, you can ensure that any crashes that do make it into production will have a limited impact and can be quickly addressed.