Jatin Mittal– Apr 20, 2022

Introduction

A/B testing is a crucial part of our production process at TapNation. This is what takes your KPIs to the next level and makes your game truly ready for a global launch.

What’s more, we’re very proud of our rigorous standards when it comes to A/B testing. And, with over 500 million downloads as of March 2022, we like to think it’s a process that works.

So, to give you a flavour of how we can help, here’s a rough guide to A/B testing at TapNation.

What is A/B Testing and Why do we do it?

A/B testing means carrying out an experiment on two versions of your game— an ‘A’ variant and a ‘B’ variant.

It’s important that we run A/B tests on slightly different versions of your game so we can understand which elements are generating revenue and which aren’t. This way, we can make changes before launch and give your game the best possible chance of becoming a hit.

Generally, A/B tests only lead to small tweaks. The core gameplay is set, and we’ll instead be focusing on things like user interface, colours, in-game advertising and other ways to maximise the LTV or ‘lifetime value’ of your game. LTV is the average revenue that a customer of your game will generate throughout their lifespan as a customer.

Our A/B Test Process

Our A/B test process generally lasts 1-3 months and involves the following steps:

Prototyping: in this phase, we’ll be testing your game’s CPI or ‘cost per install’, i.e. the average amount we have to invest to get an installation of your game. This usually involves testing elements such as camera angles, colours or environments.

‘Make or break’ phase: based on the available data, this is where we’ll choose to continue iterating your game or end the project.

Production: Here we’ll continue to iterate for 1-2 months with the aim of reaching 30% of D1 retention (this is the number of unique customers who return to your game the day after installing it.)

Final test: This is where we’ll look to introduce plug ads and carry out a final profitability test.

Publishing: If all the previous steps have been completed successfully, we’ll publish your game and enter our live ops cycle, in which we’ll try to grow and monetise your player base while maintaining and improving the standards of your game.

Boosting LTV through Gameplay

As we mentioned earlier, the aim of A/B testing is to maximise the LTV or ‘lifetime value’ of your game. Within gameplay, there are lots of different ways to do this:

• Gradually boosting level difficulty as the player progresses, e.g. using obstacles, bonus levels and boss levels.
• Unlocking new features and content as the player progresses.
• Changing camera angles to reinforce player engagement.
• Introducing special levels through rewarded watching.
• Including a skins or shop element to the game.

And the list goes on. The possibilities are plentiful, the only thing to keep in mind being to serve the LTV with these new changes, which again means testing every idea on its own.

Boosting LTV with Advertising

You can also boost LTV through testing ad monetisation techniques. These are different methods of monetising your game through advertising revenue. Here’s a few ways we might do this:

• Testing the integration of new ad networks, e.g. by adding a new bidder.
• Adding more aggressive floor prices.
• Reducing the length of your advertising waterfall.
• Using an ad mediation platform to test various advertising metrics.
• A/B testing new in-game ads, audio-only ads and ad frequency.

Different kinds of In-game Advertising

In-game advertising is probably the best monetisation model available to hypercasual games. For this reason, it’s one of the most important things we’ll A/B test during our process. It’s worth quickly exploring what kinds of in-game ads your game could include:

Rewarded video ads: These are generally very effective and a great way to let prospective players know what your gameplay mechanics are like before they download.

Interstitial ads: Full-screen ads that cover the entirety of a game’s interface. Generally displayed at pauses or points of transition in a game, they can generate positive results if they don’t interrupt gameplay.

Banner ads: Small banners displayed at the very bottom of the user’s screen. They can be shown constantly during a session and regularly refreshed, acting as complimentary revenue to interstitial and rewarded videos.

In-game ads: Ads that merge into game design without interrupting gameplay, for example a billboard that the player passes by in the game itself. These generally generate a good level of complimentary revenue and aren’t as intrusive as other forms of in-game ads.

In-app purchases: This means encouraging players to buy additional levels, powerups or currency.

They can all benefit your game in different ways, A/B testing will help you understand which one is better in which cases.

Our Testing Platform

As part of our A/B tests, we always give access to our own testing platform to our studios, giving you genuine transparency across our process. We also offer remote setup for A/B tests and live access to a wide range of useful testing metrics, helping you become part of the process and understand exactly what’s working and what isn’t.

A quick case study: Giant Rush

To get a better idea of how our A/B testing process can make a positive impact on your game, look no further than Giant Rush, one of our biggest hits to date.

In Giant Rush, you navigate a giant through a series of obstacle courses, with bonus levels and boss fights along the way. By making subtle tweaks to the game, we managed to make significant improvements to the game’s retention:

• Introducing a slow-motion animation when a player smashes through a wall boosted D1 retention by 4%.
• A new ‘special big boss’ mechanic improved D1 retention by 2% and D3 by 4%.
• Moving the gauge multiplier resulted in a 14% increase in RV watched.

Giant Rush now boasts roughly 125 million downloads and generated roughly $12 million in revenue during its first 6 months. Through our A/B testing process, we also managed to boost LTV up to 100% in 30 days, which is yet another proof of the incredibly huge positive it can bring to your game.

If you have any further questions about our A/B testing process, feel free to email us or get in touch via our social channels!