Service Introduction-A/B Testing

A/B Testing utilizes scientific experimental data to help you optimize your app experience and improve key conversion and growth indicators. You can use the service to create one or more A/B tests engaging different audiences to compare your variants of app UI design, copywriting, product functions, or marketing activities for performance indicators and find the best one that meets user requirements. This helps you make correct decisions.

Currently, you can use A/B Testing to run these experiments:

Notifications experiments: Test variants of the title, content, and post-tapping action of a push message.
Remote configuration experiments: Test variants of your new app UI design, function, or parameter design.
App messaging experiments: Test variants of the title, body, background image, and buttons of an in-app message.

Key Functions

Expand

Function	Description	Supported App
Improving product use experience through Remote Configuration	In a remote configuration experiment, you can test parameters across various groups to identify the parameter settings that can better address your needs.	Android app iOS app HarmonyOS app Web app
Attracting users through push notifications	In a notifications experiment, you can test various notification variants of your notification copywriting and message layout to find the optimal variant.	Android app
Enhancing user loyalty through in-app messages	In an app messaging experiment, you can test various in-app message variants to find the better message style and layout.	Android app iOS app HarmonyOS app

Function

Description

Supported App

Improving product use experience through Remote Configuration

In a remote configuration experiment, you can test parameters across various groups to identify the parameter settings that can better address your needs.

Android app
iOS app
HarmonyOS app
Web app

Attracting users through push notifications

In a notifications experiment, you can test various notification variants of your notification copywriting and message layout to find the optimal variant.

Android app

Enhancing user loyalty through in-app messages

In an app messaging experiment, you can test various in-app message variants to find the better message style and layout.

Android app
iOS app
HarmonyOS app

How the Service Works

Before you start an A/B test, segment your users to find out audiences for the test, based on the version, language, and other filter conditions such as user attributes and audiences collected by Prediction and HUAWEI Analytics. A notifications experiment, an app messaging experiment, or a remote configuration experiment is available to you to divide target users into different groups by proportion, and you can configure different policies for them. By analyzing user data indicators of various groups collected by HUAWEI Analytics, such as, retention, interaction duration, and click-through rate, you can reach a final solution that better meets user requirements.

Click to enlarge

Platform Support

Expand

Platform	Supported	Supported Experiment Types
Android		Notifications Remote configuration App messaging
iOS		Remote configuration App messaging
HarmonyOS		Remote configuration App messaging
Web		Remote configuration
Quick app
Server SDK	N/A	N/A
REST API	N/A	N/A

Implementation Process

Expand

No.	Step	Details
1	Formulate the A/B Testing scheme	Before performing an A/B test, analyze your service, make a testing goal, and specify the testing input.
2	Enable A/B Testing	Enable A/B Testing and enable services on which A/B Testing depends, such as Push Kit, HUAWEI Analytics, and Prediction.
3	Create an experiment	After creating a remote configuration experiment, an app messaging experiment, or a notifications experiment, you can: Define one or more treatment groups. Specify the experiment user group, for example, group of users of a specific app version. Define experiment targets, for example, the user retention rate, which will be measured for treatment groups in the experiment. Select other indicators (such as the user interaction duration and click-through rate) to better understand the experiment results and their impact on your app's user experience.
4	Manage the experiment	After an experiment is created, you can manage it as follows: Test the experiment. Start the experiment. View the experiment report. Increase the percentage of users who participate in the experiment. Release the experiment. Perform other experiment management operations.

Pricing

Free of charge.