目录

Black-box coverage

DevEco Studio supports black-box testing with coverage, without the need to develop test cases. Simply push the instrumented HAP package to the device, simulate user operations on the app/atomic service, and generate a coverage report after the test is complete. Currently, only the stage model is supported.

Constraints

Preparations

Connect the device to your PC and sign the app/atomic service. For details, see Running Your App/Atomic Service on a Local Real Device and Signing Your App/Service.

Filtering Files for Coverage Testing

Since DevEco Studio 5.1.0 Release, you can customize the files be included in coverage testing. Create the coverage-filter.json5 file in the project directory and configure related fields in the file. DevEco Studio will filter files based on the configuration in this file during compilation instrumentation.

After the configuration file is modified, incremental compilation is not triggered. You need to recompile the instrumentation and perform the test again.

[h2]Fields in the coverage-filter.json5 File

The coverage-filter.json5 file supports the following fields.

Name

Mandatory/Optional

Type

Applicable Coverage Types

Description

include

Optional

String array

Black-box coverage

Instrument coverage

Local coverage

Paths of files or folders included in coverage statistics.

  • 26.0.0 and later versions: This field is optional and supports the following wildcards:
    • *: Matches any number of characters in a file name.
    • **: Matches any directory recursively.
    • ?: Matches a single character in a file name.
  • Versions earlier than 26.0.0: This field is mandatory and supports only paths starting with a module name. Wildcards are not supported.
NOTE:

The priority of include is higher than that of exclude.

exclude

Optional

String array

Black-box coverage

Instrument coverage

Local coverage

Paths of files or folders excluded from coverage statistics.

  • 26.0.0 and later versions: The field is optional and supports the same wildcards as the include field.
  • Versions earlier than 26.0.0: This field is mandatory and supports only paths starting with a module name. Wildcards are not supported.

includeHar

Optional

String array

Black-box coverage

Instrument coverage

Local coverage

Remote source HAR package included in coverage statistics. Remote bytecode HAR packages and wildcards are not supported.

This field is supported since version 26.0.0.

extraAbilities

Optional

Object array

Black-box coverage

Supplementary configuration entries for externally imported Ability classes to collect black-box coverage data. If the ability configured in module.json5 is implemented by importing an external Ability class and exported in the current file via export, black-box coverage data cannot be generated for this ability. Configure the path field under extraAbilities with the path of the external ability to generate its black-box coverage data. Relative paths are supported, but wildcards are not. See the example below.

This field is supported since version 26.0.0.

Example configuration:

// xxx.ets
import EntryAbility  from './EntryAbility';
export default EntryAbility;

// coverage-filter.json5
{
  "extraAbilities": [
    {
      "path": "entry/src/main/ets/entryability/EntryAbility.ets",   // Relative path of the imported EntryAbility
    },
  ],
}

[h2]coverage-filter.json5 File Example

// For 26.0.0 or later versions:
{
  "include":[
    "entry/src/main/ets/pages"  // Matches all files under the pages directory without wildcards.
  ],
  "exclude":[
    "**/src/**/HspTest?.ets",   // Matches files such as HspTest1.ets and HspTest2.ets in the src directory.
    "**/src/**/*Test1.*",       // Matches all files whose names contain Test1. in the src directory.
    "**/src/**/utils"    	// Matches all files in the utils folder under the src directory.
  ],
  "includeHar":[
    "math"    
  ],
  "extraAbilities": [
    {
      "path": "entry/src/main/ets/entryability/EntryAbility.ets"
    }
  ]
}
// For versions earlier than 26.0.0:
{
  "include":[
    "entry/src/main/ets/pages/aaa.ets"
  ],
  "exclude":[
    "entry/src/main/ets/pages"
  ]
}

Performing the Coverage Test

[h2]Compilation and Installation

You can compile and install the app using DevEco Studio or the CLI by following the procedure:

[h2]Performing the Test

Perform black-box testing by simulating user operations on the device. After the testing is complete, generate coverage data using either of the following methods:
  • For UIAbility and ExtensionAbility with the onDump method, execute the following command to generate coverage data. UIAbility is supported since DevEco Studio 5.1.0 Release, and ExtensionAbility is supported since DevEco Studio 6.0.1 Beta1.
    hdc shell aa dump -c -l
  • For UIAbility, exit the app to trigger the onDestroy() callback.
  • For ExtensionAbility with the onDestroy method, coverage data is generated when the ability is destroyed and its onDestroy() callback is triggered. This feature is supported since DevEco Studio 6.0.1 Beta1.
  • For UIAbility, use the EventHub API to notify the UIAbility for coverage data generation. This feature is supported since DevEco Studio 6.0.2 Beta1.
    const context = this.getUIContext().getHostContext() as common.UIAbilityContext;
    context.eventHub.emit('coverage');

Since API version 13, if the user closes an app by clearing recent tasks, the onDestroy() callback will not be executed, which results in a failure to obtain coverage data.

[h2]Generating the Coverage Report

  1. Run the following command to download the JSON files of coverage data. This command copies all files in the cache directory to the specified LocalPath directory.
    // For apps, LocalPath is optional. If it is not specified, files are stored in the directory where the command is executed by default.
    hdc file recv data/app/el2/100/base/{bundleName}/haps/{moduleName}/cache {LocalPath}
    // For atomic services, LocalPath is mandatory.
    hdc file recv -b {bundleName} ls ./data/storage/el2/base/haps/{moduleName}/cache {LocalPath}
    • LocalPath: local path for saving the downloaded files.

    In scenarios where multiple modules interact with each other through navigation, you only need to obtain the coverage report file generated for the module that is last exited. However, in special cases where there is no navigation between modules, you must obtain the coverage report files generated for each module.

  2. Generate a coverage report file.
    hvigorw collectCoverage -p projectPath={projectPath} -p reportPath={reportPath} -p coverageFile={projectPath}/{moduleName}/.test/default/intermediates/ohosTest/init_coverage.json#{LocalPath/bjc_cov_yyyyMMdd_HHmmss_SSS.json} -p patch={patchPath}
    • projectPath: project path.
    • reportPath: path for generating the coverage report file.
    • bjc_cov_yyyyMMdd_HHmmss_SSS.json: latest JSON file saved to the specified LocalPath directory. In the file name, yyyyMMdd_HHmmss_SSS represents a timestamp (year, month, day, hour, minute, second, and millisecond) when the file is generated.
    • patchPath: (optional) path to the code patch file for incremental code coverage statistics. Only absolute paths are supported, and the file suffix must be .patch or .diff. Such files can be generated using commands such as git diff. This field is supported since version 26.0.0.

    In scenarios where modules interact with each other through navigation, the value of coverageFile is created by combining the paths of all module-specific init_coverage.json files with the bjc_cov_yyyyMMdd_HHmmss_SSS.json file name using the number sign (#).

  3. Locate the report file on your local PC and open it in a web browser. For details about how coverage is calculated, see Viewing Coverage Reports.

Viewing Coverage Reports

After the coverage test is complete, two reports are generated: one in HTML format for visual report inspection, and one in JSON format (the coverageReport.json file) that records detailed coverage data. For descriptions of the fields in the file, see Interpreting the coverageReport.json File.

[h2]Interpreting the Coverage Report

The test coverage report provides insights across three key metrics:

The following is a breakdown of the specifics for the three metrics of coverage:

[h2]Interpreting the coverageReport.json File

The coverageReport.json file records detailed coverage data. The meanings of each field in the file are described below.

Before reading this section, please review Interpreting the Coverage Report to understand the related concepts and calculation methods for line, branch, and function coverage.

  1. The summary field records the overall coverage metrics for the test, including line, function, and branch coverage, as shown in the following example.
    {
      "summary": {
        "lines": {         // Line statistics
          "total": 43,     // Number of executable lines
          "covered": 12,   // Number of lines covered
          "pct": 27.91     // Line coverage percentage
        },
        "functions": {     // Function statistics
          "total": 17,     // Number of functions
          "covered": 4,    // Number of functions covered
          "pct": 23.53     // Function coverage percentage
        },
        "branches": {      // Branch statistics
          "total": 2,      // Number of branches
          "covered": 0,    // Number of branches covered
          "pct": 0         // Branch coverage percentage
        }
      },
    }
  2. files is an array that records the detailed coverage data for all files. Each element in the array corresponds to a file.
    Taking a file as an example, the meanings of its fields are as follows.
    {
      "files": [
        {
          "version": "bjc v1.0.0",   // Coverage algorithm version
          "versionCode": 10000,      // Version code for the coverage algorithm
          "path": "D:/DevEcoStudioProjects/MyApplication36/application/src/main/ets/applicationability/ApplicationAbility.ets",  // File path
          "hash": "6828362e96a78934b93db4b980fa5ad83af85a111bf187e74da89ae0c0ec613a",  // File hash
          "lineCnt": 44,     // Total number of lines in the file
          "count": 0,        // Execution count
          "projectPath": "D:/DevEcoStudioProjects/MyApplication36",     // Project path
          "functions": [],   // List of functions in the file
          "exeLine": {},     // Executable code lines
          "summary": {}      // File-level coverage summary
        },
      ]
    }