目录

Using ImageEffect to Edit Images

When to Use

The ImageEffect class provides a series of APIs for editing images. You can use the ImageEffect APIs to process images of different input types, such as Pixelmap, NativeWindow, NativeBuffer, and Uri, to obtain filter effects.

Common development scenarios for ImageEffect are as follows:

Available APIs

For details about the APIs, see ImageEffect.

How to Develop

Adding Dynamic Link Libraries

Add the following libraries to CMakeLists.txt.

target_link_libraries(entry PUBLIC
    libace_ndk.z.so
    libimage_effect.so
    libpixelmap.so
    libnative_window.so
    libnative_buffer.so
)

Add the corresponding dynamic link library based on the processed image type: Pixelmap (libpixelmap.so), NativeWindow (libnative_window.so), or NativeBuffer (libnative_buffer.so).

Adding Header Files

#include <multimedia/image_effect/image_effect.h>
#include <multimedia/image_effect/image_effect_filter.h>
#include <multimedia/image_effect/image_effect_errors.h>

[h2]Making an Image Effect Take Effect by Using ImageEffect APIs

  1. Create an ImageEffect instance.

    OH_ImageEffect *imageEffect = OH_ImageEffect_Create("imageEdit");

  2. Add an EffectFilter filter.

    OH_EffectFilter *filter = OH_ImageEffect_AddFilter(imageEffect, filterName);
CHECK_AND_RETURN_RET_LOG(filter != nullptr, filter, "OH_ImageEffect_AddFilter fail!");

  3. Set the data to be processed.

    Scenario 1: Set the OH_PixelmapNative input type.

    For details about how to use OH_PixelmapNative, see PixelMap Data Processing (C/C++).

    ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetInputPixelmap(imageEffectPtr.get(), pixelmapNativePtr.get());
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetInputPixelMap fail! errorCode = %{public}d", errorCode);

// Set an output pixel map (optional). If this API is not called, the filter effect directly takes effect on the input pixel map.
errorCode = OH_ImageEffect_SetOutputPixelmap(imageEffectPtr.get(), pixelmapNativePtr.get());
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetOutputPixelmap fail!");

    Scenario 2: Set the OH_NativeBuffer input type.

    For details about how to use OH_NativeBuffer, see Native Buffer Development (C/C++).

    // Set an input native buffer.
ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetInputNativeBuffer(imageEffect, inputNativeBuffer);
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetInputNativeBuffer fail!");

// Set an output native buffer (optional). If this API is not called, the filter effect directly takes effect on the input native buffer.
errorCode = OH_ImageEffect_SetOutputNativeBuffer(imageEffect, outputNativeBuffer);
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetOutputNativeBuffer fail!");

    Scenario 3: Set the URI input type.

    // Set an input URI.
ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetInputUri(imageEffectPtr.get(), path.c_str());
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetInputPixelMap fail! errorCode = %{public}d", errorCode);

// Set an output URI (optional). If this API is not called, the filter effect directly takes effect on the input URI.
errorCode = OH_ImageEffect_SetOutputUri(imageEffect, path.c_str());
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_SetOutputUri fail!");

    Scenario 4: Set the texture input type.

    The texture input scenario is a high-performance scenario that uses the hardware GPU for rendering. In this scenario, the application needs to provide a valid OpenGL context and set parameters and perform rendering operations in the correct environment.

    // Set the input texture ID.
errorCode = OH_ImageEffect_SetInputTextureId(imageEffect, inputTex, ColorSpaceName::SRGB);
if (errorCode != ImageEffect_ErrorCode::EFFECT_SUCCESS) {
    OH_LOG_ERROR(LOG_APP, "OH_ImageEffect_SetInputTextureId fail!");
    return;
}

// Set the output texture ID. Note that it cannot be the same texture as the input. Otherwise, rendering exceptions may occur.
errorCode = OH_ImageEffect_SetOutputTextureId(imageEffect, outputTex);
if (errorCode != ImageEffect_ErrorCode::EFFECT_SUCCESS) {
    OH_LOG_ERROR(LOG_APP, "OH_ImageEffect_SetOutputTextureId fail!");
    return;
}

    Scenario 5: Set the OHNativeWindow input type.

    The following uses camera preview as an example to describe the OHNativeWindow input scenario. The SurfaceId provided by the XComponent for camera preview streams can be converted into OHNativeWindow at the native C++ layer. A code example is provided below.

    For details about how to use the XComponent module, see XComponent.

    For details about how to use the native window module, see OHNativeWindow.

    For details about how to use the camera, see Camera Preview (C/C++).

    (1) Add an XComponent to xxx.ets.

    XComponent({
   id: 'xcomponentId',
   type: 'surface',
   controller: this.mXComponentController,
   libraryname: 'entry'
})
.onLoad(() => {
   // Obtain the SurfaceId of the XComponent.
   this.mSurfaceId = this.mXComponentController.getXComponentSurfaceId()

   // Call the native API to obtain the input SurfaceId.
   this.mSurfaceId = imageEffect.getSurfaceId(this.mSurfaceId)

   // Call the camera API to start preview and transfer the obtained input SurfaceId to the camera framework.
   // ...
})
.width('100%')
.height('100%')

    (2) The following shows how imageEffect.getSurfaceId is implemented at the native C++ layer.

    // Create a NativeWindow based on the SurfaceId. Note that the created NativeWindow must be released by calling OH_NativeWindow_DestroyNativeWindow when it is no longer needed.
uint64_t outputSurfaceId;
std::istrstream iss(outputSurfaceIdStr);
iss >> outputSurfaceId;
OHNativeWindow *outputNativeWindow = nullptr;
int32_t res = OH_NativeWindow_CreateNativeWindowFromSurfaceId(outputSurfaceId, &outputNativeWindow);
if (res != 0) {
    OH_LOG_ERROR(LOG_APP, "OH_NativeWindow_CreateNativeWindowFromSurfaceId fail!");
    return;
}

// Set the output display surface.
ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetOutputSurface(imageEffect, outputNativeWindow);
if (errorCode != ImageEffect_ErrorCode::EFFECT_SUCCESS) {
    OH_LOG_ERROR(LOG_APP, "OH_ImageEffect_SetOutputSurface fail!");
    return;
}
// Obtain the input surface. Note that the obtained inputNativeWindow must be released by calling OH_NativeWindow_DestroyNativeWindow when it is no longer needed.
OHNativeWindow *inputNativeWindow = nullptr;
errorCode = OH_ImageEffect_GetInputSurface(imageEffect, &inputNativeWindow);
if (errorCode != ImageEffect_ErrorCode::EFFECT_SUCCESS) {
    OH_LOG_ERROR(LOG_APP, "OH_ImageEffect_GetInputSurface fail!");
    return;
}

// Obtain the SurfaceId from the input NativeWindow.
uint64_t inputSurfaceId = 0;
res = OH_NativeWindow_GetSurfaceId(inputNativeWindow, &inputSurfaceId);
if (res != 0) {
    OH_LOG_ERROR(LOG_APP, "OH_NativeWindow_GetSurfaceId fail!");
    return;
}

// Convert the SurfaceId into a string and return it.
std::string inputSurfaceIdStr = std::to_string(inputSurfaceId);

  4. Start the image effector.

    errorCode = OH_ImageEffect_Start(imageEffectPtr.get());
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_ImageEffect_Start fail! errorCode = %{public}d", errorCode);

  5. Stop the image effect (optional, available only in the input surface scenario).

    // Stop the image effector.
ImageEffect_ErrorCode errorCode = OH_ImageEffect_Stop(imageEffect);
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result, "OH_ImageEffect_Stop fail!");

  6. Serialize the image effector (optional).

    // Serialize the image effect (optional).
char *imageinfo = nullptr;
errorCode = OH_ImageEffect_Save(imageEffect, &imageinfo);
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result, "OH_ImageEffect_Save fail!");

  7. Destroy the image effector instance.

    ImageEffect_ErrorCode errorCode = OH_ImageEffect_Release(imageEffect);
CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Release fail!");

[h2]Customizing a Filter

The following steps describe how to implement and register a custom filter:

  1. Define ImageEffect_FilterDelegate.

    // Image information struct.
struct EffectBufferInfo {
    void *addr = nullptr;
    int32_t width = 0;
    int32_t height = 0;
    int32_t rowSize = 0;
    ImageEffect_Format format = ImageEffect_Format::EFFECT_PIXEL_FORMAT_UNKNOWN;
};
    // Implement a custom filter.
delegateBrightness = {
    .setValue =
        [](OH_EffectFilter *filter, const char *key, const ImageEffect_Any *value) {
            // Verify the parameters. If the verification is successful, true is returned. Otherwise, false is returned.
            if (value->dataValue.floatValue >= DATA_VALUE_MIN && value->dataValue.floatValue <= DATA_VALUE_MAX) {
                return true;
            } else {
                return false;
            }
        },
    .render = [](OH_EffectFilter *filter, OH_EffectBufferInfo *info,
                 OH_EffectFilterDelegate_PushData pushData) { return RenderBrightness(filter, info, pushData); },
    .save = SaveFilterBrightness,
    .restore = [](const char *info) -> OH_EffectFilter * {
        // Create an OH_EffectFilter instance. In this example, "CustomBrightness" is the name of the custom filter.
        OH_EffectFilter *filter = OH_EffectFilter_Create("CustomBrightness");
        CHECK_AND_RETURN_RET_LOG(filter != nullptr, nullptr, "OH_EffectFilter_Create fail!");
        // Parse the JSON string info to obtain the key and value.
        std::map<std::string, std::string> parsedJson = Json::Parse(info);
        if (parsedJson.find("values") != parsedJson.end()) {
            std::string valuesStr = parsedJson["values"];
            std::map<std::string, std::string> valuesJson = Json::Parse(valuesStr);
            if (valuesJson.find("brightness") != valuesJson.end()) {
                float brightness =
                    std::stof(valuesJson["brightness"].substr(1, valuesJson["brightness"].size() - 2));
                ImageEffect_Any value;
                value.dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT;
                value.dataValue.floatValue = brightness;
                // Set a filter parameter. value is the parameter parsed from info as JSON.
                LOG_E("brightness value: %{public}f ", value.dataValue.floatValue);
                ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, "brightness", &value);
                CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, nullptr,
                                         "OH_EffectFilter_SetValue fail!");
            }
        }
        return filter;
    }};

    The Render API can be implemented in the following two scenarios.

    Scenario 1: The custom algorithm can directly modify the pixel data in info (for example, the brightness adjustment filter).

    bool RenderBrightness(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
{
    // Obtain image details.
    EffectBufferInfo inputBufferInfo;
    OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
    OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
    OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
    OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
    OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);

    // Call the custom filter algorithm.
    ApplyCustomBrightnessAlgo(filter, inputBufferInfo);

    // After editing is complete, call pushData to directly pass the original image.
    pushData(filter, info);
    return true;
}

    Scenario 2: The custom algorithm cannot directly modify the pixel data in info (for example, the crop filter).

    bool RenderCrop(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
{
    // Obtain image details.
    EffectBufferInfo inputBufferInfo;
    OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
    OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
    OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
    OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
    OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);

    // Create output pixel information.
    EffectBufferInfo outputBufferInfo;

    // Call the custom filter algorithm.
    ApplyCustomCropAlgo(filter, inputBufferInfo, outputBufferInfo);

    // Generate outputOhInfo.
    OH_EffectBufferInfo *outputOhInfo = OH_EffectBufferInfo_Create();
    OH_EffectBufferInfo_SetAddr(outputOhInfo, outputBufferInfo.addr);
    OH_EffectBufferInfo_SetWidth(outputOhInfo, outputBufferInfo.width);
    OH_EffectBufferInfo_SetHeight(outputOhInfo, outputBufferInfo.height);
    OH_EffectBufferInfo_SetRowSize(outputOhInfo, outputBufferInfo.rowSize);
    OH_EffectBufferInfo_SetEffectFormat(outputOhInfo, outputBufferInfo.format);

    // After editing is complete, call pushData to pass outputOhInfo.
    pushData(filter, outputOhInfo);

    // Release resources.
    OH_EffectBufferInfo_Release(outputOhInfo);

    return true;
}

  2. Generate custom filter information.

    // Create an OH_EffectFilterInfo instance.
OH_EffectFilterInfo *customFilterInfo = OH_EffectFilterInfo_Create();
CHECK_AND_RETURN_RET_LOG(customFilterInfo != nullptr, result, "OH_EffectFilterInfo_Create fail!");
// Set the custom filter name.
OH_EffectFilterInfo_SetFilterName(customFilterInfo, "CustomBrightness");
// Set the memory types supported by the custom filter.
ImageEffect_BufferType bufferTypeArray[] = {ImageEffect_BufferType::EFFECT_BUFFER_TYPE_PIXEL};
OH_EffectFilterInfo_SetSupportedBufferTypes(customFilterInfo,
    sizeof(bufferTypeArray) / sizeof(ImageEffect_BufferType), bufferTypeArray);
// Set the pixel formats supported by the custom filter.
ImageEffect_Format formatArray[] = {ImageEffect_Format::EFFECT_PIXEL_FORMAT_RGBA8888};
OH_EffectFilterInfo_SetSupportedFormats(customFilterInfo,
    sizeof(formatArray) / sizeof(ImageEffect_Format), formatArray);

  3. Register ImageEffect_FilterDelegate with the image effector.

    ImageEffect_ErrorCode errorCode = OH_EffectFilter_Register(customFilterInfo, &delegateBrightness);
CHECK_AND_RETURN_RET_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result,
                         "OH_EffectFilter_Register fail! errorCode = %{public}d", errorCode);

[h2]Using EffectFilter to Quickly Implement the Processing Effect of a Single Filter

  1. Create a filter.

    // Create a filter. For example, create a contrast effector.
OH_EffectFilter *filter = OH_EffectFilter_Create(filterName);
CHECK_AND_RETURN_LOG(filter != nullptr, "OH_EffectFilter_Create fail!");

  2. Set a filter parameter.

    // Set the filter parameters, using the input parameter as the filter intensity.
ImageEffect_Any value;
value.dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT;
value.dataValue.floatValue = static_cast<float>(filterValue);
ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value);
CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_SetValue fail!");

  3. Make the filter take effect.

    // Make the filter take effect.
errorCode = OH_EffectFilter_Render(filter, inputPixelmap, outputPixelmap);
CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_Render fail!");

  4. Destroy the filter instance.

    // Destroy the filter instance.
errorCode = OH_EffectFilter_Release(filter);
CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_Release fail!");

[h2]Querying Capabilities