目录

GPU Spatial Upscaling

XEngine Kit provides the GPU spatial upscaling feature. Using single-frame input images, it performs super-sampling based on spatial neighborhood information, delivering low overhead with substantial gains. The recommended upscaling factor is [1.2, 1.5].

Constraints

API Description

The following lists OpenGL ES and Vulkan APIs for setting GPU spatial upscaling. For details about more setting and query APIs, please refer to API Reference.

API Description
const GLubyte * HMS_XEG_GetString (GLenum name) Queries XEngine OpenGL ES extensions.
GL_APICALL void GL_APIENTRY HMS_XEG_SpatialUpscaleParameter (GLenum pname, GLvoid *param) Sets the input parameters for GPU spatial upscaling.
GL_APICALL void GL_APIENTRY HMS_XEG_RenderSpatialUpscale (GLuint inputTexture) Executes the rendering command for GPU spatial upscaling.
VKAPI_ATTR VkResult VKAPI_CALL HMS_XEG_EnumerateDeviceExtensionProperties (VkPhysicalDevice physicalDevice, uint32_t *pPropertyCount, XEG_ExtensionProperties *pProperties) Queries XEngine Vulkan extensions.
VKAPI_ATTR VkResult VKAPI_CALL HMS_XEG_CreateSpatialUpscale (VkDevice device, const XEG_SpatialUpscaleCreateInfo *pXegSpatialUpscaleCreateInfo, XEG_SpatialUpscale *pXegSpatialUpscale) Creates an XEG_SpatialUpscale object.
VKAPI_ATTR void VKAPI_CALL HMS_XEG_CmdRenderSpatialUpscale (VkCommandBuffer commandBuffer, XEG_SpatialUpscale xegSpatialUpscale, XEG_SpatialUpscaleDescription *pXegSpatialUpscaleDescription) Executes the rendering command for GPU spatial upscaling.
VKAPI_ATTR void VKAPI_CALL HMS_XEG_DestroySpatialUpscale (XEG_SpatialUpscale xegSpatialUpscale) Destroys an XEG_SpatialUpscale object.

Service Process

  1. When a user enters the game's initialization scene, call the HMS_XEG_GetString API to query the list of features supported by XEngine Kit. Check whether the returned list contains XEG_SPATIAL_UPSCALE_EXTENSION_NAME. If not, the current device does not support this feature, and the process ends.
  2. Call the HMS_XEG_SpatialUpscaleParameter API to set the upscaling parameters.
  3. When the game is running, it renders the current frame texture to be upscaled.
  4. When the rendering of the texture to be upscaled is completed, call the HMS_XEG_RenderSpatialUpscale API to perform upscaling on the texture.
  5. When the upscaling rendering is completed, the game renders the remaining textures, such as the UI. Perform the final presentation operation after the frame rendering is completed.
  6. When the game exits, the upscaling resources are automatically released.
  1. When a user enters the game's initialization scene, call the HMS_XEG_EnumerateDeviceExtensionProperties API to query the list of features supported by XEngine Kit. Check whether the returned list contains XEG_SPATIAL_UPSCALE_EXTENSION_NAME. If not, the current device does not support this feature, and the process ends.
  2. Call the HMS_XEG_CreateSpatialUpscale API to create an upscaling instance.
  3. When the game is running, it renders the current frame texture to be upscaled.
  4. When the rendering of the texture to be upscaled is completed, call the HMS_XEG_CmdRenderSpatialUpscale API to perform upscaling on the texture.
  5. When the upscaling rendering is completed, the game renders the remaining textures, such as the UI. Perform the final presentation operation after the frame rendering is completed.
  6. When the game exits, call HMS_XEG_DestroySpatialUpscale to destroy the upscaling instance.

Development Procedure

This section uses the OpenGL ES and Vulkan graphics APIs as examples to describe the XEngine Kit integration process.

[h2]Configuring Your Project

During HAP compilation, the native layer's .so files must be built with a dependency on libxengine.so from the NDK.

[h2]Integrating the GPU Spatial Upscaling Capability of XEngine Kit (OpenGL ES)

EGL and OpenGL ES graphics APIs are used to set up the image rendering pipeline and integrate GPU spatial upscaling at the native layer. The rendering results are displayed on the screen through XComponent.

This section describes how to use GPU spatial upscaling through OpenGL ES graphics APIs. For details about the sample code, please refer to Demo (XEngine Kit-GLES).

Before calling XEngine Kit capabilities, you need to use SysCap to check whether your target device supports the SystemCapability.Graphic.XEngine system capability.

  1. Call the HMS_XEG_GetString API to obtain the extensions supported by XEngine Kit. The GPU spatial upscaling APIs can be used only when the XEG_SPATIAL_UPSCALE_EXTENSION_NAME extension is supported.

    // Query GLES extensions supported by XEngine Kit.
std::string extensionStr = (const char*)HMS_XEG_GetString(XEG_EXTENSIONS);
std::vector<std::string> extensions;
std::istringstream istringstream(extensionStr);
std::string word;
while (istringstream >> word) {
    extensions.push_back(word);
}
    
// ...
// Check whether GPU spatial upscaling is supported.
if (std::find(extensions.begin(), extensions.end(), XEG_SPATIAL_UPSCALE_EXTENSION_NAME) != extensions.end()) {
    // Normal processing logic.
    // ...
} else {
    // Handle the error.
    // ...
}

  2. Call the HMS_XEG_SpatialUpscaleParameter API to assign values to the GPU spatial upscaling parameters.

    // m_lowResWidth and m_lowResHeight indicate the custom rendering width and height, respectively.
// upscaleScissor indicates the sampling area of the upscaling input image.
int upscaleScissor[4] = {0, 0, static_cast<int>(m_lowResWidth), static_cast<int>(m_lowResHeight)};

// m_sharpness indicates the custom upscaling sharpening parameter.
HMS_XEG_SpatialUpscaleParameter(XEG_SPATIAL_UPSCALE_SHARPNESS, &m_sharpness);
HMS_XEG_SpatialUpscaleParameter(XEG_SPATIAL_UPSCALE_SCISSOR, upscaleScissor);

  3. Call the HMS_XEG_RenderSpatialUpscale API to perform upscaling.

    // m_upscaleFBO indicates the custom frame buffer.
glBindFramebuffer(GL_FRAMEBUFFER, m_upscaleFBO);
// m_highResWidth and m_highResHeight indicate the custom upscaling width and height, respectively.
glViewport(0, 0, m_highResWidth, m_highResHeight);
glScissor(0, 0, m_highResWidth, m_highResHeight);
// m_lowLightColorTexture indicates the texture attachment. You can customize this parameter as required.
HMS_XEG_RenderSpatialUpscale(m_lowLightColorTexture);

    m_upscaleFBO indicates the frame buffer you have created, which will be bound to the texture. The image will be drawn onto the texture after you call the upscaling API.

[h2]Integrating GPU Spatial Upscaling of XEngine Kit (Vulkan)

Vulkan graphics APIs are used to set up the image rendering pipeline and integrate GPU spatial upscaling at the native layer. The rendering results are displayed on the screen through XComponent.

This section describes how to use GPU spatial upscaling through Vulkan graphics APIs. For details about the sample code, please refer to Demo (XEngine Kit-Vulkan).

Before calling XEngine Kit capabilities, you need to use SysCap to check whether your target device supports the SystemCapability.Graphic.XEngine system capability.

  1. Call the HMS_XEG_EnumerateDeviceExtensionProperties API to obtain the extensions supported by XEngine Kit. The GPU spatial upscaling APIs can be used only when the XEG_SPATIAL_UPSCALE_EXTENSION_NAME extension is supported.

    // Query the Vulkan extensions supported by XEngine Kit.
std::vector<std::string> supportedExtensions;
uint32_t pPropertyCount;
// physicalDevice indicates a Vulkan physical device. You need to initialize it.
HMS_XEG_EnumerateDeviceExtensionProperties(physicalDevice, &pPropertyCount, nullptr);
if (pPropertyCount > 0) {
    std::vector<XEG_ExtensionProperties> pProperties(pPropertyCount);
    if (HMS_XEG_EnumerateDeviceExtensionProperties(physicalDevice, &pPropertyCount,
        &pProperties.front()) == VK_SUCCESS) {
        for (auto ext : pProperties) {
            supportedExtensions.push_back(ext.extensionName);
        }
    }
}
// Check whether GPU spatial upscaling is supported.
if (std::find(supportedExtensions.begin(), supportedExtensions.end(), XEG_SPATIAL_UPSCALE_EXTENSION_NAME) ==
    supportedExtensions.end()) {
    // Handle the error.
    // ...
}

  2. Declare the instance handle.

    XEG_SpatialUpscale xegSpatialUpscale = nullptr;

  3. Call the HMS_XEG_CreateSpatialUpscale API to create an upscaling instance.

    // VkRect2D indicates the 2D area structure specified by Vulkan.
// srcRect2D indicates the upscaling input texture area. You can customize this parameter as required.
VkRect2D srcRect2D;
// srcRect2D.offset.x and srcRect2D.offset.y indicate the offset from the origin.
srcRect2D.offset.x = 0;
srcRect2D.offset.y = 0;
// srcRect2D.extent.width and srcRect2D.extent.height indicate the width and height of the input texture, respectively.
// lowResWidth and lowResHeight indicate the custom rendering width and height, respectively.
srcRect2D.extent.width = lowResWidth;
srcRect2D.extent.height = lowResHeight;

// dstRect2D indicates the upscaling output texture area. You can customize this parameter as required.
VkRect2D dstRect2D;
// dstRect2D.offset.x and dstRect2D.offset.y indicate the offset from the origin.
dstRect2D.offset.x = 0;
dstRect2D.offset.y = 0;
// dstRect2D.extent.width and dstRect2D.extent.height indicate the width and height of the upscaling texture, respectively.
// highResWidth and highResHeight indicate the custom width and height after upscaling, respectively.
dstRect2D.extent.width = highResWidth;
dstRect2D.extent.height = highResHeight;

XEG_SpatialUpscaleCreateInfo createInfo;
createInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
// sharpness indicates the custom sharpening parameter. Here, the value 0.2f is used as an example.
createInfo.sharpness = 0.2f;
createInfo.outputSize = dstRect2D.extent;
createInfo.inputRegion = srcRect2D;
createInfo.inputSize = srcRect2D.extent;
createInfo.outputRegion = dstRect2D;
// Logical device. You need to initialize it.
HMS_XEG_CreateSpatialUpscale(device, &createInfo, &xegSpatialUpscale);

  4. Call the HMS_XEG_CmdRenderSpatialUpscale API for each frame to deliver the upscaling command.

    XEG_SpatialUpscaleDescription xegDescription{0};
// inputColorView indicates the VkImageView of the upscaling input image you have created.
xegDescription.inputImage = inputColorView;
// outputColorView indicates the VkImageView of the upscaling output image you have created.
xegDescription.outputImage = outputColorView;
// drawCmdBuffers[currentBuffer] indicates the command buffer and needs to be initialized.
HMS_XEG_CmdRenderSpatialUpscale(drawCmdBuffers[currentBuffer], xegSpatialUpscale, &xegDescription);

  5. Call the HMS_XEG_DestroySpatialUpscale API to destroy the instance.

    HMS_XEG_DestroySpatialUpscale(xegSpatialUpscale);