Provide the capability of flexibly editing the meta model online and support data models of various types and the service management process.
Provide complete support of data standards and design logical models to satisfy service management and control.
Automatically establish the mapping between the logical model and physical object and complete the quality verification and anonymization during data extraction, loading, and cleansing.
Each computing engine will use the unified logical model and improve the comprehensive capabilities of the Universe Analytics Platform.
- Data Quality Management
- Data Security Management
- Data Model Management
- Data Standard Management
Data quality management measures data quality availability, integrity, and accuracy.
The Data Governance provides the unified data quality audit that can be performed in the data production process. Then the audit result can be used to control data production, which radically solves the data quality management problems.
Data quality audit rules associate universal data quality rules with databases and database tables (including fields), providing a basis for data quality analysis.
Creating Data Quality Audit Rules
After data quality rules are configured, the Data Governance server reports the audit task information to the Unified Scheduling. Data audit for flows is completed after the flows are configured and executed in the Unified Scheduling. Operators can check the audit result through the data quality monitoring function.
Quality Audit Result
The data quality monitoring page displays the status of the latest quality audits for real-time data quality monitoring, helping find quality issue timely. Operators can check the data quality of each data layer and specific data entities from different dimensions.
Quality Dimensions and Data Quality Details in Each Layer
Quality Monitoring Status
The DG provides pages for processing issues generated during production and O&M, reported data quality issues, issues detected during maintenance, and issues reported by service personnel, adding the processing results to the knowledge base, and classifying and recording the issue causes and solutions, which helps implement experience accumulation and query.
Data Quality Knowledge Base
Data security privacy management is mandatory for data openness, ensuring carrier data asset security, information security, and user privacy security.
Anonymization policy management includes data anonymization policy configuration, algorithm key management, and logical model-based anonymization algorithm configuration.
Anonymization Policy Management
Data encryption is a common method used for data anonymization. Users assigned the permission to view data plaintext can obtain the plaintext through the data decryption service.
Data Encryption and Decryption
− Encryption service
During data loading or export, data can be encrypted based on configured anonymization policies.
The Data Governance provides the encryption service for third parties. A third party provides plaintext and the Data Governance encrypts it and returns the ciphertext.
− Decryption service
Operators who have been assigned the permission to access plaintext can obtain the data in plaintext through the data decryption service.
The Data Governance provides the decryption service for third parties. A third party provides ciphertext and the Data Governance decrypts it and returns the plaintext.
− Key management
Encryption algorithms have their own keys. All keys are encrypted and stored in the database. An operator needs to enter the login password to read the keys.
The data access permission on a new layer or model of the current tenant can be assigned to other tenants.
Data Access Control
A unified metadata library is constructed to store metadata of each module (including the entity metadata, flow metadata, and data governance rule metadata). Analysis is performed based on the metadata that is uniformly stored, ensuring the timeliness and consistency of metadata production and analysis.
After a data model is created, the DG automatically creates a physical entity based on the data structure and storage rules of the data model and creates the mapping between the physical entity and data model.
The mapping between the data model
The data landscape macroscopically provides a global data asset map including data tiering and data model information. The data landscape helps users quickly understand data distribution and perform data value analysis.
The data landscape supports the drilling function. An operator can click a data layer to access next data layers till data models are accessed.
The data storage period can be set, and data aging is automatically executed. That is, if an aging period is set, when the data reaches the preset data aging time, the system automatically deletes aged data.
Setting the data model storage period
After a data model is submitted, the model reviewer needs to review the data model. Only the approved model can be used.
Reviewing Data Models
The model development report provides data model development records and model usage information, including:
− Model Creator
− Operation Time
− Operation Type
Model Development Report
Operators can copy data models or export data models to other environments.
To export data models to a third-party metadata management platform, information about the third-party metadata management platform needs to be configured in advanced. Currently, the third-party metadata management platform Primeton MetaCube is supported.
Through the openness of metadata-related interfaces, the Data Governance can be integrated into a third-party system, facilitating suite adaption to a third-party big data platform.
Optimal data governance rules help customers manage data assets based on unified standards, ensuring data asset consistency with the enterprise.
The Data Governance provides data tiering standards, entity design standards, service terminologies, and code lists.
Service terminologies are used to uniformly define and describe service languages within an enterprise. Service terminologies are viewed by users and have no technical restraints on the service logic.
The code list uniformly specifies values in the drop-down list boxes of service metadata parameters and checks values of specific fields in some data tables.
Code List Management
Quality rules can be created based on database fields, a database table, or multiple database tables.
For each quality audit rule, a default quality threshold can be specified. If this threshold is exceeded, associated actions are triggered, for example, log recording and alarm reporting.
The Data Governance supports model management by layer and domain. One logical entity can belong to only one layer. Layer or subject area information can be added, modified, or deleted based on service requirements. A maximum of five layers are supported.
Data tiering standards standardize the data layer design process. Data layer naming rules can be formulated using audit statements. Both the names and abbreviations of models and fields must comply with the naming rules.
Data Tiering Standards
Data model design standards standardize the model design process.
Data model and field naming rules can be formulated using audit statements. Both the names and abbreviations of models and fields must comply with the naming rules.
Data tiering standard
Unified document management includes document storage, query, download, and update. Documents are viewed by users and have no technical constraints on the service logic.