HUAWEI HiAI is an open artificial intelligence (AI) capability platform for smart devices, which adopts a "chip-device-cloud" architecture, opening up chip, app, and service capabilities for a fully intelligent ecosystem. This assists developers in delivering an optimal smart app experience for users, by fully leveraging Huawei's powerful AI processing capabilities.

In order to open up app capabilities, HUAWEI HiAI Engine provides a wide range of AI capabilities that can be integrated into apps. Table recognition, one such AI capability, empowers apps to:

For more information, please visit our official website at the following URL:

What You Will Create

In this codelab, you can use this AI capability to create an Android app that recognizes table content in an image and generates an Excel file based on results, for less costly and time-consuming app development.

For example:
The following is a table image.

The app recognizes the table content, and can generate an Excel file as follows.

What you'll learn

What You Will Need

Hardware Requirements

Software Requirements

Step 1 Start Android Studio and choose Start a new Android Studio project.

Step 2 Create an Android project, following the project creation wizard, and click Next.

Set Application name to a user-defined app name, for example, visionfun.

Set Company domain to a user-defined company or organization domain name, for example, hiai.plx.huawei.com.

Set Project location to a user-defined project directory, for example, D:/code/visionfun.

Select an API whose version number is later than 26, and click Next.

Select Empty Activity as the default window and click Next.

Set Activity Name to the default value MainActivity.

Set Layout Name to the default value activity_main and click Finish.

Android Studio will then create and compile the project code.

When the message "Gradle build finished" displays in the lower left corner, the project compilation has been completed. Otherwise, some errors may occur, such as the network connection error. In this case, check the network connection, proxy settings, and digital certificate.

Step 1 Save the downloaded HiAI SDK package under app > libs for your project.

Step 2 Configure the build.gradle file for the project.

Make sure that the build.gradle file contains Google Maven repository.

allprojects { repositories { google() ... mavenLocal() } }

Make sure that the build.gradle file contains the following content:

android { compileSdkVersion 28 defaultConfig { ... targetSdkVersion 28 ... } repositories{ flatDir { dirs 'libs' } } } dependencies { ... implementation (name: 'huawei-hiai-vision', ext: 'aar') implementation 'com.google.code.gson:gson:2.8.2' ... }

Step 3 Update the AndroidManifest.xml file.

Add the camera access permission configuration to the app > manifests > AndroidManifest.xml file for your app in Android Studio.

<?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" package="..." > ... <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.CAMERA" /> ...

You can refer to the following example, when designing the UI.

Step 1 Initialize HiAI.

public class MainActivity extends AppCompatActivity { ... @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); //To connect vision service VisionBase.init(getApplicationContext(), new ConnectionCallback() { @Override public void onServiceConnect() { Log.i(TAG, "onServiceConnect "); } @Override public void onServiceDisconnect() { Log.i(TAG, "onServiceDisconnect"); } }); ... } }

Step 2 Initialize widgets and set the tap event listener.

public class MainActivity extends AppCompatActivity { Button btnCamera; Button btnGallery; Button btnRun; ImageView imgMain; Thread mThread; Bitmap mBitmap; Bitmap mBitmapSR; private Uri imageUri; ... @Override protected void onCreate(Bundle savedInstanceState) { ... imgMain = (ImageView) findViewById(R.id.img_main); btnCamera = (Button) findViewById(R.id.btn_camera); btnGallery = (Button) findViewById(R.id.btn_gallery); btnRun = (Button) findViewById(R.id.btn_run); btnCamera.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { onCameraClick(); } }); btnGallery.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { onGalleryClick(); } }); btnRun.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { mThread = new Thread(r); mThread.start(); } }); requestPermissions(); }

Step 3 Apply for the camera and storage execution permissions.

public class MainActivity extends AppCompatActivity { ... private void requestPermissions(){ try { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { int permission = ActivityCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE); if(permission != PackageManager.PERMISSION_GRANTED) { ActivityCompat.requestPermissions(this, new String[] {Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE}, REQUEST_IMAGE_GALLERY); } permission = ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA); if(permission != PackageManager.PERMISSION_GRANTED) { ActivityCompat.requestPermissions(this, new String[] {Manifest.permission.CAMERA}, REQUEST_IMAGE_CAPTURE); } } } catch (Exception e) { e.printStackTrace(); } } @Override public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) { super.onRequestPermissionsResult(requestCode, permissions, grantResults); } }

Step 4 Add the methods for opening the camera and gallery, taking a picture, and selecting an image.

public class MainActivity extends AppCompatActivity { ... private void onCameraClick() { takePictureFromCamera(); } private void onGalleryClick() { chooseImageFromStorage(); } }

The takePictureFromCamera() and chooseImageFromStorage() methods are used respectively, to take a picture and select an image from gallery. Define them as follows:

public class MainActivity extends AppCompatActivity { ... private void takePictureFromCamera() { File fileImage = new File(getExternalCacheDir(), "image.jpg"); try { if (fileImage.exists()){ fileImage.delete(); } fileImage.createNewFile(); } catch (IOException e) { e.printStackTrace(); } if (Build.VERSION.SDK_INT >= 24){ imageUri = FileProvider.getUriForFile(MainActivity.this, "com.huawei.hiai.tutorial.visionfun.fileProvider", fileImage); } else { imageUri = Uri.fromFile(fileImage); } Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE); intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri); if (intent.resolveActivity(getPackageManager()) != null) { startActivityForResult(intent, REQUEST_IMAGE_CAPTURE); } } private void chooseImageFromStorage() { Intent intent = new Intent(Intent.ACTION_GET_CONTENT); intent.setType("image/*"); if(intent.resolveActivity(getPackageManager() )!= null){ startActivityForResult(intent, REQUEST_IMAGE_GALLERY); } } }

In the preceding code, the getExternalCacheDir() method is used to obtain the directory for storing cache data of the current app in the SD card.

The getUriForFile() method is used to receive the Context object, any unique character string, and the File object. On Android 7.0or later, the real local directory cannot be used because the system considers it insecure and will throw the FileExposedException exception if it is used. The FileProvider class is a special content provider that protects data using a mechanism similar to that of the content provider.

Register FileProvider with the AndroidManifest.xml file and add the following provider section to the application section:

<provider android:name="android.support.v4.content.FileProvider" android:authorities="com.huawei.hiai.tutorial.visionfun.fileProvider" android:exported="false" android:grantUriPermissions="true"> <!-- Specify the shared path for Uri --> <meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/file_provider_paths" /> </provider>

In the code, meta-data references xml to specify the shared directory.

Hover the mouse pointer over the missing resource definition (@xml/file_provider_paths in red). When a bulb icon displays at the beginning of this line, click the drop-down arrow and select Create xml resource "file_provider_path.xm"**".

Click OK and add the following definition to the XML file that is created:

<external-path name="img" path=""/>

The following figure shows the configuration effect.

Define the onActivityResult() method to receive the result of taking a picture or selecting an image.

@Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { //super.onActivityResult(requestCode, resultCode, data); if(resultCode == RESULT_CANCELED) return; if (resultCode == RESULT_OK){ try { if (requestCode == REQUEST_IMAGE_CAPTURE) mBitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(imageUri)); imgMain.setImageBitmap(mBitmap); else if (requestCode == REQUEST_IMAGE_GALLERY) mBitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(data.getData())); imgMain.setImageBitmap(mBitmap); }catch (FileNotFoundException e) { e.printStackTrace(); } } }

Step 5 Call the API in a child thread.

The mThread runs as a child thread independent from the main thread. During the running, the mThread waits for the notify event of the mWaitResult object. Each time the mThread obtains the event, it calls the specified CV HiAI model or algorithm to process the image synchronously. After the processing has completed, the result will display.

The Java Handler message mechanism (mHander) uses a message to asynchronously send the processing result to the main thread, which improves the system scalability and facilitates future maintenance.

public class MainActivity extends AppCompatActivity { ... private Runnable r = new Runnable() { @Override public void run() { if (mBitmap == null) return; Log.e("tag", "start"); Frame frame = new Frame(); frame.setBitmap(mBitmap); TableDetector mTableDetector = new TableDetector(MainActivity.this); JSONObject json = mTableDetector.detect(frame, null); String newJson = json.toString().replaceAll("\\\\", ""); Log.e("tag", " json=" + newJson); Table table = mTableDetector.convertResult(json); if (table == null) { return; } Log.e("tag", " table=" + table); int count = table.getTableCount(); List<TableContent> Listtc = table.getTableContent(); StringBuilder sbTableCell = new StringBuilder(); List<TableCell> tableCell = Listtc.get(0).getBody(); for (TableCell c : tableCell) { List<String> words = c.getWord(); StringBuilder sb = new StringBuilder(); for (String s : words) { sb.append(s).append(","); } String cell = c.getStartRow() + ":" + c.getEndRow() + ": " + c.getStartColumn() + ":" + c.getEndColumn() + "; " + sb.toString(); sbTableCell.append(cell).append("\n"); } Log.e("tag", " tableCOunt=" + count); Log.e("tag", " sbTableCell=" + sbTableCell.toString()); // Return messages to the main thread through Handler Message msg = new Message(); msg.what = 100; msg.obj = sbTableCell.toString(); mHander.sendMessage(msg); } }; }

Step 6 Process the message returned by the Handler.

public class MainActivity extends AppCompatActivity { ... private Handler mHander = new Handler() @Override public void handleMessage(Message msg) { super.handleMessage(msg); int status = msg.what; switch (status) { ... case 100: if (msg.obj == null) { return; } String result = (String) msg.obj; // Execute your actions after you get the results break; ... } }; }

Step 1 Verify the mobile phone connection.

To verify that your Huawei mobile phone has been successfully connected to the computer through a USB cable, start the command console and run the following command:

adb devices

You can expect for the serial number of your mobile phone to display. An example of the output is as follows:

List of devices attached

The preceding information indicates that the mobile phone has successfully connected.

Step 2 Start the app.

In Android Studio, click Run. Then, select your mobile phone as the deployment target and click OK to start the app on the mobile phone.

Well done. You have successfully completed this codelab, and learned:

For more information about HUAWEI HiAI, please visit our official website at the following URL:


Code copied