StorySign

Introduction to StorySign

StorySign is a reading app for children with hearing impairment. It supports the sign language translation of selected classics for children, making it easier for deaf-mute children to enjoy reading and for families with hearing-impaired children to have parent-child reading time. StorySign brings innovative changes to the literacy education of children with hearing impairment.

Pain Points

Reading based on phonics is one of the main learning methods for young beginners to acquire new knowledge. Nowadays, most children learn how to read by listening to their teachers and parents' talking. However, there are 320 million deaf-mute children in the world, who are faced with great obstacles in reading because of hearing loss or impairment. Among them, 90% are born to parents with normal hearing, which means their parents do not know how to use the sign language and thus have trouble in teaching them how to read. This makes parent-child reading even harder.

Severe shortage of sign language teachers: As the number of deaf-mute young learners is large, the problem of serious shortage of hand language teachers has been persistent.

Inability of parents to use sign language: Parents of 90% of deaf-mute children have normal hearing and do not know how to use the sign language, so they cannot teach their children how to read.

Unbalanced distribution of digital education resources: The existing digital education resources are still concentrated on voice teaching which is the mainstream teaching mode. The digital education resources and modes available for deaf-mute learners are still very limited.

Solutions

HUAWEI HiAI worked with the non-profit organization European Union for the Deaf, Penguin, and animation giant Aardman and developed the free reading and sign language translation software StorySign, turning mobile phones into the best tool for deaf children to learn sign languages and reading. StorySign selects classics for children and supports the recognition of seven languages and their translation into sign language. Users may choose a language that is suitable for them. Open StorySign and aim the phone camera at the book page. Then StorySign can quickly recognize the texts. The user only needs to touch the words on the screen to translate them into sign language.

Automatic calibration to the angle of the image captured: On the strength of the document recognition and calibration capability of HUAWEI HiAI Engine, StorySign can effectively recognize and calibrate the book page image it detects. Even if the relative angle between the mobile phone and the book page is not good enough, StorySign can display the detected image in a regular pattern, making subsequent text recognition easier and more accurate.

Automatic recognition of seven languages: The general text recognition capability of HUAWEI HiAI Engine enables StorySign to recognize multiple languages, so that StorySign can quickly and accurately extract the texts in any of these languages on book pages and translate them into sign language. In addition, StorySign supports the translation into multiple countries' sign languages. Users can select the sign language of the UK, France, or Italy based on their preferences.

Benefits

Benefits to StorySign:

The solution benefits 320 million families with hearing-impaired children and enables them to enjoy reading anytime and anywhere, bringing innovative changes to the literacy education of deaf children.

Benefits to the users:

Children with impaired hearing and their partners with normal hearing can use StorySign to enjoy reading together.