Supersense AI

Created By: Mediate Labs (2020)
Role: Product & Research Director
Project Site: www.supersense.app
Supported By: National Science Foundation & US Veteran Affairs

Supersense is a mobile assistive technology application for visually impaired people & blind (VI&B) Mobile artificial intelligence systems present a significant opportunity for improving the lives of more than 300 million VI&B by reducing the challenges in their everyday life such as navigation, orientation and mobility (O&M), reading, and more. Accessibility-enabled applications often yield poor and overwhelming experiences for VI&B who has different needs and drastically different ways of interacting with mobile technology. In making Supersense, we adopted a user-centered design process and introduced several guidelines for designing accessibility-first applications. The projectis funded by the MIT DesignX program, the National Science Foundation, and the Veteran Affairs. It has reached more than one hundred thousand blind users globally.

ACCESSIBILITY FIRST DESIGN

Supersense is an assistive mobile app that provides valuable and timely information to the user. Initially designed as a spatial exploration system that blind individuals might use to find objects and understand the space around them, Supersense became a complete solution kit following our extensive user research. Currently, the system has two disinct modes: Reading and Navigation. In the reading mode, the app helps the blind user to access printed information, identify currency, obtain precise details through barcodes, or magnify the image. The reading task has been one of the most challenging: the user has to orient the device at a correct angle and gather a complete picture before scanning. One of our innovations is the guidance system instructs users to move their phones to capture a printed document. In navigation mode, Supersense acts as an object explorer that helps an individual to navigate toward an object of interest, such as an empty chair or a door.

Creating experiences driven by deep neural networks deployed in a mobile device is relatively new territory. Supersense is one of the earliest examples of its kind. We tackle the technical challenges of running models, where we cannot trade accuracy for performance and user experience challenges stemming from interacting with systems running in real-time.