Artificial intelligence becomes a guide for the blind. The NaviSense app finds objects with a voice command

Researchers at Penn State University have introduced a new app called NaviSense that allows blind and visually impaired people to identify and navigate to objects with just a simple voice command.

This application uses the smartphone camera to track objects in real time and guides the user to the target with 3D spatial sound, left, right, up, down, and phone vibration, without the need for manual marking or complex settings.

Unlike previous systems that required pre-trained models, NaviSense is able to recognize any desired object without prior training by connecting to external servers and utilizing advanced artificial intelligence models.

If the user’s command is ambiguous, the application asks additional questions to act more accurately.

This application was developed with the aim of increasing the independence and safety of blind people in indoor and outdoor spaces.