Helper For Visually Impaired People
What a blind person needs is not a teacher but another self.
Overview
Technological innovation and advances in Android and IoS operating systems made our everyday lives to be focused on smartphones. Measuring the success of mobile technologies and apps is based on its ability to respond to the needs of individuals within an entire population including visually impaired people. The creative innovation of today’s mobile technology includes Text-to-Speech, navigation assistance, object identification, color identification, speech recognition facilities are integrated with basic mobile services. It also provides easy-to-use user interfaces which include a fully automated voice-recognition system to support visually impaired users. Hence, the aim of this app is to developed mobile applications which are targeted at bringing back the joy in the life of visually impaired people.
Introduction
we have include 8 functionality,
- Object detection
- Currency detection
- Extract text from the image
- Calling app
- Message sender
- Location
- Weather checker
- Battery checker
Snapshot of App
Object detection
In this module, Custom Object Detection is done using TensorFlow Teachable Machine. we have categorized objects in 8 different datasets like Animal, Electronic items, Food, Furniture, Person, vehicle, Obstacle, and Others.
Teachable Machine use to recognize our own image dataset, sounds, & poses. It is a fast, easy way to create machine learning models for our sites, apps, and more. The magic behind Teachable Machine is based on a popular deep learning technique called transfer learning. Most of the neural network architecture of a fully trained model is retained while replacing a minor part of it based on the data. This approach not only requires less compute power but also requires a smaller dataset for training. Google is leveraging some of the best deep learning and neural network models for Teachable Machine.
for the blind, we have included 7 different mp3 files that will play based on any detected objects.
Currency Detection
This module helps blind people to detect currency through a webcam. This currency detection is again done through TensorFlow Teachable Machine. The currency is categorized into 5 different datasets like 50, 100, 200, 500, 200 currency notes.
After the detection is done the person gets feedback by Text to Speech API of android studio and this helps the user to correctly identify the currency without any external help.
Extract text from the image
This module helps users to read the sentences and text from the images. So here the user has to capture an image and for this <uses-permission android:name=”android.permission.CAMERA”/> needs to be included in AndroidManifest.xml. Once an image is captured text will be extracted and will be converted in speech using TextToSpeech of Android to assist people.
Calling app
To make the call easier for blind people I made the call feature that is operated using gestures detector like swipe up, swipe left, swipe up. as those actions don’t require vision. It could easily be done using implicit Intent with appropriate actions. Uses permissions must be provided in AndroidManifest.xml file <uses-permission android:name=”android.permission.CALL_PHONE” />
Android supports Google inbuilt text to speak API using RecognizerIntent. ACTION_RECOGNIZE_SPEECH. Here, when the user clicks on the call button, he/she will get to know all the information like how to operate it. Here the user can speak the number by doing swipe up on the screen, confirm the number using swipe-left, and call the person using swipe-right.
Message Sender
Android provides SmsManager API or devices Built-in SMS application to send SMS. Here I have used the SmsManager object to send an SMS to the given mobile number. In this, we also need to create implicit intent. for that, we need to add uses-permission in AndroidManifest.xml file <uses-permission android:name=”android.permission.SEND_SMS” />
As a call, the same type of gesture pattern is used. To speck phone number and message, the user has to swipe-up and swipe down respectively. to confirm the message and phone number and send the message user have to swipe-left and swipe-right respectively.
Location
This is the most important module of the application. The threat to the blind person is the unknown place where he is. So to know the current location is a very important task. So here the user will be provided with this facility and the current exact location will be given to help them. Also when the results are shown they are automatically converted in speech using TextToSpeech of Android to assist the visually impaired person. To implement this we need to add <uses-permission android:name=”android.permission.ACCESS_FINE_LOCATION”/> In AndroiManifest.xml file.
Weather checker
In this module, the person needs to speak the place name where he wants to find the weather. This module works perfectly for places outside of India too. To implement this module we have created a weather API from the OpenWeatherMap website. The information like Temperature, Humidity, Description, Pressure, etc. can be obtained by the weather API. When the results are shown they are automatically converted in speech using TextToSpeech of Android to assist the visually impaired person.
Battery Checker
In this module, the person can be assisted by knowing the battery percentage of his phone. Also, information on whether the phone is charging or discharging can be obtained. For this BatteryManager is used to implement. This takes permission to access the phone’s battery information. After the information is received it is automatically converted in speech using TextToSpeech of Android to assist a visually impaired person.
Demo Video
Conclusion
At present mobile apps are able to do most of the activities. But it is difficult for visually impaired people to use apps. So this app will guide the person at each step using voice feedback and taking inputs from the person’s voice directly. Also, the modules are designed keeping in mind people's daily needs and thus this app proves to be highly beneficial for such people.
