BlindAid is an app that performs real time object detection by continuously detecting the objects (bounding boxes and classes) in the frames seen by your device's back camera, using a quantized MobileNet SSD model trained on the COCO dataset.
Additional features:
- Speech promopted service. Trigger object detection by saying 'guide me/ start guiding me'
- The app then performs position detection of different objects in a frame (left, centre and right) with speech
- 'Stop/ Stop guiding me' to stop the detection
- Speech prompted text detection. Trigger OCR with the words 'what's written in front of me/read'
- Performs barcode detection
The model files are downloaded via Gradle scripts when you build and run. You don't need to perform any steps to download TFLite models into the project explicitly.
Application can run either on device or emulator.
"Banana on the left, Orange on the right"
Barcode detection
Text detection example 1
Text detection example 2
The model used currently recognizes 80 different classes of objects, and will be trained on extended datasets to further increase the efficiency of the app.
- If you don't have it already, install Android Studio, by following the instructions on the website.
- You need an Android device and Android development environment with minimum API 21.
- Android Studio 3.2 or later.
- Clone the repository
- Open Android Studio, and from the Welcome screen, select Open an existing Android Studio project
- From the Open File or Project window that appears, navigate to and select the directory from wherever you cloned the project
- If it asks you to do a Gradle Sync, click OK.
- You may also need to install various platforms and tools, if you get errors like "Failed to find target with hash string 'android-21'" and similar.
- Run the app.
- Also, you need to have an Android device plugged in with developer options enabled at this point. See here for more details on setting up developer devices.
Downloading, extracting and placing it in assets folder has been managed automatically by download.gradle.
If you explicitly want to download the model, you can download it from here. Extract the zip to get the .tflite and label file.
Please do not delete the assets folder content. If you explicitly deleted the files, then please choose Build->Rebuild from menu to re-download the deleted model files into assets folder.