Raspberry Setup Guide and Overall Introduction

This guide explains how to setup your raspberry pi and explains the general structure of the project

Introduction

VAAK is a college project under the TARP course. Under this course we had to make groups and come up with creative solutions to real world problems and implement them. Our group decided to make a project that helped specially-abled people who cannot speak by making a device that took input from their fingers in a fixed format(For example: American Sign Language), process it and play out the processed data via a speaker that is either interfaced with an android smartphone or simply a smartphone's inbuilt speaker. Members under the project are:

• Naman Arora
• Rishabh Agarwal
• Soham Sengupta
• Aditya Kshetriya
• Arpan Satpathi
• Dharoori Rakesh Acharya

Check out out our product website here. Also check out our project report here

Tech Used

• Arduino IDE 1.8.9 (for burning the code into the Arduino)
• Python 2.7 (to upload the alphabets received by the Raspi to the Firebase)
• Android Studio 3.1.4 (for making an app for text to speech conversion)

Brief working of the project

We use flex sensors for getting the input from the fingers of the user. These sensors are connected to an Arduino Uno, which processes and sends the data to a raspberry pi serially, which then sends the data to a Firebase Realtime database. The android app interacts with this database and reads out the interpreted text.

For now, we did not study the American Sign Language, so we used the sensors to send alphabet combination instead. For example, since we have 26 alphabets, we use 5 flex sensors and assign each binary combinaion of the input to a specific alphabet. In this repository, however, we have both used 2 and 5 sensors, so make sure to use the correct .ino file.

The project can be divided in the following parts:

• Getting the data from Arduino Uno
• Processing the data inside the Arduino script
• Sending the data to a Raspberry Pi, or a wifi module
• Transmitting the data to a specific Firebase Realtime database
• Using the android app to consume the data from the database
• Using the app to control the speakers connected to the device and reading out the text/alphabet

The code given in this repo is only for alphabet reading and nothing else.

We used a raspberry pi to send the data to the Firebase Realtime database using the rev.py script. You can learn more about it here.

Important

Before you move on to the next section, you need to set up your input circuit with the Arduino. To learn how to do that go to Circuit.md. The application requires Arduino IDE so that you can push your code into your Arduino.

Prerequisites

For Android App

In RPi/app/src/main/java/c/vaak/vaak/MainActivity.java, replace #YOUR_EMAIL with your Gmail ID with which firebase authentication is enabled and #YOUR_PASSWORD with the password for authentication.

To get the values mentioned above, you need to:

• Create an Authentication Method
• Create a Realtime Database

For Authentication Method

Go to your firebase console and create a new project. Now go to the console and click on Authentication > Sign-in method > Email/Password. Add your Gmail ID and your access password. These will be the values of #YOUR_EMAIL and #YOUR_PASSWORD respectively.

For Realtime Database

Go to the console and click on Database > Create Database > (Start in test mode) > Enable. This will create an instance of both Firestore and Realtime Database. To go to Realtime database click on the dropdown menu and select Realtime Database. The link that you see will be the value of #Link_to_your_realtime_database in the next section.

For Raspberry Pi

In all files within the Raspberry Code folder, replace #Location_of_your_private_key_generated_in_firebase(a json file) with the location of the json file in your device which will be generated by firebase. You can generate the file for this @Project Settings (Icon next to Project Overview) > Service Accounts > Generate new private key.

Also replace #Link_to_your_realtime_database with the link to your firebase Realtime database. You can find it by going to Develop > Database >Realtime Database > (copy the link in the form "https://Project-name.firebaseio.com")

To learn more about the method we have used go to Introduction to the Admin Database API and use the Python language.

Installation for Raspberry Pi

You will need to install the following dependencies on your RPi:

$ pip install serial
$ pip install firebase-admin
$ pip install time      #only if you want to give delays like: time.sleep(3)

After making sure you are getting the accurate values from your Arduino, connect your Raspberry Ri with your Arduino via the included USB cable. Now our Arduino code is running on a serial connection on the Raspberry Pi continuously. You can check if your raspberry pi detected your Arduino by running the following command in your terminal:

ls /dev/tty*

If /dev/ttyACM0 shows up in the list, you're good to go! Now run the python script by running the following command in your terminal:

python rev.py

This will send your Arduino data to your firebase Realtime database. Which you can use in the android app.

Buiding the Android App

Open Android studio and click on Import existing project. Now browse to the location where you have downloaded the repository and import the folder named "Android App for using data from RPi". Now that the project is imported you can run the app in your emulator or make an apk for installation on your own device.