Team Members: Jake Ashmore Chris Turner

Overview¶

This project demonstrates the use of the OpenBCI Ganglion, as well as a MyoWare muscle sensor, to control a small robotic car. Previous work using the myoware sensor can be seen here. The OpenBCI Ganglion is a board that will allows for the detection of brain activity. For this project, we will be detecting Alpha Waves and using them as a control input to the car. Much of this work is a derivative of previous efforts by EEG Hacker and OpenBCI.

Getting Started¶

This project will be using a slightly modified version of the OpenBCI GUI found here. This code is written and run with processing.

The hardware used in this project is:

• 2 x MBED LPC1768
• 1 x OpenBCI Ganglion
• 1 x Gold cup electrodes
• 1 x Conductive Paste
• 1 x Muscle Sensor with extension boards
• 2 x nRF24L01 RF transceiver
• 1 x H-Bridge breakout
• 1 x Battery Holder with Barrel Jack
• 1 x breadboard compatible barrel jack
• 2 x DC Motors
• 1 x Shadow Chassis

The Robot¶

The instructions for building the Shadow Chassis can be found here. Note that we did not use most of the parts listed in the guide.

Vehicle Robot Hardware Setup¶

The robot side of the hardware setup consists of a mbed device, nrf transceiver, h-bridge, both motors, and the robot vehicle. First setup the robot kit according to the shadow robot instructions mbed page. The h-bridge should be connected following the sparkfun magician robot instructions, and finally connect the nRF transceiver according the mbed wiki instructions.

Below is a combination list of the used pins for the code demo provided.

This is the underside of the hbridge, so if the device is connected to the bread board be sure to flip the left and right sides.

Here is the code that was used on the robot.

#include "mbed.h"
#include "nRF24L01P.h"
#include "motordriver.h"
 
#define TRANSFER_SIZE 2

Serial pc(USBTX, USBRX); // tx, rx
nRF24L01P my_nrf24l01p(p5, p6, p7, p8, p9, p10);    // mosi, miso, sck, csn, ce, irq
Motor  right(p21, p22, p23, 1); // pwm, fwd, rev, has brake feature
Motor left(p26, p25, p24, 1);
AnalogIn sig(p15);
char rxData[TRANSFER_SIZE];
int rxDataCnt = 0;
float leftVal= 0, rightVal = 0;
DigitalOut led1(LED1);
DigitalOut led4(LED4);

void goBotGo()
{
        
        if ( rxData[0] == 'A' && rxData[1] == 'C' )
        {
            //add hit code here
            leftVal = leftVal + 0.2;
            rightVal = rightVal + 0.2;
        } else if ( rxData[0] == 'A' && rxData[1] == 'R' ) {
            rightVal = rightVal + 0.2;
            if (leftVal > 0 ) {
                leftVal = leftVal - 0.2;
            } else if (leftVal < 0 ) {
                leftVal = leftVal + 0.2;
            }   
        } else if ( rxData[0] == 'A' && rxData[1] == 'L' ) {
            leftVal = leftVal + 0.2;
            if (rightVal > 0 ) {
                rightVal = rightVal - 0.2;
            } else if (rightVal < 0 ) {
                rightVal = rightVal + 0.2;
            } 
        } else if (rxData[0] == 'S' ) {
             if (rightVal > 0 ) {
                rightVal = rightVal - 0.2;
            } else if (rightVal < 0 ) {
                rightVal =  rightVal + 0.2;
            } 
            if (leftVal > 0 ) {
                leftVal = leftVal - 0.2;
            } else if (leftVal < 0 ) {
                leftVal = leftVal + 0.2;
            }   
        }
        
        
        // Correct values in case of 
        if ( leftVal > 1.0 )
        {
            leftVal = 1.0;
        } else if ( rightVal > 1.0 ) {
            rightVal = 1.0;
        } else if ( leftVal < -1.0 ) {
            leftVal = -1.0;
        } else if ( rightVal < -1.0 ) {
            rightVal = -1.0;
        }
        left.speed(leftVal);
        right.speed(rightVal);
        //wait(0.1);
}
 
int main() {
    my_nrf24l01p.powerUp();

    // Display the (default) setup of the nRF24L01+ chip
    pc.printf( "nRF24L01+ Frequency    : %d MHz\r\n",  my_nrf24l01p.getRfFrequency() );
    pc.printf( "nRF24L01+ Output power : %d dBm\r\n",  my_nrf24l01p.getRfOutputPower() );
    pc.printf( "nRF24L01+ Data Rate    : %d kbps\r\n", my_nrf24l01p.getAirDataRate() );
    pc.printf( "nRF24L01+ TX Address   : 0x%010llX\r\n", my_nrf24l01p.getTxAddress() );
    pc.printf( "nRF24L01+ RX Address   : 0x%010llX\r\n", my_nrf24l01p.getRxAddress() );
 
    pc.printf( "Type keys to test transfers:\r\n  (transfers are grouped into %d characters)\r\n", TRANSFER_SIZE );
 
    my_nrf24l01p.setTransferSize( TRANSFER_SIZE );
 
    my_nrf24l01p.setReceiveMode();
    my_nrf24l01p.enable();
       
    while (1) {
        //rxData[0] = 'N';
        //rxData[1] = 'N';
        if ( my_nrf24l01p.readable() ) {
            led1 = !led1;
            rxDataCnt = my_nrf24l01p.read( NRF24L01P_PIPE_P0, rxData, sizeof( rxData ) );                  
            pc.printf("    - Received %d bytes\n\r", rxDataCnt);
            pc.printf(rxData);
            goBotGo();
        }
        //pc.printf("Checking Data.");
        
        
        
    }
}

Transmission Hardware Setup¶

Alpha wave detection is the main control input for the car. When alpha wave production is detected, the computer sends a command to the MBED controlling the muscle sensor. This then sends a packet to the MBED controlling the car via a RF module. Getting sent an 'A' (alpha wave detected) increases the speed of the motors by 10%, while getting sent an 'S' (no alpha waves detected) decreases the speed of the motors by 10%. The direction of the car is determined by the myoware muscle sensor. We have split the range of the muscle sensor into three levels. Between 0 and .4 the MBED will send an 'L' (turn left), between .4 and .8 the MBED will send a 'C' (center up), and between .8 and 1.0 the MBED will send an 'R' (turn right). The MBED on board the car takes the extracts the sent information and then makes the necessary adjustments to the motors. Please refer to the page linked above for more information about the myoware sensor.

On the transmission side of the setup, the second mbed device is connected to the nRF according to the same instructions, and then to a computer system through the mini-usb cable. The usb device is setup as a serial connection to receive signal from the processing software, which processes the analog signals from the BCI board, which is connected via bluetooth.

Here is the code used to transmit signals to the robot.

#include "mbed.h"
#include "nRF24L01P.h"
 
Serial pc(USBTX, USBRX); // tx, rx
nRF24L01P my_nrf24l01p(p5, p6, p7, p8, p9, p10);    // mosi, miso, sck, csn, ce, irq
AnalogIn sig(p15);       // Gives the rectified and integrated EMG signal 
 
int main() {
#define TRANSFER_SIZE 2
 
    char txData[TRANSFER_SIZE];
    int txDataCnt=0;
    my_nrf24l01p.powerUp();
    pc.printf( "nRF24L01+ Frequency    : %d MHz\r\n",  my_nrf24l01p.getRfFrequency() );
    pc.printf( "nRF24L01+ Output power : %d dBm\r\n",  my_nrf24l01p.getRfOutputPower() );
    pc.printf( "nRF24L01+ Data Rate    : %d kbps\r\n", my_nrf24l01p.getAirDataRate() );
    pc.printf( "nRF24L01+ TX Address   : 0x%010llX\r\n", my_nrf24l01p.getTxAddress() );
    pc.printf( "nRF24L01+ RX Address   : 0x%010llX\r\n", my_nrf24l01p.getRxAddress() );
    my_nrf24l01p.setTransferSize( TRANSFER_SIZE );
    my_nrf24l01p.setTransmitMode();
    my_nrf24l01p.enable();
    pc.printf( "Setup complete, Starting While loop\r\n");
  
    while (1) {

        float analogValue=sig;
        if ( pc.readable() ) {
            // ...add it to the transmit buffer
            txData[txDataCnt++] = pc.getc();
            if(analogValue > 0.0 && analogValue <= .3) {
                txData[txDataCnt++] = 'L'; 
            } else if(analogValue >.4 && analogValue <= .8) {
                txData[txDataCnt++] = 'C';
            } else if(analogValue >.8 && analogValue <= 1.0) {
                txData[txDataCnt++] = 'R';     
            }
            // If the transmit buffer is full
            if ( txDataCnt >= sizeof( txData ) ) {
                // Send the transmitbuffer via the nRF24L01+
                my_nrf24l01p.write( NRF24L01P_PIPE_P0, txData, sizeof( txData ));

            txDataCnt = 0;

            }
        }
    }
}

Connecting the OpenBCI Ganglion¶

The ganglion we are using was loaded with an older version of firmware, which needed to be updated in order to program it. Although we did not end up changing any code on the device, the instructions to update the firmware and to program the Simblee can be found here.

All of the information for setting up the Ganglion for recording, as well as the proper configuration of the electrodes can be found here. This project makes use of the Ganglion to collect EEG signals. This is done via the software that can be found here. There is an issue with the Ganglion compatible software, where you can not modify the code and build it from source due to a library packaging issue. Until this is resolved, live alpha wave capture using the ganglion will not be possible. The way we worked around this, was to capture our EEG signals in the newest software and save it to a file in the OpenBCI format. We could then load the recorded session into the older software, which we edited to detect alpha waves and transmit this information to the MBED controlling the car. Alpha waves can be reliably produced in most humans simply by closing the eyes. In the future, there are many avenues to produce control signals just from the motion of the face. We plan on integrating these in future iterations of this work. There is an example of detecting these motions that is located at cfausn's repo on github.

Processing¶

This project is currently only compatible with Processing 2.2.1. The following edits need to be made in the program you downloaded earlier. Open all of the files into processing and you should be presented with a structure that looks like this:

We will be editing the OpenBCI_GUI tab as well as the EEG_Processing tab. First in the OpenBCI_GUI tab make the following changes:

Create a new serial port at line 49:

Next, initialize the serial port at line 172:

If you have multiple serial devices connected to your computer, you will need to adjust the value picked from the Serial.list() command.

Now in the EEG_Processing tab at line 130 add in the control signals to send to the robot:

Another possible change that will be required to the code is to adjust the detection parameters for Alpha Waves. They are normally found in the 7.5–12.5 Hz range on most people, but will take some experimentation to find the correct values. In the sketch the detection is based on the alpha wave peak being larger than the other signals produced by the brain. Adjusting the detection threshold value will make it harder or easier for Alpha Waves to be detected.

The detection parameters are located at line 47:

You should now be able to run the GUI by pressing the play button at the upper left hand corner.

When starting the program, you will get feedback about the serial port being initialized.

Correct setup will return a screen that looks similar to this:

You can now load the file that was previously recorded.

After starting the stream and playing the file, if you got a good recording of your brain producing alpha waves, you will get detection triggers from the recording. In this particular recording, I was trying to produce alternating 15 second periods of alpha wave generation and non-alpha wave generation.

The edited processing files can be downloaded from this link.

Demo¶