Users » apcastelein » Notebook » Alexa Inventory System

Alexa Inventory System

Page last updated 10 Dec 2016, by Arend Peter Castelein. 0 replies

Warehouse Navigator

With this project we've created a robot that's controlled by Alexa on the Amazon Echo Dot. We can issue commands such as "Get Coke", and then Alexa will send a path to the robot using a cloud 9 server and the robot will navigate to the inventory slot where coke is stored and it will fetch that inventory slot.

Team Members

Harmeet Bindra
Nikita Desai
Arend Peter Castelein

Setup

/media/uploads/apcastelein/robot_nav.png

The area has inventory slots and navigational markings. The navigational markings are pieces of colored paper and every time the robot's color sensor sees a new paper it will execute the next command in the sequence of the path. The individual commands can be forward, left, right, arm up, or arm down.

/media/uploads/apcastelein/robo_field.png

The inventory slots are made up of small cups which are held off the ground by cardboard boxes and chopsticks. Once the robot is under a cup it can raise it's arms to lift the cup from the boxes and then navigate back to the starting point.

Communication between Alexa and Mbed

The Alexa skill is written to take a user command in the format of "get/put [item]" and then send it to a Cloud9 server. The server will then update a database by either adding or removing an item from that slot (depending on if it was a put or get command). The server will also checks for cases where the user is trying to either get from and empty slot, or put on a full slot, it will send a error response back accordingly. If the command was valid, the server will then write the corresponding path to that item in a file (alexa.txt).

The mbed uses an ethernet connection to regularly check that file for changes. Once it sees an update it will execute that path.

Setting up Alexa Skill

You can refer to this notebook page to set up your alexa skill for this project (lambda function for warehouse navigator is provided below): https://developer.mbed.org/users/hbindra3/notebook/alexa-dot/

Mbed Components

The mbed uses the Adafruit TCS34725 color sensor, to read the colors on the floor https://developer.mbed.org/users/raj1995/notebook/adafruit-tcs34725-rgb-color-sensor/

It uses ethernet in order to obtain the pathing instructions https://developer.mbed.org/cookbook/Ethernet /media/uploads/apcastelein/xscreen_shot_2016-10-31_at_1.33.31_pm.png

Motors along with an h bridges are used to control the wheels of the robot as well as the arms. https://developer.mbed.org/cookbook/Motor

Robot pinout

=Mbed	H-Bridge	Motor
p21	PWMB
p22	BIN2
p23	BIN1
p24	AIN1
p25	AIN2
p26	PWMA
VOUT	VCC, STBY
VIN	VMOT
GND	GND
	AO1	Right Motor +
	AO2	Right Motor -
	BO1	Left Motor +
	BO2	Left Motor -

Mbed software

Import program Warehouse_navigator

Warehouse navigator robot controlled by amazon dot.

Mbed side code

#include "mbed.h"
#include "motordriver.h"
#include "EthernetNetIf.h"
#include "HTTPClient.h"
#include <string.h>
#define PI 3.14159

EthernetNetIf eth;
HTTPClient http;
HTTPText txt;
HTTPResult r;
Ticker flipper;
void flip();
I2C color_sensor(p28, p27); //pins for I2C communication (SDA, SCL)
int color_addr = 41 << 1;
int color_thresh = 460;//threshold for green before it triggers
bool on_color = false;
const char* command;
bool commandLocked = false;
float rotSpeed = .5;


//wheel motors
Motor left(p21, p20, p23, 1);
Motor right(p26, p25, p24, 1);

//arm motor
Motor arm(p22, p7, p6,1);

Timer t;
DigitalOut led1(LED1);
DigitalOut led2(LED2);
DigitalOut led3(LED3);
DigitalOut led4(LED4);

Serial pc(USBTX, USBRX);

//returns true when color is entered (subsequently returns false until color is exited and entered again)
bool colorEntered(){
    char red_reg[1] = {150};
    char red_data[2] = {0,0};
    color_sensor.write(color_addr,red_reg,1, true);
    color_sensor.read(color_addr,red_data,2, false);
    
    int red_value = ((int)red_data[1] << 8) | red_data[0];

    bool is_red = red_value > color_thresh;
    bool color_triggered = !on_color && is_red;
    on_color = is_red;
    //pc.printf("%d %d\r\n",red_value, color_triggered);
    return color_triggered;
}

void loadArm(){
    arm.speed(-.3);
    wait(2.7);
    arm.speed(0);        
}

//arem technology is yet to be determined, will probably involve adjusting a motor
void armUp(){
    arm.speed(.4);
    wait(2.5);
    arm.speed(0);            
}
void armDown(){
    arm.speed(-.4);
    wait(2.5);
    arm.speed(0);                
}


void turnRight() {
    led3 = 1;
    while(!colorEntered()){
        left.speed(rotSpeed);
        right.speed(rotSpeed);
        wait(.27);
        left.speed(0);
        right.speed(0);
        wait(1);
    }
    //while(!colorEntered()) wait(.01);
    wait(.1);
    led3 = 0;
}

void turnLeft() {
    led4=1;
    while(!colorEntered()){
        left.speed(-rotSpeed);
        right.speed(-rotSpeed);
        wait(.27);
        left.speed(0);
        right.speed(0);
        wait(1);
    }
    //while(!colorEntered()) wait(.01);
    wait(.1);
    led4 = 0;
}

void moveForward() {
    led2 = 1;
    t.start();
    int numEntered = 0;
    while(numEntered < 2) {
        left.speed(.4);
        right.speed(-.4);
        if(colorEntered()){
            numEntered++;    
            if(numEntered == 1){
                led3 = 1;    
            }
        }
    }
    t.stop();
    t.reset();
    left.speed(0);
    right.speed(0);
    led2 = 0;
    led3 = 0;
}

void HTTPGetCallbackEvent(HTTPResult result) {

    int iRetValue = http.getHTTPResponseCode();
    printf("HTTPGetCallbackEvent->Result: %d \r\n", iRetValue);
    if (result==HTTP_OK) {
        const char* temp = txt.gets();
        printf("Result ok : %s\r\n", temp);
        /*printf("got here\r\n");
        printf("Char test %c\r\n", 'C');
        printf("Char test %c\r\n", temp[0]);*/
        /*while(temp[i] != '\0'){
            printf("Char %c\r\n", temp[i]);
            command = temp;
            i++;
        }*/
        command = temp;
        printf("exit\r\n");
        //command[i] = '\0'; 
}
}

void initColorSensor(){
    pc.baud(9600);
    
    // Connect to the Color sensor and verify
    color_sensor.frequency(200000);
    
    char id_regval[1] = {146};
    char data[1] = {0};
    color_sensor.write(color_addr,id_regval,1, true);
    color_sensor.read(color_addr,data,1,false);

    // Initialize color sensor
    
    char timing_register[2] = {129,0};
    color_sensor.write(color_addr,timing_register,2,false);
    
    char control_register[2] = {143,0};
    color_sensor.write(color_addr,control_register,2,false);
    
    char enable_register[2] = {128,3};
    color_sensor.write(color_addr,enable_register,2,false);   
}

//initialization for all components
void init(){
      printf("entering init\n");
      loadArm();
    //initIMU();    
    initColorSensor();
    printf("exiting init\n");
}

// Sends a request to the server and waits for a path response
char* getPathFromServer(){
    //use wifi module here
    //https://developer.mbed.org/users/electromotivated/notebook/wifi-pid-redbot-robot-webserver/    
    char* path = "";
    return path;
}

void executePath(){
    commandLocked = true;
    printf("-------------------------------------executing %s\r\n",command);
    led1 = 1;
    int index = 0;
    bool done=false;
    while(!done && command[index] != '\0'){
        printf("-----------------------------------cmd: %c\r\n",command[index]);
        switch(command[index]){
            case 'F': moveForward(); break;    
            case 'L': turnLeft(); break;
            case 'R': turnRight(); break;
            case 'U': armUp(); break;
            case 'D': /*armDown();*/ break;
            default: printf("invalid char: %c\r\n",command[index]); 
            done=true;
            break;
        }
        index++;    
    }
    r = http.get("https://alexa-robot-hbindra3.c9users.io/started", &txt);
    command="-1";
    wait(1);
    led1 = 0;
    commandLocked = false;
}


int main() {
    led1 = 1;
    led2 = 1;
    led3 = 1;
    led4 = 1;
    init();   // initializes all the components
    led4 = 0;
      
    
    printf("Setting up...\r\n");
    EthernetErr ethErr = eth.setup();
    if (ethErr) {
        printf("Error %d in setup.\r\n", ethErr);
        return -1;
    }
    printf("Setup OK\r\n");

    http.setRequestHeader("Connection", "Keep-Alive");
    printf("Send message\r\n");
    
    led1 = 0;
    led2 = 0;
    led3 = 0;
    led4 = 0;
    /*char* command = "FRFURRFLF";
    executePath();
    r = http.get("https://alexa-robot-hbindra3.c9users.io/started", &txt);
    command="-1";
    flipper.attach(&flip, 2.0);
    while (1) {
        Net::poll();
        if(strcmp(command, "") == 0 || strcmp(command, "-1") == 0 || strcmp(command, "undefined") == 0) {
            printf("INVALID %s\r\n",command);
            continue;    
        } else {
            commandLocked = true;
            led1 = 1;
            led2 = 1;
            led3 = 1;
            led4 = 1;
            wait(1);
            led4 = 0;
            wait(1);
            led3 = 0;
            wait(1);
            led2 = 0;
            wait(1);
            led1 = 0;
            wait(1);
            executePath();    
        }
    }
}

void flip() {
    if(!commandLocked){
    r = http.get("https://sanmeetsingh.com/api/alexa.txt", &txt, HTTPGetCallbackEvent);
    //https://alexa-robot-hbindra3.c9users.io/
    }
}

Lambda function for Amazon Dot

var http = require('http')
 
exports.handler = (event, context) => {
 
  try {
 
    if (event.session.new) {
      // New Session
      console.log("NEW SESSION")
    }
 
    switch (event.request.type) {
 
      case "LaunchRequest":
        // Launch Request
        console.log(`LAUNCH REQUEST`)
        context.succeed(
          generateResponse(
            buildSpeechletResponse("Welcome to E C E 41 80", true),
            {}
          )
        )
        break;
 
      case "IntentRequest":
        // Intent Request
        console.log(`INTENT REQUEST`)
 
        switch(event.request.intent.name) {

        case "getBall":
            var endpoint = "http://alexa-robot-hbindra3.c9users.io/set/ball" // ENDPOINT GOES HERE
            var body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                var data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Getting ball`, true),
                    {}
                  )
                )
              })
            })
            break;
            
        case "putBall":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/put/ball" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                if(data == 100) {
                    context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Rack is full`, true),
                    {}
                  )
                )
                }
                 else {
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Putting ball`, true),
                    {}
                  )
                )
                 }
              })
            })
            break;
            
        case "getPens":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/set/pens" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Getting pens`, true),
                    {}
                  )
                )
              })
            })
             
            break;
            
        case "putPens":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/put/pens" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                if(data == 100) {
                    context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Rack is full`, true),
                    {}
                  )
                )
                }
                 else {
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Putting pens`, true),
                    {}
                  )
                )
                 }
              })
            })
            break;
        case "getCoke":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/set/coke" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                if(data == -1 || data == undefined) {
                    context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Coke is out of stock`, true),
                    {}
                  )
                )
                }
                else {
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse('Getting coke', true),
                    {}
                  )
                )
                }
              })
            })
            
            break;
            
        case "putCoke":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/put/coke" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                if(data == 100 || data == -1) {
                    context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Rack is full`, true),
                    {}
                  )
                )
                }
                 else {
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse("putting coke", true),
                    {}
                  )
                )
                 }
              })
            })
            break;
        case "getTablet":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/set/tablet" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Getting tablet`, true),
                    {}
                  )
                )
              })
            })
            
            break;
            
        case "putTablet":
            endpoint = "http://alexa-robot-hbindra3.c9users.io/put/tablet" // ENDPOINT GOES HERE
            body = ""
            http.get(endpoint, (response) => {
              response.on('data', (chunk) => { body += chunk })
              response.on('end', () => {
                data = body
                //var subscriberCount = data.items[0].statistics.subscriberCount
                if(data == 100) {
                    context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Rack is full`, true),
                    {}
                  )
                )
                }
                 else {
                context.succeed(
                  generateResponse(
                    buildSpeechletResponse(`Putting tablet`, true),
                    {}
                  )
                )
                 }
              })
            })
            break;
            
          default:
            throw "Invalid intent"
        }
 
        break;
 
      case "SessionEndedRequest":
        // Session Ended Request
        console.log(`SESSION ENDED REQUEST`)
        break;
 
      default:
        context.fail(`INVALID REQUEST TYPE: ${event.request.type}`)
 
    }
 
  } catch(error) { context.fail(`Exception: ${error}`) }
 
}
 
// Helper method to turn text to speech
buildSpeechletResponse = (outputText, shouldEndSession) => {
 
  return {
    outputSpeech: {
      type: "PlainText",
      text: outputText
    },
    shouldEndSession: shouldEndSession
  }
 
}
 
generateResponse = (speechletResponse, sessionAttributes) => {
 
  return {
    version: "1.0",
    sessionAttributes: sessionAttributes,
    response: speechletResponse
  }
 
}

Videos

[insert alexa interaction here]

Offline Demos

Please log in to post comments.

Alexa Inventory System

Warehouse Navigator

Team Members

Setup

Communication between Alexa and Mbed

Setting up Alexa Skill

Mbed Components

Mbed software

Import programWarehouse_navigator

Mbed side code

Lambda function for Amazon Dot

Videos

Alexa Inventory System

Important Information for this Arm website

Access Warning

Import program Warehouse_navigator