Introduction

The amazon dot is currently an affordable device integrated with the Amazon Web Services server. It is capable of voice processing, listening to a users input it responds appropriately. The appropriate response is determined by code called a 'skill' which can be customized to a users preference and this opens a plethora of applications. 'Alexa' is the name of the voice processor.

Two way communication is put into practice by making an Alexa voice controlled Remote Control Car. Alexa sends commands the the mbed, calling functions in mbed though RPC commands, and the mbed return status flags back to the server that are read out by Alexa.

This team consists of: Oluwagbemiga Mabogunje, Graham Miles and Stephan Wang.

Electronics

Mbed

Mbed is the embedded systems processor that does all the computation and controls for Dot Bot. The Mbed chip has many pins which are capable of analog and digital inputs in addition to digital, analog I2C, and PWM outputs. Mbed can do real-time processing to accept I/O inputs and make decisions to change robot motion according to these inputs.

Amazon Dot

The amazon dot Uses the Alexa Voice Service to play music, provide information, read the news, set alarms, control smart home devices, and more using just your voice. Always getting smarter and adding custom features and skills. In this project it listens for utterances programed into a custom skill and executes the appropriate lambda function, that in turn is programmed to communicate with the mbed.

ESP8266 Wifi Chip

The Wifi Chip is used to connect the mbed to a network unitizing an I2C connection. The wiring to the mbed can be found can be found below. More information about the chip can be found here

Motor Control with Hall Effect Sensors

The motors that move the robot are 'two-wire DC' motors. They are driven by an H-bridge driver circuit and are powered by an external power supply (a +6V battery pack). The H-bridge only supplies the output voltages for the two wires of the DC motor without regard for the physical load. Therefore, motors under different loads would spin robot wheels at different rates with the same output voltages.

To account for the differences between applied motor voltages and expected motor rotation, a hall effect sensor was attached to each motor. The Hall effect sensors provided feedback to Mbed by showing how much each motor had rotated. Using the sensor data, the robot actively make proportional corrections to straighten its forward movement.

Connections

Images

Demo Video

Code can be found below in procedure section

Procedure

This section walks the reader through the software steps that should be taken the achieve the same functionality achieved in in this project, assuming that an appropriately assembled and wired mbed controlled robot is being used.

mbed Program

The following program was written and used on the mbed.

Amazon Lambda Function

1. If you do not already have an account on AWS, go to Amazon Web Services and create an account.

2. Log in to the AWS Management Console and navigate to AWS Lambda.

3. Click the region drop-down in the upper-right corner of the console and select either US East (N. Virginia) or EU (Ireland). Note: Lambda functions for Alexa skills must be hosted in either the US East (N. Virginia) or EU (Ireland) region.

5. If you have no Lambda functions yet, click Get Started Now. Otherwise, click Create a Lambda Function.

6. Select a blank blueprint and trigger should be set to 'Alexa Skills Kit'.

7. The code in the 'Lambda Code' section at the end of the page should be used as the the the inline code:

8. Insert the following code in Actions -> Configure Test event :

{
  "session": {
    "sessionId": "amzn1.echo-api.session.[unique-value-here]",
    "application": {
      "applicationId": "amzn1.ask.skill.[unique-value-here]"
    },
    "attributes": {},
    "user": {
      "userId": "amzn1.ask.account.[unique-value-here]"
    },
    "new": true
  },
  "request": {
    "type": "IntentRequest",
    "requestId": "amzn1.echo-api.request.[unique-value-here]",
    "locale": "en-US",
    "timestamp": "2016-12-12T02:28:38Z",
    "intent": {
      "name": "DistTraveledIntent"
    }
  },
  "version": "1.0"
}

Take note of the ARN that can be see on the top right of the page.

More information on how to make a custom lambda function can be found here

Alexa Skill

5. Sign into the Amazon Developer portal

6. Navigate to Alexa -> Alexa Skills Kit -> Add New Skill and follow the instructions

7. The intent schema should resemble the following

{
  "intents": [
    {
      "intent": "ForwardIntent",
      "slots": [
        {
          "name": "Forward_Distance",
          "type": "AMAZON.NUMBER"
        }
      ]
    },
    {
      "intent": "BackwardIntent",
      "slots": [
        {
          "name": "Backward_Distance",
          "type": "AMAZON.NUMBER"
        }
      ]
    },
    {
      "intent": "TurnIntent",
      "slots": [
        {
          "name": "Turn_Degree",
          "type": "AMAZON.NUMBER"
        }
      ]
    },
    {
      "intent": "RightIntent"
    },
    {
      "intent": "LeftIntent"
    },
    {
      "intent": "DistTraveledIntent"
    },
    {
      "intent": "AMAZON.HelpIntent"
    }
  ]
}

The Sample Utterances determine what skill Alexa should carry out when certain phrases are heard. Some examples are as follows

ForwardIntent Go Forwards {Forward_Distance}
ForwardIntent Move Forwards {Forward_Distance}
ForwardIntent Go Forward {Forward_Distance}
ForwardIntent Move Forward {Forward_Distance}
BackwardIntent Go Backwards {Backward_Distance}
BackwardIntent Go Backward {Backward_Distance}
TurnIntent Turn {Turn_Degree}
TurnIntent Car Turn {Turn_Degree}
TurnIntent Robot Turn {Turn_Degree}
RightIntent Turn Right
RightIntent Right
LeftIntent Turn Left
LeftIntent Left
DistTraveledIntent How far have I traveled
DistTraveledIntent Whats my status
DistTraveledIntent What's my status

8. In configuration: Configure the Endpoint to be 'AWS Lambda ARN (Amazon Resource Name)'. In the text box enter the ARN of the lambda function

9. Proceed to the next page and test at will

Possible Improvements

• Looking into a way to make the Lambda function wait for a response as opposed to the instantaneous response received in this project
• Using an IMU to determine heading
• Make an ArcTangent function to go to specific grid location
• More detailed encoder feedback algorithm will account for movement error

Alternate Projects

• Build Libraries for and use the Dot's Bluetooth Capability

Lambda Code

"""
This sample demonstrates a simple skill built with the Amazon Alexa Skills Kit.
The Intent Schema, Custom Slots, and Sample Utterances for this skill, as well
as testing instructions are located at http://amzn.to/1LzFrj6

For additional samples, visit the Alexa Skills Kit Getting Started guide at
http://amzn.to/1LGWsLG
"""

from __future__ import print_function
import socket
HOST = '73.43.57.11'    # The remote host
PORT = 1035              # The same port as used by the server


# --------------- Helpers that build all of the responses ----------------------

def build_speechlet_response(title, output, reprompt_text, should_end_session):
    return {
        'outputSpeech': {
            'type': 'PlainText',
            'text': output
        },
        'card': {
            'type': 'Simple',
            'title': "SessionSpeechlet - " + title,
            'content': "SessionSpeechlet - " + output
        },
        'reprompt': {
            'outputSpeech': {
                'type': 'PlainText',
                'text': reprompt_text
            }
        },
        'shouldEndSession': should_end_session
    }


def build_response(session_attributes, speechlet_response):
    return {
        'version': '1.0',
        'sessionAttributes': session_attributes,
        'response': speechlet_response
    }


# --------------- Functions that control the skill's behavior ------------------

def get_welcome_response():
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """

    session_attributes = {}
    card_title = "Welcome"
    speech_output = "You now get to control the Dot Bot.  Cowabunga!"
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = "Please tell me your command.  For example, say, " \
                    "Move forward five."
    should_end_session = False
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))


def handle_session_end_request():
    card_title = "Session Ended"
    speech_output = "Thanks for hanging out with me and the dot bot." \
                    "Have a nice day! "
    # Setting this to true ends the session and exits the skill.
    should_end_session = True
    return build_response({}, build_speechlet_response(
        card_title, speech_output, None, should_end_session))

"""
def create_command_attributes(command):
    return {"command": command}
"""


def send_forward(intent, session):

    card_title = intent['name']
    session_attributes = {}
    should_end_session = False

    if intent['slots']['Forward_Distance']['value']:
        forward_distance = intent['slots']['Forward_Distance']['value']
        #session_attributes = create_command_attributes(command)
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect((HOST, PORT))
        msg = '/moveForward/run ' + forward_distance
        msg = str(unichr(48+len(msg))) + msg
        s.sendall(msg)
        payload = s.recv(1024)
        s.close()
        speech_output = "Ok.  I will Move the bot Forward " + \
                        forward_distance + " inches. Before this, I had moved " +\
                        payload + " total inches." 
        reprompt_text = "I'm bored. Give me something to do."
    else:
        speech_output = "I'm not sure what your command is. " \
                        "Please try again."
        reprompt_text = "I'm not sure what your command is. " \
                        "You can tell me your command by saying, " \
                        "my command is command name."
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
        
def send_backward(intent, session):

    card_title = intent['name']
    session_attributes = {}
    should_end_session = False

    if intent['slots']['Backward_Distance']['value']:
        backward_distance = intent['slots']['Backward_Distance']['value']
        #session_attributes = create_command_attributes(command)
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect((HOST, PORT))
        msg = '/moveBackward/run ' + backward_distance
        msg = str(unichr(48+len(msg))) + msg
        s.sendall(msg)
        payload = s.recv(1024)
        s.close()
        speech_output = "Ok.  I will Move the bot backward " + \
                        backward_distance + " inches. Before this, I had moved " +\
                        payload + " total inches." 
        reprompt_text = "I'm bored. Give me something to do."
    else:
        speech_output = "I'm not sure what your command is. " \
                        "Please try again."
        reprompt_text = "I'm not sure what your command is. " \
                        "You can tell me your command by saying, " \
                        "my command is command name."
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
        
def send_turn(intent, session):

    card_title = intent['name']
    session_attributes = {}
    should_end_session = False

    if intent['slots']['Turn_Degree']['value']:
        turn_angle = intent['slots']['Turn_Degree']['value']
        #session_attributes = create_command_attributes(command)
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect((HOST, PORT))
        msg = '/turn/run ' + float(turn_angle)/90.0*170.0
        msg = str(unichr(48+len(msg))) + msg
        s.sendall(msg)
        payload = s.recv(1024)
        s.close()
        speech_output = "Ok.  I will turn the bot " + \
                        turn_angle + " degrees. Before this, I had moved " +\
                        payload + " total inches." 
        reprompt_text = "I'm bored. Give me something to do."
    else:
        speech_output = "I'm not sure what your command is. " \
                        "Please try again."
        reprompt_text = "I'm not sure what your command is. " \
                        "You can tell me your command by saying, " \
                        "my command is command name."
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))

def send_right_turn(intent, session):

    card_title = intent['name']
    session_attributes = {}
    should_end_session = False
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    msg = '/turnRight/run'
    msg = str(unichr(48+len(msg))) + msg
    s.sendall(msg)
    payload = s.recv(1024)
    s.close()
    speech_output = "Ok.  I will turn the bot right" + \
                    " 90 degrees. Before this, I had moved " +\
                    payload + " total inches." 
    reprompt_text = "I'm bored. Give me something to do."
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
        
def send_left_turn(intent, session):

    card_title = intent['name']
    session_attributes = {}
    should_end_session = False
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    msg = '/turnLeft/run'
    msg = str(unichr(48+len(msg))) + msg
    s.sendall(msg)
    payload = s.recv(1024)
    s.close()
    speech_output = "Ok.  I will turn the bot left" + \
                    " 90 degrees. Before this, I had moved " +\
                    payload + " total inches." 
    reprompt_text = "I'm bored. Give me something to do."
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
        
def get_dist_traveled(intent, session):
    card_title = intent['name']
    session_attributes = {}
    reprompt_text = None
    should_end_session = False
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    msg = 'does not matter what this is'
    s.sendall(msg)
    payload = s.recv(1024)
    speech_output = "Hello adventurer. " + \
                    "You have traveled " + payload + " inches."
    s.close()

    # Setting reprompt_text to None signifies that we do not want to reprompt
    # the user. If the user does not respond or says something that is not
    # understood, the session will end.
    return build_response(session_attributes, build_speechlet_response(
        intent['name'], speech_output, reprompt_text, should_end_session))

# --------------- Events ------------------

def on_session_started(session_started_request, session):
    """ Called when the session starts """

    print("on_session_started requestId=" + session_started_request['requestId']
          + ", sessionId=" + session['sessionId'])


def on_launch(launch_request, session):
    """ Called when the user launches the skill without specifying what they
    want
    """

    print("on_launch requestId=" + launch_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # Dispatch to your skill's launch
    return get_welcome_response()


def on_intent(intent_request, session):
    """ Called when the user specifies an intent for this skill """

    print("on_intent requestId=" + intent_request['requestId'] +
          ", sessionId=" + session['sessionId'])

    intent = intent_request['intent']
    intent_name = intent_request['intent']['name']

    # Dispatch to your skill's intent handlers
    if intent_name == "ForwardIntent":
        return send_forward(intent, session)
    elif intent_name == "BackwardIntent":
        return send_backward(intent, session)
    elif intent_name == "TurnIntent":
        return send_turn(intent, session)
    elif intent_name == "RightIntent":
        return send_right_turn(intent, session)
    elif intent_name == "LeftIntent":
        return send_left_turn(intent, session)
    elif intent_name == "DistTraveledIntent":
        return get_dist_traveled(intent, session)
    elif intent_name == "AMAZON.HelpIntent":
        return get_welcome_response()
    elif intent_name == "AMAZON.CancelIntent" or intent_name == "AMAZON.StopIntent":
        return handle_session_end_request()
    else:
        raise ValueError("Invalid intent")


def on_session_ended(session_ended_request, session):
    """ Called when the user ends the session.

    Is not called when the skill returns should_end_session=true
    """
    print("on_session_ended requestId=" + session_ended_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # add cleanup logic here


# --------------- Main handler ------------------

def lambda_handler(event, context):
    """ Route the incoming request based on type (LaunchRequest, IntentRequest,
    etc.) The JSON body of the request is provided in the event parameter.
    """
    print("event.session.application.applicationId=" +
          event['session']['application']['applicationId'])

    """
    Uncomment this if statement and populate with your skill's application ID to
    prevent someone else from configuring a skill that sends requests to this
    function.
    """
    # if (event['session']['application']['applicationId'] !=
    #         "amzn1.echo-sdk-ams.app.[unique-value-here]"):
    #     raise ValueError("Invalid Application ID")

    if event['session']['new']:
        on_session_started({'requestId': event['request']['requestId']},
                           event['session'])

    if event['request']['type'] == "LaunchRequest":
        return on_launch(event['request'], event['session'])
    elif event['request']['type'] == "IntentRequest":
        return on_intent(event['request'], event['session'])
    elif event['request']['type'] == "SessionEndedRequest":
        return on_session_ended(event['request'], event['session'])