Introduction

The Amazon Alexa is a personal voice assistant, which can automatically recognize the human voice input and transform them into some specific machine instructions. Currently there are several device platforms supporting this service, namely Amazon Echo and Amazon Echo Dot. Our team aim to use the Amazon Echo Dot and take benefit of its online Amazon Web Services server to create our unique instructions. The goal is to create a smart system that uses a servo-controlled mechanical part to flip the switch of the light in a room. The system consists of

• an Echo Dot to process voice input
• an mbed develop board to receive information from the AWS server and to drive the servo
• a servo to flip the switch and a 3D printed frame to support the servo

Our team members are Xingzhen Wu and Xingyuan Zhu

Voice Control

Amazon Echo Dot can take in the human voice input and process the audio with its online speech recognition engine. Users can use the Amazon developer console to create their own "skills" to interact with Dot and the voice recognition engine. Here we set the voice pattern that Alexa will be recognizing to trigger the "intent" function. Amazon Web Services (AWS) is also needed to create the detailed function ("Lambda Function")that will be processed when the intent functions are called. Here we set the Lambda function to send messages through socket to the mbed platform under the same LAN networks. Details of how to create these function are discussed in Code section.

Embedded System

Mbed Board

Mbed is the embedded systems processor that does all the computation and controls for the light switch. Here the mbed LPC1768 modules board is used to communicate with wifi module and output signals to control the servo.

Wifi Module

The Adafruit ESP8266 Huzzah board is used to connect the mbed to a network. The board communicate with mbed using I2C, whose wiring to can be found can be found below. More information about the ESP8266 and ESP8266 Huzzah can be found through the links.

Servo

A typical radio control (RC) servo has a drive wheel that is controlled by a PWM coded signal. The wheel moves around 90-180 degrees in normal operation. The HS-322HD servo being used is shown below.

Here we used some external parts and mounted them on the servo wheel to mechanically flip the light switch. A 3D printed frame is designed to mount the servo over the light switch. All measurements are taken using actual parts and the frame is printed by Georgia Tech Van Leer Senior Design Workshop.

Pictures of the 3D printed frame mounting on a wall switch.

Power Supply

A 4*1.5 V DC battery case is used to supply decent power.

Video Demo

Code and Setup

Amazon AWS Lambda Function

• Create an account on Amazon AWS. It has one year of free services for first time users.
• Log in to the AWS Management Console and navigate to AWS Lambda Function.
• Select region "US East (N. Virginia)" or "EU (Ireland)" to enable Lambda function services.
• Create a Lambda function with blank blueprint and a trigger set to "Amazon Skill Kits."
• Insert Lambda code as followed:

from __future__ import print_function
import socket

HOST = '143.215.99.177'  # The remote host
PORT = 80  # The same port as used by the server
# --------------- Helpers that build all of the responses ----------------------

def build_speechlet_response(title, output, reprompt_text, should_end_session):
    return {
        'outputSpeech': {
            'type': 'PlainText',
            'text': output
        },
        'card': {
            'type': 'Simple',
            'title': "SessionSpeechlet - " + title,
            'content': "SessionSpeechlet - " + output
        },
        '''reprompt': {
            'outputSpeech': {
                'type': 'PlainText',
                'text': reprompt_text
            }
        },
        '''
        'shouldEndSession': should_end_session
    }


def build_response(session_attributes, speechlet_response):
    return {
        'version': '1.0',
        'sessionAttributes': session_attributes,
        'response': speechlet_response
    }


# --------------- Functions that control the skill's behavior ------------------

def get_welcome_response():
    """ If we wanted to initialize the session to have some attributes we could
    add those here
    """
    session_attributes = {}
    card_title = "Welcome"
    speech_output = "You now get to voice control the light Switch!"
    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.
    reprompt_text = ""
    should_end_session = False
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))


def handle_session_end_request():
    card_title = "Session Ended"
    speech_output = "Switch to manual control." \
                    "Have a nice day! "
    # Setting this to true ends the session and exits the skill.
    should_end_session = True
    return build_response({}, build_speechlet_response(
        card_title, speech_output, None, should_end_session))


"""
def create_command_attributes(command):
    return {"command": command}
"""

def Light_on(intent, session):
    card_title = intent['name']
    session_attributes = {}
    should_end_session = False
        # session_attributes = create_command_attributes(command)
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    msg = '/on/run'
    msg = str(unichr(48 + len(msg))) + msg
   
    s.sendall(msg)
    payload = s.recv(1024)
    s.close()
    speech_output = "Ok.  Light is on "
    reprompt_text = ""
    should_end_session = False
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))


def Light_off(intent, session):
    card_title = intent['name']
    session_attributes = {}
    should_end_session = False
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    msg = '/off/run'
    msg = str(unichr(48 + len(msg))) + msg
    s.sendall(msg)
    payload = s.recv(1024)
    s.close()
    speech_output = "Ok.  I will turn off the light "
    reprompt_text = ""
    return build_response(session_attributes, build_speechlet_response(
        card_title, speech_output, reprompt_text, should_end_session))
# --------------- Events ------------------

def on_session_started(session_started_request, session):
    """ Called when the session starts """

    print("on_session_started requestId=" + session_started_request['requestId']
          + ", sessionId=" + session['sessionId'])


def on_launch(launch_request, session):
    """ Called when the user launches the skill without specifying what they
    want
    """

    print("on_launch requestId=" + launch_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # Dispatch to your skill's launch
    return get_welcome_response()


def on_intent(intent_request, session):
    """ Called when the user specifies an intent for this skill """

    print("on_intent requestId=" + intent_request['requestId'] +
          ", sessionId=" + session['sessionId'])

    intent = intent_request['intent']
    intent_name = intent_request['intent']['name']

    # Dispatch to your skill's intent handlers
    if intent_name == "Light_on":
        return Light_on(intent, session)
    elif intent_name == "Light_off":
        return Light_off(intent, session)
    elif intent_name == "AMAZON.HelpIntent":
        return get_welcome_response()
    elif intent_name == "AMAZON.CancelIntent":
        return handle_session_end_request()
    else:
        raise ValueError("Invalid intent")


def on_session_ended(session_ended_request, session):
    """ Called when the user ends the session.

    Is not called when the skill returns should_end_session=true
    """
    print("on_session_ended requestId=" + session_ended_request['requestId'] +
          ", sessionId=" + session['sessionId'])
    # add cleanup logic here


# --------------- Main handler ------------------

def lambda_handler(event, context):
    """ Route the incoming request based on type (LaunchRequest, IntentRequest,
    etc.) The JSON body of the request is provided in the event parameter.
    """
    print("event.session.application.applicationId=" +
          event['session']['application']['applicationId'])

    """
    Uncomment this if statement and populate with your skill's application ID to
    prevent someone else from configuring a skill that sends requests to this
    function.
    """
    # if (event['session']['application']['applicationId'] !=
    #         "amzn1.echo-sdk-ams.app.[unique-value-here]"):
    #     raise ValueError("Invalid Application ID")

    if event['session']['new']:
        on_session_started({'requestId': event['request']['requestId']},
                           event['session'])

    if event['request']['type'] == "LaunchRequest":
        return on_launch(event['request'], event['session'])
    elif event['request']['type'] == "IntentRequest":
        return on_intent(event['request'], event['session'])
    elif event['request']['type'] == "SessionEndedRequest":
        return on_session_ended(event['request'], event['session'])

• Configure the test event as followed:

{
  "session": {
    "sessionId": "amzn1.echo-api.session.[unique-value-here]",
    "application": {
      "applicationId": "amzn1.ask.skill.[unique-value-here]"
    },
    "attributes": {},
    "user": {
      "userId": "amzn1.ask.account.[unique-value-here]"
    },
    "new": true
  },
  "request": {
    "type": "IntentRequest",
    "requestId": "amzn1.echo-api.request.[unique-value-here]",
    "locale": "en-US",
    "timestamp": "2016-12-12T02:28:38Z",
    "intent": {
      "name": "DistTraveledIntent"
    }
  },
  "version": "1.0"
}

• Record the ARN number at the top right corner at the page.

Alexa Skills

• Sign in into Amazon Developer Consol
• Add a new skill following the instructions.
• Insert the intent schema as followed:

{
  "intents": [
    {
      "intent": "Light_on"
    },
    {
      "intent": "Light_off"
    },
    {
      "intent": "AMAZON.StopIntent"
    },
    {
      "intent": "AMAZON.HelpIntent"
    },
    {
      "intent": "AMAZON.CancelIntent"
    }
  ]
}

• Insert the sample utterances as followed:

Light_on Turn On The Light
Light_on Light On
Light_on Lights On
Light_on Let There Be Light
Light_on Turn The Light On
Light_on Switch On
Light_off Turn Off The Light
Light_off Turn The Light Off
Light_off Light Off
Light_off Lights Off
Light_off Switch Off
Light_off Bring Darkness
Light_off I Am Leaving
Light_off Leaving

• Configure the endpoint page and fill in the default text box with the ARN number recorded before.
• Test the skill with virtual input text.

Mbed Code

Future Improvements

• Separate the mbed board from the bread board and make it inside a case with the servo and the power supply.
• Add functionality that can disconnect the servo power so that users can manually flip the switch if they want. Could be done using mosfet switch board.
• Add more smart controls over the light. For example synchronizing with alarm to light up the room in the morning.

Special Thanks

We want to thank Professor Hamblen and lab TA Varun for their guidance and help over our project. Also we appreciate the assistance from ECE senior design lab and Kevin for helping us do the 3D printing. Special thanks for our friend Yiyang Zhou for providing advice over the flipping mechanism and mounting design.