POV Display Controlled by Amazon Echo Dot

Project team: Frederick Lemuel Ravibabu, Madhurya Srinivas, Aneesa Sonawalla

Project Description

This project combines the Amazon Echo Dot with a classic persistence of vision (POV) display to make a digital clock of sorts that can display time, temperature, and location on command using the Echo Dot. The requests are voice-initiated (e.g. "Alexa, can you tell me the temperature?") and generate both an audio and a visual response: Alexa answers your question, and her answer is printed on the spinning LEDs of the display. A response will persist for 5 seconds on the display before it reverts back to the default mode, in which it multiplexes between time and date, again switching every 5 seconds.

Components & Setup

/media/uploads/asonawalla/full1.jpeg

Components Used

The motor is held upright with simple blocks of wood and a clamp. The spinning surface is part of the Sparkfun shadow chassis kit, repurposed for this project with small breadboards, the NeoPixel LEDs, and a battery pack attached to it. After all of the components were attached to the surface, it was balanced with counterweights taped wherever needed (i.e., between the two breadboards and to counter the battery pack) so that the display can spin smoothly.

The other major components of this project are the web apps used to complete the connection between the Echo Dot, the information requested, and the mbeds on the display. IFTTT, ngrok, and the Python Flask app are used in conjunction with Amazon Web Services to trigger a request with a voice command, get the requested data, and send that information back to the mbeds. To implement this project yourself, you will need an account on AWS and on IFTTT. The lambda function (see below) will be running on your AWS account, and then you will need to set up a trigger on IFTTT for an incoming voice command. The service ngrok is used to tunnel localhost to publicly accessible URL. Triggers are set up on the IFTTT service to send POST requests to the server set up from localhost. The trigger to IFTTT is the voice command to Alexa, and the action is a corresponding POST request to the server using the IFTTT Maker channel.

Spinning mbed

/media/uploads/asonawalla/mbed_spinning.jpeg

/media/uploads/asonawalla/xbee_spinning.jpeg

mbedNeoPixelXbeebattery pack
GNDGNDGNDGND
p20DIN
5VDC+
3.3V (VOUT)VCC
p10DOUT
p9DIN
p11RST

The spinning mbed controls the NeoPixel LEDs. In default mode, i.e. when there is no active request over the Echo Dot, it multiplexes the display between the date and the current time, showing each for 5 seconds before switching. When a request is made, the spinning mbed receives the response data from the stationary mbed via Zigbee and processes it to display it on the LEDs. The individual characters are hard-coded arrays that are accessed through the Characters class. This class does not include the full set of ASCII characters, but can easily be amended with any new characters needed.

Import programPOV_EchoControl

POV project controlled by Amazon Echo over WiFi

Stationary mbed

/media/uploads/asonawalla/stat_side.jpeg

mbedH-bridge (breadboard)XbeeHall sensorH-bridge (motor side)motor12V ext power
GNDGNDGNDGND
GNDGND
VIN+
OUTA+
OUTB-
p15white
5V (VU)VDDVCCred
p21INA
p25ENA/DIAGA
p23PWM
p20CS
p22INB
p10DOUT
p9DIN
p11RST

Note that there is also a 100 uF capacitor across VDD and GND on the breadboard pins of the H-bridge.

The stationary mbed remains connected via USB to a computer running the Python data passing code. When a request is made to the Echo Dot, the response data is sent serially to the stationary mbed (see Python code below for how this works), which then immediately forwards the received data over Zigbee to the spinning mbed. The stationary mbed also controls the motor speed and implements proportional feedback to try to steady the display by using the Hall effect sensor signal as a motor encoder. The feedback parameter and reference speeds can be tuned in the macros of the main file here as needed for different motors.

/media/uploads/asonawalla/motor_close.jpeg

The 8-pole magnet from the wheel encoder kit is attached to the bottom shaft of the motor, and the Hall effect sensor is placed such that it can detect this changing magnetic field as both the motor and the magnet spin. The bottom shaft spins more than once for 1 full rotation of the top motor shaft (which holds the spinning LEDs), which the motor control code accounts for. This motor encodes 30 counts on the sensor for each rotation of the top shaft.

Import programPOV_EchoControl_Stationarymbed

Code for motor control and data processing for POV display controlled by Amazon Echo (to run on non-spinning mbed)

Code for Amazon Echo Dot

Two programs are needed to connect the Echo Dot to the POV display and to transmit response data to the stationary mbed. The first is the lambda function, written in Javascript and implemented through Amazon Web Services. This is the function that tells the Echo Dot what to do for a given voice command. The second program is the Python Flask app that handles the GET and POST requests. It receives the response from the Amazon web services and passes the received data serially to the stationary mbed. Here, Weather Underground, which is a commercial weather service, is used to provide real-time weather information to the Echo Dot.

Lambda Function

Lambda function for AWS

//MAIN POV CODE

'use strict';

let https = require('https'),
  http = require('http');
  
const WundergroundApiKey = 'YOUR WUNDERGROUND API KEY HERE',
    WundergroundCity = 'YOUR CITY HERE',
    WundergroundState = 'YOUR STATE HERE (2-letter abbrev)';
    
var sendTime,sendLat,sendLong,sendElev,sendTemp;

 // Route the incoming request based on type (LaunchRequest, IntentRequest,
// etc.) The JSON body of the request is provided in the event parameter.
exports.handler = function (event, context) {
    try {
        console.log("event.session.application.applicationId=" + event.session.application.applicationId);
        
        if (event.session.new) {
            onSessionStarted({requestId: event.request.requestId}, event.session);
        }

        if (event.request.type === "LaunchRequest") {
            onLaunch(event.request,
                event.session,
                function callback(sessionAttributes, speechletResponse) {
                    context.succeed(buildResponse(sessionAttributes, speechletResponse));
                });
        } else if (event.request.type === "IntentRequest") {
            
            onIntent(event.request,
                event.session,
                function callback(sessionAttributes, speechletResponse) {
                    context.succeed(buildResponse(sessionAttributes, speechletResponse));
                });

        } else if (event.request.type === "SessionEndedRequest") {
            onSessionEnded(event.request, event.session);
            context.succeed();
        }
    } catch (e) {
        context.fail("Exception: " + e);
    }
};

/**
 * Called when the session starts.
 */
function getWeatherData(city, state, callback) {
  const req = http.request({
    hostname: 'api.wunderground.com',
    port: '80',
    path: '/api/'+WundergroundApiKey+'/astronomy/conditions/forecast10day/q/'+state+'/'+city+'.json',
    method: 'GET'
  }, (res) => {
    let body = '';
    res.setEncoding('utf8');
    res.on('data', (chunk) => body += chunk);
    res.on('end', () => {
      console.log('Successfully finished processing HTTP response');
      body = JSON.parse(body);
      console.log(body);
      sendTemp=body.current_observation.temp_f;
      sendLat=body.current_observation.display_location.latitude;
      sendLong=body.current_observation.display_location.longitude;
      sendElev=body.current_observation.display_location.elevation;
      callback(null, body);
    });
  });
  req.on('error', callback);
  req.end();
}
 
function onSessionStarted(sessionStartedRequest, session) {
    console.log("onSessionStarted requestId=" + sessionStartedRequest.requestId
        + ", sessionId=" + session.sessionId);
    
        // add any session init logic here
}


/**
 * Called when the user invokes the skill without specifying what they want.
 */
function onLaunch(launchRequest, session, callback) {
    console.log("onLaunch requestId=" + launchRequest.requestId
        + ", sessionId=" + session.sessionId);

    var cardTitle = "Hello, World!"
    var speechOutput = "You can tell Hello, World! to say Hello, World!"
    callback(session.attributes,
        buildSpeechletResponseWithoutCard("", "", "true"));
}

/**
 * Called when the user specifies an intent for this skill.
 */
function onIntent(intentRequest, session, callback) {
    console.log("onIntent requestId=" + intentRequest.requestId
        + ", sessionId=" + session.sessionId);

    var intent = intentRequest.intent,
        intentName = intentRequest.intent.name;

    // dispatch custom intents to handlers here
    if (intentName == 'DisplayTime') {
         new DisplayTime(intent, session, callback);
        
               
    }
    else if (intentName=='DisplayTemperature')
    {
        new DisplayTemperature(intent,session,callback);
    }
    else if (intentName=='DisplayLocation')
    {
        new DisplayLocation(intent,session,callback);
    }
    else
    {
        throw "Invalid intent";
    }
}

/**
 * Called when the user ends the session.
 * Is not called when the skill returns shouldEndSession=true.
 */
function onSessionEnded(sessionEndedRequest, session) {
    console.log("onSessionEnded requestId=" + sessionEndedRequest.requestId
        + ", sessionId=" + session.sessionId);

    // Add any cleanup logic here
}

function onSessionTest(sessionEndedRequest, session) {
    console.log("onSessionEnded requestId=" + sessionEndedRequest.requestId
        + ", sessionId=" + session.sessionId);

    // Add any cleanup logic here
}

function DisplayTime(intent, session, callback)
{
    var fullDate=new Date();
    
    var arr=fullDate.toString().split(" ")
    
    console.log(fullDate);
    
    var time=arr[4].toString().split(":");
    var hr=parseInt(time[0]);
    
    if(hr-4<0)
    {
        hr=hr+24-4;   
    }
    else
        hr=hr-4;
    
    var value1= hr;
    var value2= time[1];
    var value3= time[2];
    
    sendTime=hr+":"+time[1]+":"+time[2];
    
    var body='';
    
    getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{
            
        callback(session.attributes,
        buildSpeechletResponseWithoutCard("Time is "+hr+" "+time[1]+" and "+time[2]+" seconds", "", "true"));
        
     });
     
    
    var jsonObject2 = {
        'value1' : sendTime
    };
  
    // the post options
    const optionspost = {
        host: 'maker.ifttt.com',
        port:'80',
        path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz',
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
        }
    };

    const reqPost = http.request(optionspost, function(res) {
        console.log("statusCode: ", res.statusCode);
        res.setEncoding('utf8');
        res.on('data', function (chunk) {
            body += chunk;
        });
      
    });
    
    reqPost.write(JSON.stringify(jsonObject2));
    reqPost.end();    
   

}

function DisplayTemperature(intent, session, callback)
{
     var currentTemp='hello';
     var jsonObject;
     var data;
     getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{
            
            callback(session.attributes,
            buildSpeechletResponseWithoutCard("The temperature is "+sendTemp+" degrees fahrenheit", "", "true"));

     });
     
    var body='';
    
      jsonObject = {
                'value1' : sendTemp+"F"
                };
    // the post options
    const optionspost = {
        host: 'maker.ifttt.com',
        port:'80',
        path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz',
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
        }
    };

    const reqPost = http.request(optionspost, function(res) {
        console.log("statusCode: ", res.statusCode);
        res.setEncoding('utf8');
        res.on('data', function (chunk) {
            body += chunk;
        });
      
    });
    
    reqPost.write(JSON.stringify(jsonObject));
    reqPost.end();
}

function DisplayLocation(intent, session, callback)
{
     var jsonObject;
     
     getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{
            
            callback(session.attributes,
            buildSpeechletResponseWithoutCard("Your location is "+sendLat.substring(0,4)+" degrees north "+sendLong.substring(1,5)+" degrees west at an elevation of "+sendElev+" metres", "", "true"));

     });
     
    var body='';
    console.log(sendTemp);
    console.log(sendLat);
    var LocationString=sendLat.substring(0,4)+"N-"+sendLong.substring(1,5)+"W";
      jsonObject = {
                'value1' : LocationString
      }
    // the post options
    const optionspost = {
        host: 'maker.ifttt.com',
        port:'80',
        path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz',
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
        }
    };

    const reqPost = http.request(optionspost, function(res) {
        console.log("statusCode: ", res.statusCode);
        res.setEncoding('utf8');
        res.on('data', function (chunk) {
            body += chunk;
        });
      
    });
    
    reqPost.write(JSON.stringify(jsonObject));
    reqPost.end();
}

// ------- Helper functions to build responses -------

function buildSpeechletResponse(title, output, repromptText, shouldEndSession) {
    return {
        outputSpeech: {
            type: "PlainText",
            text: output
        },
        card: {
            type: "Simple",
            title: title,
            content: output
        },
        reprompt: {
            outputSpeech: {
                type: "PlainText",
                text: repromptText
            }
        },
        shouldEndSession: shouldEndSession
    };
}

function buildSpeechletResponseWithoutCard(output, repromptText, shouldEndSession) {
    return {
        outputSpeech: {
            type: "PlainText",
            text: output
        },
        reprompt: {
            outputSpeech: {
                type: "PlainText",
                text: repromptText
            }
        },
        shouldEndSession: shouldEndSession
    };
}

function buildResponse(sessionAttributes, speechletResponse) {
    return {
        version: "1.0",
        sessionAttributes: sessionAttributes,
        response: speechletResponse
    };
}

Python Flask app Code

Python program for passing data from Echo Dot

from flask import Flask, render_template, request, url_for, jsonify,current_app
app = Flask(__name__)

import serial
import time
import datetime

@app.route('/')
def hello_world():
    return 'Hello, Worlds!'


@app.route('/', methods=['GET', 'POST'])
def check():
    if request.method == 'POST':
serdev = '/dev/ttyACM0'
s = serial.Serial(serdev,9600)
s.write(request.data)
s.write("\r")
s.close()
        sendData=request.data

if(len(sendData)>12):
pattern = '%d.%m.%Y %H:%M:%S'
epoch = int(time.mktime(time.strptime(sendData, pattern)))
#s.write(request.data)
print sendData

else:
print sendData
return current_app.send_static_file('index.html')

    else:
return 'Good,bye'

Video Demo

Future Work

  • Create a sturdier mount for the display (more weight on the base, a firmer clamp for the motor)
  • Custom-design a balanced surface for the spinning mbed, or replace the current surface with a pre-made breadboard designed for POV displays
  • Tune the feedback loop using a program like Simulink to achieve a more stable display
  • Implement an analog clock (needs stable display to work)
  • Implement image display (need to map jpg/png/etc images from the internet to POV display bits)


Please log in to post comments.