POV Display Controlled by Amazon Echo Dot
Project team: Frederick Lemuel Ravibabu, Madhurya Srinivas, Aneesa Sonawalla
Project Description
This project combines the Amazon Echo Dot with a classic persistence of vision (POV) display to make a digital clock of sorts that can display time, temperature, and location on command using the Echo Dot. The requests are voice-initiated (e.g. "Alexa, can you tell me the temperature?") and generate both an audio and a visual response: Alexa answers your question, and her answer is printed on the spinning LEDs of the display. A response will persist for 5 seconds on the display before it reverts back to the default mode, in which it multiplexes between time and date, again switching every 5 seconds.
Components & Setup
Components Used
- Amazon Echo Dot (will work with regular Echo as well)
- 2 x mbed LPC 1768
- 2 x Adafruit NeoPixel Stick
- 4 AA battery pack (6V total)
- 2 x XBee breakout boards
- Hall effect sensor & 8-pole magnet
- 12V DC motor
- 12V battery
- VNH5019 Motor Driver Carrier
The motor is held upright with simple blocks of wood and a clamp. The spinning surface is part of the Sparkfun shadow chassis kit, repurposed for this project with small breadboards, the NeoPixel LEDs, and a battery pack attached to it. After all of the components were attached to the surface, it was balanced with counterweights taped wherever needed (i.e., between the two breadboards and to counter the battery pack) so that the display can spin smoothly.
The other major components of this project are the web apps used to complete the connection between the Echo Dot, the information requested, and the mbeds on the display. IFTTT, ngrok, and the Python Flask app are used in conjunction with Amazon Web Services to trigger a request with a voice command, get the requested data, and send that information back to the mbeds. To implement this project yourself, you will need an account on AWS and on IFTTT. The lambda function (see below) will be running on your AWS account, and then you will need to set up a trigger on IFTTT for an incoming voice command. The service ngrok is used to tunnel localhost to publicly accessible URL. Triggers are set up on the IFTTT service to send POST requests to the server set up from localhost. The trigger to IFTTT is the voice command to Alexa, and the action is a corresponding POST request to the server using the IFTTT Maker channel.
Spinning mbed
mbed | NeoPixel | Xbee | battery pack |
---|---|---|---|
GND | GND | GND | GND |
p20 | DIN | ||
5VDC | + | ||
3.3V (VOUT) | VCC | ||
p10 | DOUT | ||
p9 | DIN | ||
p11 | RST |
The spinning mbed controls the NeoPixel LEDs. In default mode, i.e. when there is no active request over the Echo Dot, it multiplexes the display between the date and the current time, showing each for 5 seconds before switching. When a request is made, the spinning mbed receives the response data from the stationary mbed via Zigbee and processes it to display it on the LEDs. The individual characters are hard-coded arrays that are accessed through the Characters
class. This class does not include the full set of ASCII characters, but can easily be amended with any new characters needed.
Import programPOV_EchoControl
POV project controlled by Amazon Echo over WiFi
Stationary mbed
mbed | H-bridge (breadboard) | Xbee | Hall sensor | H-bridge (motor side) | motor | 12V ext power |
---|---|---|---|---|---|---|
GND | GND | GND | GND | |||
GND | GND | |||||
VIN | + | |||||
OUTA | + | |||||
OUTB | - | |||||
p15 | white | |||||
5V (VU) | VDD | VCC | red | |||
p21 | INA | |||||
p25 | ENA/DIAGA | |||||
p23 | PWM | |||||
p20 | CS | |||||
p22 | INB | |||||
p10 | DOUT | |||||
p9 | DIN | |||||
p11 | RST |
Note that there is also a 100 uF capacitor across VDD and GND on the breadboard pins of the H-bridge.
The stationary mbed remains connected via USB to a computer running the Python data passing code. When a request is made to the Echo Dot, the response data is sent serially to the stationary mbed (see Python code below for how this works), which then immediately forwards the received data over Zigbee to the spinning mbed. The stationary mbed also controls the motor speed and implements proportional feedback to try to steady the display by using the Hall effect sensor signal as a motor encoder. The feedback parameter and reference speeds can be tuned in the macros of the main file here as needed for different motors.
The 8-pole magnet from the wheel encoder kit is attached to the bottom shaft of the motor, and the Hall effect sensor is placed such that it can detect this changing magnetic field as both the motor and the magnet spin. The bottom shaft spins more than once for 1 full rotation of the top motor shaft (which holds the spinning LEDs), which the motor control code accounts for. This motor encodes 30 counts on the sensor for each rotation of the top shaft.
Import programPOV_EchoControl_Stationarymbed
Code for motor control and data processing for POV display controlled by Amazon Echo (to run on non-spinning mbed)
Code for Amazon Echo Dot
Two programs are needed to connect the Echo Dot to the POV display and to transmit response data to the stationary mbed. The first is the lambda function, written in Javascript and implemented through Amazon Web Services. This is the function that tells the Echo Dot what to do for a given voice command. The second program is the Python Flask app that handles the GET and POST requests. It receives the response from the Amazon web services and passes the received data serially to the stationary mbed. Here, Weather Underground, which is a commercial weather service, is used to provide real-time weather information to the Echo Dot.
Lambda Function
Lambda function for AWS
//MAIN POV CODE 'use strict'; let https = require('https'), http = require('http'); const WundergroundApiKey = 'YOUR WUNDERGROUND API KEY HERE', WundergroundCity = 'YOUR CITY HERE', WundergroundState = 'YOUR STATE HERE (2-letter abbrev)'; var sendTime,sendLat,sendLong,sendElev,sendTemp; // Route the incoming request based on type (LaunchRequest, IntentRequest, // etc.) The JSON body of the request is provided in the event parameter. exports.handler = function (event, context) { try { console.log("event.session.application.applicationId=" + event.session.application.applicationId); if (event.session.new) { onSessionStarted({requestId: event.request.requestId}, event.session); } if (event.request.type === "LaunchRequest") { onLaunch(event.request, event.session, function callback(sessionAttributes, speechletResponse) { context.succeed(buildResponse(sessionAttributes, speechletResponse)); }); } else if (event.request.type === "IntentRequest") { onIntent(event.request, event.session, function callback(sessionAttributes, speechletResponse) { context.succeed(buildResponse(sessionAttributes, speechletResponse)); }); } else if (event.request.type === "SessionEndedRequest") { onSessionEnded(event.request, event.session); context.succeed(); } } catch (e) { context.fail("Exception: " + e); } }; /** * Called when the session starts. */ function getWeatherData(city, state, callback) { const req = http.request({ hostname: 'api.wunderground.com', port: '80', path: '/api/'+WundergroundApiKey+'/astronomy/conditions/forecast10day/q/'+state+'/'+city+'.json', method: 'GET' }, (res) => { let body = ''; res.setEncoding('utf8'); res.on('data', (chunk) => body += chunk); res.on('end', () => { console.log('Successfully finished processing HTTP response'); body = JSON.parse(body); console.log(body); sendTemp=body.current_observation.temp_f; sendLat=body.current_observation.display_location.latitude; sendLong=body.current_observation.display_location.longitude; sendElev=body.current_observation.display_location.elevation; callback(null, body); }); }); req.on('error', callback); req.end(); } function onSessionStarted(sessionStartedRequest, session) { console.log("onSessionStarted requestId=" + sessionStartedRequest.requestId + ", sessionId=" + session.sessionId); // add any session init logic here } /** * Called when the user invokes the skill without specifying what they want. */ function onLaunch(launchRequest, session, callback) { console.log("onLaunch requestId=" + launchRequest.requestId + ", sessionId=" + session.sessionId); var cardTitle = "Hello, World!" var speechOutput = "You can tell Hello, World! to say Hello, World!" callback(session.attributes, buildSpeechletResponseWithoutCard("", "", "true")); } /** * Called when the user specifies an intent for this skill. */ function onIntent(intentRequest, session, callback) { console.log("onIntent requestId=" + intentRequest.requestId + ", sessionId=" + session.sessionId); var intent = intentRequest.intent, intentName = intentRequest.intent.name; // dispatch custom intents to handlers here if (intentName == 'DisplayTime') { new DisplayTime(intent, session, callback); } else if (intentName=='DisplayTemperature') { new DisplayTemperature(intent,session,callback); } else if (intentName=='DisplayLocation') { new DisplayLocation(intent,session,callback); } else { throw "Invalid intent"; } } /** * Called when the user ends the session. * Is not called when the skill returns shouldEndSession=true. */ function onSessionEnded(sessionEndedRequest, session) { console.log("onSessionEnded requestId=" + sessionEndedRequest.requestId + ", sessionId=" + session.sessionId); // Add any cleanup logic here } function onSessionTest(sessionEndedRequest, session) { console.log("onSessionEnded requestId=" + sessionEndedRequest.requestId + ", sessionId=" + session.sessionId); // Add any cleanup logic here } function DisplayTime(intent, session, callback) { var fullDate=new Date(); var arr=fullDate.toString().split(" ") console.log(fullDate); var time=arr[4].toString().split(":"); var hr=parseInt(time[0]); if(hr-4<0) { hr=hr+24-4; } else hr=hr-4; var value1= hr; var value2= time[1]; var value3= time[2]; sendTime=hr+":"+time[1]+":"+time[2]; var body=''; getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{ callback(session.attributes, buildSpeechletResponseWithoutCard("Time is "+hr+" "+time[1]+" and "+time[2]+" seconds", "", "true")); }); var jsonObject2 = { 'value1' : sendTime }; // the post options const optionspost = { host: 'maker.ifttt.com', port:'80', path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz', method: 'POST', headers: { 'Content-Type': 'application/json', } }; const reqPost = http.request(optionspost, function(res) { console.log("statusCode: ", res.statusCode); res.setEncoding('utf8'); res.on('data', function (chunk) { body += chunk; }); }); reqPost.write(JSON.stringify(jsonObject2)); reqPost.end(); } function DisplayTemperature(intent, session, callback) { var currentTemp='hello'; var jsonObject; var data; getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{ callback(session.attributes, buildSpeechletResponseWithoutCard("The temperature is "+sendTemp+" degrees fahrenheit", "", "true")); }); var body=''; jsonObject = { 'value1' : sendTemp+"F" }; // the post options const optionspost = { host: 'maker.ifttt.com', port:'80', path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz', method: 'POST', headers: { 'Content-Type': 'application/json', } }; const reqPost = http.request(optionspost, function(res) { console.log("statusCode: ", res.statusCode); res.setEncoding('utf8'); res.on('data', function (chunk) { body += chunk; }); }); reqPost.write(JSON.stringify(jsonObject)); reqPost.end(); } function DisplayLocation(intent, session, callback) { var jsonObject; getWeatherData(WundergroundCity, WundergroundState, (error, data)=>{ callback(session.attributes, buildSpeechletResponseWithoutCard("Your location is "+sendLat.substring(0,4)+" degrees north "+sendLong.substring(1,5)+" degrees west at an elevation of "+sendElev+" metres", "", "true")); }); var body=''; console.log(sendTemp); console.log(sendLat); var LocationString=sendLat.substring(0,4)+"N-"+sendLong.substring(1,5)+"W"; jsonObject = { 'value1' : LocationString } // the post options const optionspost = { host: 'maker.ifttt.com', port:'80', path: '/trigger/Test_web/with/key/IampRXW4OJ4soAyyoZWTz', method: 'POST', headers: { 'Content-Type': 'application/json', } }; const reqPost = http.request(optionspost, function(res) { console.log("statusCode: ", res.statusCode); res.setEncoding('utf8'); res.on('data', function (chunk) { body += chunk; }); }); reqPost.write(JSON.stringify(jsonObject)); reqPost.end(); } // ------- Helper functions to build responses ------- function buildSpeechletResponse(title, output, repromptText, shouldEndSession) { return { outputSpeech: { type: "PlainText", text: output }, card: { type: "Simple", title: title, content: output }, reprompt: { outputSpeech: { type: "PlainText", text: repromptText } }, shouldEndSession: shouldEndSession }; } function buildSpeechletResponseWithoutCard(output, repromptText, shouldEndSession) { return { outputSpeech: { type: "PlainText", text: output }, reprompt: { outputSpeech: { type: "PlainText", text: repromptText } }, shouldEndSession: shouldEndSession }; } function buildResponse(sessionAttributes, speechletResponse) { return { version: "1.0", sessionAttributes: sessionAttributes, response: speechletResponse }; }
Python Flask app Code
Python program for passing data from Echo Dot
from flask import Flask, render_template, request, url_for, jsonify,current_app app = Flask(__name__) import serial import time import datetime @app.route('/') def hello_world(): return 'Hello, Worlds!' @app.route('/', methods=['GET', 'POST']) def check(): if request.method == 'POST': serdev = '/dev/ttyACM0' s = serial.Serial(serdev,9600) s.write(request.data) s.write("\r") s.close() sendData=request.data if(len(sendData)>12): pattern = '%d.%m.%Y %H:%M:%S' epoch = int(time.mktime(time.strptime(sendData, pattern))) #s.write(request.data) print sendData else: print sendData return current_app.send_static_file('index.html') else: return 'Good,bye'
Video Demo
Future Work
- Create a sturdier mount for the display (more weight on the base, a firmer clamp for the motor)
- Custom-design a balanced surface for the spinning mbed, or replace the current surface with a pre-made breadboard designed for POV displays
- Tune the feedback loop using a program like Simulink to achieve a more stable display
- Implement an analog clock (needs stable display to work)
- Implement image display (need to map jpg/png/etc images from the internet to POV display bits)
Please log in to post comments.