Yet Another Billy Bass modification

Users » tombracer » Notebook » Yet Another Billy Bass modification

Yet Another Billy Bass modification

Page last updated 14 Aug 2010, by Tommi Laurila. 7 replies bass, billy

Introduction

The general idea is to make the singing fish Billy Bass the voice of the house, babbling out home automation data, weather forecasts, news and other stuff I come up with. I intend not to use prerecorded sounds, but instead let Billy (=the mbed) download things to say in form of wav-files from the home server.

This approach needs some form of IoT (Internet of Things) or WoT (Web of Things) ideology, because the nodes (server and mbed) needs to be able to communicate without direct user interaction.

It sure would be very nice to have fish on the wall babbling out all sorts of stuff all day, even when no-one's at home :). Have to remember to program him to be quiet at night.

Home server

Home automation (e.g. wav generation, image and video data) and other stuff require more computing power and storage than embedded systems can provide, so a pc-based server is required. It will be an Ubuntu Linux in server mode, as I already have one. The server will generate wav-files for Billy to say using Festivals' text2wave program. This is because there are no online services that provide wav generation in finnish. I haven't tested if the generated wav-files are in the right format for playing on the mbed.

Making sound files with Festivals text2wave gives Billy a somewhat dry and monotonic voice, but I'll have to make do until something better comes up.

The server has to be able to notify the mbed in some way that there is new stuff ready for playing every time the server has generated something new.

Billy (and the mbed)

Billy will get a brain transplant in the form of the mbed (as the other example on this site), which will be in charge of operations, ie. moving the fish and playing sounds.

The plan is to get the mbed to download the sound to play to its sd card memory from the home server, when the server notifies that there is something new to play. This could be done running a http server on the mbed, which could receive notifications from the server.

This means the mbed will need to run both a http server (for receiving notifications) and a client (for downloading sounds) at the same time.

I have no idea of how to implement some sort of lip-synch based on the soudn files for the fish yet.

All this sounds to be quite a lot of work for the mbed. Too much?

All kind of ideas, thougths, advice and comments are welcome!

Project update, Jan. 24, 2010

My Billy Bass brain surgery project is proceeding according to plan. I finally got my L293D motor control chip (ordered from England via Ebay) and a 1 GB Transcend microSD card with a miniSD and regular SD adapter. I used the miniSD adapter as a fancy breakout box for the breadboard, so I can use the regular when checking out the card in my laptop.

Ethernet is also working with the help of a RJ-45 jack robbed from an old network card. Here is the setup so far (click on the image for a bigger version):

As you can see, I'm an expert in soldering :) The miniSD adapter was a pain to solder. The miniSD (and micro and regular) pin layout can be found here. The RJ-45 connector is held in place with a rubber band, and as you also can see, it's been a while since I last used my breadboard (a 'little' dusty, eh?). Here's a closer look from the other side (click on the image for a bigger version):

As for the coding, I've been able to put together a program that connects to the Internet and downloads a wav-file and stores it on the sd card. I used the excellent examples found in the cookbook on how to use httpclient and writing to the sd card.

So far everything has gone very smooth thanks to the advice from forum members, the easy to understand examples of the cookbook and the easy to interface mbed itself.

Next task is to connect the L293D and make Billy move.

I ordered a ready made audio amp circuit via Ebay, and I'll try to make Billy speak when it arrives.

Project update, August 14, 2010

This project has come a long way since last update. Now Billy does what was planned (more or less successfully :) ), and he has transformed into Billy 'Frankenfish' Bass PSE where PSE stands for Power Saver Edition. The Frankenfish addition comes from the significantly enlarged circuit board that drives him. And the power saving comes from a relay circuit that turns the audio amps on and off when necessary (in earlier editions the amps were always on).

Billy's abilities

Reading the news

Billy reads out the daily news every day at 7.00 pm. This is done with some help from the home server that downloads an RSS feed from a domestic news site and parses out the news items. Then the server creates a wav-file from the text (using Festivals' text-to-speech) and puts it in a directory from where Billy can download the sound with his Httpclient. The server also creates a 'Billy Movement File, BMF' based on the text. The BMF file is simply a row of 0's and 1's, where a 1 is for every wovel and a 0 is for every other character (except for numbers which have their own number sequences depending on pronounciation).

The BMF file for the phrase 'Hello world!' would look like this: 01001001000. Space and special character codes as 0's.

Just as the wave sound the bmf file is put where Billy can download it.

After downloading the wave file and the bmf file Billy plays the sound using the renewed EA_wavplayer by Tom Coxon. At the same time a second ticker is started that feeds the bmf file data (the 0's and 1's) directly to the L293 DC motor driver chips inputs making Billy move his mouth in sync with the sound. Every 1 makes Billy's mouth open, and every 0 makes it close. Using this approach you get a satisfying quality lipsync after some fiddling with the ticker speed (as good as in dubbed movies) which is enough for me.

When playback is complete Billy notifies the server with a simple http get call to a php page that deletes the played sound from the server. The server then knows it is played and generates a new sound (and bmf file) to be played if there is something in queue in a Mysql database table.

Watching sauna temperatures

Because I live in Finland, which is the promised land of sauna (as you may have read in the news recently), Billy also calls out my sauna temperature changes every 10 degrees, starting at 40 degrees Celsius. This is because my sauna stove is heated with firewood (it has no electronics that I could hack into). Temperature readings are made by the home server using 1-wire sensors, and the speaking part is done just like the news part above.

Acting doorbell

Billy also works as a doorbell after I took apart the receiver unit (the one with the speaker) from a cheap wireless doorbell and bolted it on Billy's circuit board. When the doorbell button is pressed Billy sounds a 'ding-dong' sound which is stored on his SD card, and then he utters something like 'There's someone at the door, get up and open it!'. This sentence is also stored permanently on the SD card. The doorbell triggers an interruptIn that starts the sound (well, it actually just sets a flag, and then the main routine does the playing).

SMS-to-speech

Billy can also speak out SMS text messages sent to him. The messages actually go to the home server which has a mobile phone connected via serial port. The phone is operated by the Gnokii software.

Pictures and explanations (click on them to see 1200 pix versions)

The larger board is Billy 'Frankenfish' Bass's new brain. The small one on the right is the original brain. Talk about upgrade!

The sligthly elevated circuit board in the back corner is the audio amplifier circuit ripped from a computer speaker. It's great with volume control knob and all. You can see the audio output cable plug in front. The speaker element connects there. I found using unpluggable parts is very handy during debugging (it's easy to remove the board for inspection etc.).

The vertical positioned circuit board is from the doorbell receiver. I put it like that for better signal reception.

The Ethernet RJ-45 jack (robbed from an old network card) sits right beside the doorbell circuit.

You can also see that the mbed is cooled with a small fan. It's a 12 volt fan that runs on 5 volts for more quiet operation. The whole shabang is powered with an ac adapter taken from an ancient external HP cd burner. The ac adapter is great because it outputs both 12 volt and 5 volt voltages.

The SD 'breakout board' is in the front. It's simply an SD card adapter that houses the 2 GB microSD I use.

The beefy capacitors are needed for power reserve, because when the mbed activates the audio amp and the fish's DC motors at the same time it resets due to power lack without the two 3000uF caps.

The fish's motor cable connectors are beside the doorbell circuit. You can barely see the top of one of them behind the fan.

This is Billy's home: a vintage fishing creel ordered all the way from US via eBay(couldn't find one any closer). All components fit well inside the creel, and it's not too airtight for better cooling and ventilation. And it looks good on the shelf too. Power and network cables come out at the back (under the lid).

Here are all parts in the same picture. The speaker box is just a plastic foodstuff can which is not very good for audio quality, I have to try something better.

The tiny circuit in front is Billy's original brain.

Fried parts so far

Even though I suck at soldering and have a very vague understanding of analog electronics I have only managed to fry one microSD card so far (knock, knock). The SD cards seem to be very sensitive to voltage peaks.

Other problems

Darn, C++ is hard to master..

Future plans

Add tons of stuff to Billy's repertoire (after getting the basic stuff to work..).

The program

Here's the program. Feel free to look/use/copy/improve/other at your own risk.

Billy2

7 comments

Vlad Cazan

# 01 Mar 2010

Good stuff! Lets get that L293D into action and billy will be alive!

Todd Birkenholtz

# 21 May 2010

How do you plan to sync up the sound with the movement?

Dan Ros

# 21 May 2010

I can reveal that the secret to the mouth/voice synchronisation in the mbed billy was a serverside script incorporating something like this: http://cpansearch.perl.org/src/GREGFAST/Lingua-EN-Syllable-0.251/Syllable.pm (but in python)

The result was converted to a 'movement file' which roughly lined up mouth movements with the text by the simple rule of dividing the syallables by the time duration of the recording, since if it's machine read you can assume its evenly timed throughout the recording. As for the body, I just got him to flap somewhat randomly in between words.

Todd Birkenholtz

# 26 May 2010

Great Idea... I was thinking about routing the audio back into a Analog in and create some type of vu meter ie.. high level=mouth open low level= mouth closed

I think this should work for speech but when you play music the mouth would be open most of the time.

What do you think?

Tommi Laurila

# 26 May 2010

My project has finally reached "beta" phase and is operational. I just haven't got around to update my notebook (coming soon..) about it.

I implemented my own lipsync method, which goes something like this: first the server reads its database checking for new things to say (everything Billy says is stored in a database on my home server). If it finds an unprocessed message, it creates a wave file from it (using Festival's text2wave) and stores it in a folder Billy can access (using the http client). The home server runs a www-server serving the files for Billy.

The server also creates a "Billy movement file" based on the read message and puts it in the same folder as the wave file. The movement file is constructed by analyzing the message read from the database. It's basically a text file with a series of 0's and 1's. A one (1) is written for every wovel, and a zero for every consonant or other character.

Billy then downloads the wave file and the movement file to its SD card. After downloading Billy plays the wave file (using the waveplayer code found here) and reads the movement file simultaneously (using tickers). The ones and zeros of the movement file are then sent straight to the output pins that control the L293D chip. This way Billy opens his mouth for every wovel, and shuts it for every other character. With this method I get an satisfying lipsync (same quality as dubbed movies :) ). Timing the ticker improves accuracy also.

I'll post a better explanation with images when I get some time to do it.

I hope you get at least some idea of how it works.

Randy Russo

# 11 Jun 2010

Tommi, great to see another Billy Bass. I am modding Frankie the fish, which should actually be easier (less motors), but we'll see how it goes. I really like the idea of the lip synch. I'll "borrow" that one. Did you use the aforementioned python script to get the syllables out? I'm going to run into the same issue, or did you just implement the vowel mouth open in code by text string searching? I hope I can get this running enough to help and not just leech the info. Severe Noob here, but I have smart friends...

Tommi Laurila

# 11 Jun 2010

Hi Randy,

and good luck with your fish project! It's really great to see the fish coming alive (it's aaaaliiiiiveee.. IT'S AAAAALIIIVVEEEEHHHH... [Dr. Frankenstein pronounciation]).

I didn't use the python script, I just coded a function in php (command line php in Linux is great!) that reads every character of the message one by one and adds a 1 or a 0 to a string that is written to the movement file. The movement file for a message would look like this (message above, movement file below):

H	e	l	l	o		w	o	r	l	d	!
0	1	0	0	1	0	0	1	0	0	0	0

This way the data in the movement file is easy to push out on the io-pins that control the motor driver chip.

Most Billy Bass examples I have found on the net are based on an older version of Billy, when he had three motors. My newer version has only two motors: one for the mouth, and one for body/tail (changing motor turning direction flaps body or tail).

Future improvements include adding some random body and tail flaps here and there in some way.

My biggest problems was to get a strong enough audio output signal. Luckily I got the help needed from the forum here (thanks to all once again!). I'm quite a noob at this myself too.

Feel free to copy as much as you like! (I did it too :-) )

- Tommi

You need to log in to post a comment