4 years, 5 months ago.

Audio Sampling for Tensorflow Micro Speech Example

Hello everyone,

Thanks for reading! My question is, how can I modify the Adafruit Micro Speech Demo to get it to work on an mbed dev board?

https://learn.adafruit.com/tensorflow-lite-for-edgebadge-kit-quickstart/micro-speech-demo

Currently, I'm trying to run the Adafruit Micro Speech Demo on my Nucleo-F429ZI dev board, though it should really work on any Mbed enabled board.

I modified arduino_audio_provider.cpp and use Mbed os 5 functions. Here are the classes I used:

• Serial - logging predictions
• Ticker - setting sampling frequency
• AnalogIn - ADC from microphone
• EventQueue - avoid Mutex error
• Thread - enable separate thread for constant audio sampling

Here's how I initialize the audio sampling:

• float frequency = (float)1.0/ (float)16000.0;
• eventThread.start(callback(&q, &EventQueue::dispatch_forever));
• sampleAudioTicker.attach(q.event(&TIMER_CALLBACK), frequency);

In TIMER_CALLBACK, I use this:

• sample = (int16_t)(micout2.read_u16());

Full audio_provider code attached and here: https://github.com/jasonwhwang/tensorflow_micro_speech_mbed/blob/master/micro_speech/audio_provider.cpp /https:/os.mbed.com/media/uploads/jhwang51/audio_provider.cpp /https:/os.mbed.com/media/uploads/jhwang51/main.cpp /https:/os.mbed.com/media/uploads/jhwang51/main_functions.cpp

After replacing the Adafruit & Arduino functions with Mbed OS 5 functions, the dev board logs incorrect predictions due to the incorrectly sampled audio data. It would be great if I could make the correct modifications to get the speech recognition to work. I have tried modifying the sample frequency down to 12000HZ and accounting for the microphone bias, but the predictions are severely lagged and inaccurate.

I apologize for the vagueness of the question, but I would really appreciate any help and will answer any questions. Additional links are down below. Feel free to try it out on your own board by starting with my repository (it has all the necessary tensorflow library files).

Project Repository https://github.com/jasonwhwang/tensorflow_micro_speech_mbed

Tensorflow Repository https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro/examples/micro_speech

Dev Board https://os.mbed.com/platforms/ST-Nucleo-F429ZI/

Microphone https://learn.adafruit.com/adafruit-agc-electret-microphone-amplifier-max9814/overview

Thanks for the help! Jason