From search engines to drug discoveries, machine learning has demonstrated remarkable success. Traditional machine learning systems are typically deployed on GPU and server-farms. Only recently, we are able to bring machine learning to personal devices making our lives even more convenient. However, the question is, can we scale it to even smaller platforms? Can this help to change IoT?

IoT devices are unique in a sense that they interface the digital and physical world. They have some special constraints:

• Deployed in large number: Cost-sensitive
• Power efficiency: Batteries and energy harvesting devices
• Bandwidth: Low Power and Wide Area network
• Limited computing resource: Clock speed and memory

These constraints seemly contradict the traditional deployment of AI. It was a common belief that machine learning was not practical for IoT edge nodes. However, recent advancements in algorithms, software, hardware have proven otherwise. We are learning how to represent information more efficiently, tweaking model complexities and working with more efficient software and hardware. This has allowed us to build uTensor, the first machine-learning-inference engine for microcontrollers.

uTensor workflow: the neural network is compiled into the application binary. The neural network is trained in Tensorflow. After device deployment, the model can be updated via FOTA.

uTensor is a framework that deploys Tensorflow trained models onto the Mbed microcontroller platform. It converts neural network models into C++ source code ready to be imported into your projects. It is both an open source project and a community that brings data scientists and embedded engineers together.

In an earlier post, I wrote about why we should bring Machine Learning to the edge. Today, IoT edge nodes go very little beyond forwarding sensory data to the cloud. The physical world is full of information. Machine learning capability on the edge means we can make sensors more powerful, conserve bandwidth, improve cost-efficiency and intelligence of the whole system. Future AI systems will understand our world more accurately and provide us with the insights we engineered them for.

I will be speaking at Mbed Connect at Arm TechCon on Tuesday October 16, 2018 in San Jose, CA about uTensor, its applications, and design considerations. We are excited to imagine what you can build with edge computing.

Find out more about Mbed Connect USA and the full day's of agenda here.