Startup brings Machine Learning to sensors

A startup founded by Pete Warden,, former technical lead of Google’s TensorFlow Mobile team, has embedded Machine Learning (ML) systems into sensors. The idea behind Useful Sensors came from Warden’s realisation that the model of software frameworks that worked so well for ML adoption on some types of devices didn’t work as well on others. So he thought about how to make ML capabilities more accessible with modules as small and inexpensive as sensors.

Integrating ML into small devices is not something new. Many of the mobile phones we know combine these types of Artificial Intelligence (AI) systems, for example, to identify songs or set up modes of use for cameras, such as night vision. However, there are still challenges to incorporating AI into smaller devices. That’s where what we call TinyML comes in.

Put simply, TinyML takes Machine Learning solutions to small pieces of hardware and trains them for a specific function without having to take the training data to process it externally, e.g. in the cloud. This makes it possible to generate locally updated ML models on the device itself, which can, for example, train a smart speaker with a new trigger word or change an industrial process on the ML sensors themselves. This is real-time training at the edge, not in the cloud or on large servers running ML systems.

Pete Warden is considered a TinyML guru and coauthored the book “TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers“, a reference in the field.

Useful Sensors’ first product, the Person Sensor, uses a camera module pre-programmed with algorithms that detect nearby faces and returns, via a simple interface, information about how many there are and where they are about the device, as well as performing facial recognition. It is designed to input larger systems, for example, to activate a kiosk screen from sleep mode when someone approaches, mute a microphone when no one is present or direct fans to point at the nearest person. The sensor is available on the Sparkfun developer site for $10.

While thinking about how to put ML resources in the hands of more people, Warden was also concerned about the possible abuses that the proliferation of confidential data would generate via cameras and microphones. So he decided to move the ML processing from the central microcontroller to peripherals – in this case, sensors – whose attack surface would be very small, as they do not have shared memory with the rest of the system and can also be audited by third parties to ensure reliability. According to Warden, ML sensors are easier to develop and more secure.

Applications and trends in TinyML

There is a lot of demand for TinyML in various industries. For example, in the industrial segment, machines with ML sensors can learn to detect noises or vibrations that signal problems and send alerts so that an evaluation can be made before they stop working. TinyML can also be applied for refined tracking of items and goods on production lines. In healthcare, wearables with TinyML can, for example, process heart rate and temperature data locally or from insulin pumps to suggest appropriate courses of action, all without Internet access.

More broadly, the advancement of TinyML is also opening up a new perspective in the Internet of Things (IoT) field due to features such as low latency, effective bandwidth utilisation, data privacy and security, and cost reduction. These qualities of TinyML allow IoT devices to function reliably without consistent access to cloud services. In particular, in places with inadequate connectivity resources, TinyML can perform AI analytics locally, ensuring substantial benefits to IoT services.

According to ABI Research, TinyML vendors are democratising TinyML solutions at a rapid pace, with the TinyML Software-as-a-Service (SaaS) market predicted to surpass $220 million by 2022 and become a relevant component from 2025 onwards. By 2030, the TinyML SaaS and related professional services segment have the potential to become a billion-dollar market. Another indication of the expansion of TinyML solutions recently is the growth of the TinyML Foundation, which brings together most of the leading vendors in this field.

Sensing systems in environments and audio processing remain the most common applications of TinyML, with sound architectures accounting for nearly 50% of the market by 2022, according to ABI Research. “Any sensory data in environments will likely rely on an ML model,” points out David Lobina, artificial intelligence and machine learning research analyst at ABI Research. The personal and work devices sector will see the biggest growth soon.

However, there are also pitfalls in this field, for which ABI Research believes there are well-identified solutions. “The physical constraints of TinyML devices are genuine. These devices are suitable for small and compact ML models requiring software-level innovation for specific use cases. Software vendors will be the most active in the TinyML market,” explains Lobina.