Tensorflow ESP32

 

Introducing ESP32: The Wi-Fi MCU

We are glad to announce TensorFlow Lite Micro support for the ESP32 chipset.

The ESP32 is a Wi-Fi/BT/BLE enabled MCU (micro-controller) that is widely used by hobbyists and makers to build cool and interesting projects that sense or modify real world data/object, and also commonly deployed in smart home appliances like light bulbs, switches, refrigerators, and air conditioners to provide connectivity.
The interesting part of ESP32 is that it’s a unique SoC that can be used right from quick prototypes to high-volume production. A wide community, numerous development kits, and plethora of tutorials/SDKs make it a great vehicle for quick prototypes with almost any vertical you might be interested in. The all-in-one package (Wi-Fi/BT/MCU) and existing high volume deployments in the field make it ideal for building end-products with.

ESP32 is already being used in a number of smart-home/connected-device projects with a variety of sensors and actuators connected to the microcontroller to sense the environment and act accordingly. With TensorFlow Lite for Microcontrollers executing on ESP32, this opens up scenarios for all kinds of use-cases that are triggered by local inference. ESP32 has 2 CPU cores and a bunch of optimizations, making it easier to run heavy TF Micro workfloads. The Wi-Fi backhaul helps to raise remote events and trigger actions based on the inferences made.

Person Detection or a Door-Bell Camera?

As an example, we have modified the person_detection example that you all might be familiar with to make it a smart door-bell camera. We use the ESP-EYE developer kit for this demonstration. Note that this example uses person detection (it detects when a face is in front of the camera), and not person identification (identifying who the person is).

The ESP-EYE dev-kit includes the ESP32 Wi-Fi/BT MCU coupled with a 2MP camera.

In Action

In our example, we will use this camera to observe and send out an email notification if we detect a person in the vicinity.

Building it for yourself

  1. Order the ESP-EYE: You can get the ESP-EYE Development Kit from your favourite distributor, or from here. You will need a USB to micro-USB cable for connecting this to your Windows/Linux/macOS host.
  2. Clone the repository: https://github.com/espressif/tensorflow/
  3. Setup your development host: Setup your development host with toolchains and utilities required to cross-build for ESP32. Follow the instructions of the ESP-IDF get started guide to set up the toolchain and the ESP-IDF itself.
  4. Generate the example: The example project can be generated with the following command:
    make -f tensorflow/lite/micro/tools/make/Makefile TARGET=esp generate_doorbell_camera_esp_project
  5. Build the example:

    a. Go to the example project directory
    cd tensorflow/lite/micro/tools/make/gen/esp_xtensa-esp32/prj/doorbell_camera/esp-idf

    b. Clone the esp32-camera component with following command:
    $ git clone https://github.com/espressif/esp32-camera components/esp32-camera

    c. Configure the camera and the email address:
    idf.py menuconfig

    d. Enter the Camera Pins configuration and SMTP Configuration menus to select the camera details, and also the email details.

    e. Build the example:
    idf.py build
  6. Flash and Run the program: Use the following command to flash and run the program:
    idf.py --port /dev/ttyUSB0 flash monitor
  7. Now, whenever a person’s face is detected, the program will send out an email to the configured email address.

What Next?

Now that you have tried the door bell camera example, you may try the other applications that are part of the TF Micro repository: hello_world and micro_speech.

ESP32 is pretty powerful for a microcontroller. Clocked at 240MHz, with just a single core it can do the detection well under 1 second (roughly ~700ms; additional optimizations are on the way to reduce this even further). This leaves the second core free for other tasks from your application.

The TinyML book is an excellent resource for a thorough understanding of TensorFlow Lite for Microcontrollers.
Diamond Article Divider
Community TensorFlow Lite

Comments

Post a Comment

Popular posts from this blog

Flutter for Single-Page Scrollable Websites with Navigator 2.0

Image
  Single page scrollable websites are everywhere. In this website design pattern, all the content is presented in one long-scrolling layout that contains multiple sections. Visitors can scroll or jump to a section by clicking buttons of a sticky menu. This pattern is a good fit for small content such as brochure websites, software library documentation, portfolios, and landing pages that are used to convert users [ 1 ]. Designers also love this pattern because it is simple, clean, and enables cool scroll animations. Jetbrains Mono single-page scrollable website Hive docs multi-page scrollable website In this series of articles, we will explore how to build a  single-page scrollable website  using Flutter. We will benefit from the Navigator 2.0 API to provide a good navigation experience to the users. You can find the source code of this series’ sample apps on my  Github page . The implementation details of the sample apps will be explained in the following articles: ...
Read more

A Data Science Portfolio is More Valuable than a Resume

Image
A Data Science Portfolio is More Valuable than a Resume Benjamin Obi Tayo Ph.D. Following Nov 20  · 5 min read D ata Science is a practical field. Hands-on skills are very important especially when you are interested in working outside academia as a practicing data scientist. In academia, you need more theoretical and research skills. While in-depth knowledge in the theoretical foundations of data science is important, as a practicing data scientist, hands-on experience is very crucial and one way to showcase you hands-on skills is via building a very great portfolio. Companies interested in hiring you would definitely be asking you for a portfolio, as it gives evidence of your strengths in fundamental data science concepts. While a data science resume is important, a data science portfolio is more valuable than a resume. This article will discuss 3 important platforms that can be used for portfolio buildi...
Read more

Better File Storage in Oracle Cloud

Image
Use Oracle Application Express with Oracle Cloud Infrastructure REST APIs to upload, download, and manage big files in Oracle Cloud. By Adrian Png November 27, 2019 Oracle Cloud Infrastructure  provides a comprehensive suite of REST APIs for accessing and managing cloud resources. For example, administrators can use REST APIs to create and manage autonomous databases and initiate backups. Oracle Application Express (Oracle APEX)  makes working with REST easy. The  APEX_WEB_SERVICE  package, available since Oracle APEX 4.0, provides developers with a function for consuming REST APIs. Oracle APEX 5.0 introduced the  APEX_JSON  package, allowing easy parsing of JSON responses. Oracle APEX 18.1 introduced the Web Source Module (WSM), which enables developers to access REST services and use the data in APEX components such as reports, interactive reports, and interactive grids. These features, when used together with Oracle Cloud Infrastructure REST ...
Read more