Embedded Coding

Keyword Spotting on ESP32-C3-Lyra V2 Using ESP-IDF

noreply@blogger.com (Maxwell Clarke) — Fri, 30 Jan 2026 23:13:00 +0000

ESP32-C3 Keyword Spotting (TFLite Micro micro_speech) with Onboard Mic (ADC)

ESP32-C3 Keyword Spotting (“Yes/No”) with TFLite Micro `micro_speech` using the Onboard Mic (ADC)

This post documents how to run TensorFlow Lite for Microcontrollers (TFLM) (now branded as LiteRT for Microcontrollers) keyword spotting example (micro_speech) on an ESP32-C3 board, and how to adapt the example to use the onboard analog microphone routed through the ESP32-C3 ADC. To set up the ESP32-C3-lyra V2, see this post: Hello World on ESP-32-C3-Lyra V2.0

Environment / Versions

Target board: ESP32-C3 (ESP32-C3-Lyra)
ESP-IDF version: v6.x (or v6.0-dev)
Example project: esp-tflite-micro:micro_speech (keyword spotting “yes/no”)

Note (ESP-IDF v6+): ESP-IDF v6 removed the legacy ADC header (driver/adc.h) and renamed some ADC attenuation enums. The ADC code below reflects those v6+ changes.

Command Log

1) Load the ESP-IDF environment

. $HOME/esp/esp-idf/export.sh

2) Create a new project from the `micro_speech` example

cd ~
idf.py create-project-from-example "espressif/esp-tflite-micro=1.3.0:micro_speech"
mv ~/micro_speech ~/keyword_spotting_tflm

3) Set the target to ESP32-C3

cd ~/keyword_spotting_tflm && idf.py set-target esp32c3

4) Install/verify ESP-IDF tools for ESP32-C3 (v6+ toolchain)

python3 $IDF_PATH/tools/idf_tools.py install --targets esp32c3
. $HOME/esp/esp-idf/export.sh

5) Build

cd ~/keyword_spotting_tflm && idf.py build

6) Flash and open the serial monitor

cd ~/keyword_spotting_tflm && idf.py -p /dev/ttyUSB0 flash monitor

Exit the monitor: Press Ctrl + ]

Troubleshooting

Issue: Default example tried I2S and failed

The upstream micro_speech project’s audio capture path attempted to configure I2S pins (I2S microphone). On this setup, we used the onboard mic through ADC instead. The following error occurs:

E (...) i2s_set_pin(...): bck_io_num invalid
E (...) TF_LITE_AUDIO_PROVIDER: Error in i2s_set_pin

Fix: Switch audio capture from I2S to ADC continuous sampling

Key points of the ADC implementation:

Use esp_adc/adc_continuous.h continuous mode to sample at 16 kHz.
Convert 12-bit unsigned ADC samples into signed 16-bit PCM-like samples centered around mid-scale.
Write samples into the existing ring buffer so the model’s GetAudioSamples() continues to work.

Replace audio_provider.cc with the working ADC version (ESP32-C3 TYPE2) by replacing the entire contents of:

~/keyword_spotting_tflm/main/audio_provider.cc

with the following code:

/* ADC-based audio provider for ESP32-C3-Lyra (MIC_ADC on IO0 / ADC1 CH0) */

#include "audio_provider.h"

#include <cstring>

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

#include "esp_log.h"

#include "esp_adc/adc_continuous.h"

#include "ringbuf.h"
#include "micro_model_settings.h"

static const char* TAG = "TF_LITE_AUDIO_PROVIDER";

ringbuf_t* g_audio_capture_buffer;
volatile int32_t g_latest_audio_timestamp = 0;

constexpr int32_t history_samples_to_keep =
    ((kFeatureDurationMs - kFeatureStrideMs) * (kAudioSampleFrequency / 1000));
constexpr int32_t new_samples_to_get =
    (kFeatureStrideMs * (kAudioSampleFrequency / 1000));

const int32_t kAudioCaptureBufferSize = 40000;

namespace {
int16_t g_audio_output_buffer[kMaxAudioSampleSize * 32];
bool g_is_audio_initialized = false;
int16_t g_history_buffer[history_samples_to_keep];

adc_continuous_handle_t g_adc_handle = NULL;

// Read buffer (raw ADC frames)
static constexpr size_t kAdcReadBytes = 1024;
uint8_t g_adc_read_buf[kAdcReadBytes];

// Temporary PCM buffer (int16)
int16_t g_pcm_buf[kAdcReadBytes / sizeof(adc_digi_output_data_t)];

// ESP32-C3-Lyra MIC_ADC is routed to IO0 => ADC1 channel 0
static constexpr adc_unit_t kAdcUnit = ADC_UNIT_1;
static constexpr adc_channel_t kAdcChannel = ADC_CHANNEL_0;
static constexpr adc_atten_t kAdcAtten = ADC_ATTEN_DB_12;
static constexpr adc_bitwidth_t kAdcBitwidth = ADC_BITWIDTH_12;
}  // namespace

static void adc_init_continuous() {
  adc_continuous_handle_cfg_t handle_cfg = {
      .max_store_buf_size = 4096,
      .conv_frame_size = 1024,
  };
  ESP_ERROR_CHECK(adc_continuous_new_handle(&handle_cfg, &g_adc_handle));

  adc_digi_pattern_config_t pattern = {};
  pattern.atten = kAdcAtten;
  pattern.channel = kAdcChannel;
  pattern.unit = kAdcUnit;
  pattern.bit_width = kAdcBitwidth;

  adc_continuous_config_t dig_cfg = {};
  dig_cfg.sample_freq_hz = kAudioSampleFrequency;  // 16 kHz
  dig_cfg.conv_mode = ADC_CONV_SINGLE_UNIT_1;

  // ESP32-C3 DMA output uses TYPE2 layout
  dig_cfg.format = ADC_DIGI_OUTPUT_FORMAT_TYPE2;

  dig_cfg.pattern_num = 1;
  dig_cfg.adc_pattern = &pattern;

  ESP_ERROR_CHECK(adc_continuous_config(g_adc_handle, &dig_cfg));
  ESP_ERROR_CHECK(adc_continuous_start(g_adc_handle));
}

static inline int16_t adc12_to_pcm16(uint16_t adc12) {
  int32_t centered = (int32_t)adc12 - 2048;
  int32_t pcm = centered << 4;  // scale 12-bit to ~16-bit
  if (pcm > 32767) pcm = 32767;
  if (pcm < -32768) pcm = -32768;
  return (int16_t)pcm;
}

static void CaptureSamples(void* arg) {
  adc_init_continuous();

  while (true) {
    uint32_t out_bytes = 0;
    esp_err_t ret = adc_continuous_read(
        g_adc_handle, g_adc_read_buf, kAdcReadBytes, &out_bytes, pdMS_TO_TICKS(200));

    if (ret == ESP_OK && out_bytes > 0) {
      const size_t n_frames = out_bytes / sizeof(adc_digi_output_data_t);

      for (size_t i = 0; i < n_frames; i++) {
        const adc_digi_output_data_t* p =
            (const adc_digi_output_data_t*)(g_adc_read_buf +
                                            i * sizeof(adc_digi_output_data_t));

        // ESP32-C3 uses type2 layout (type1 will not compile)
        uint16_t raw = (uint16_t)(p->type2.data);

        g_pcm_buf[i] = adc12_to_pcm16(raw);
      }

      const int bytes_to_write = (int)(n_frames * sizeof(int16_t));
      const int bytes_written = rb_write(g_audio_capture_buffer,
                                         (uint8_t*)g_pcm_buf,
                                         bytes_to_write,
                                         pdMS_TO_TICKS(200));

      if (bytes_written > 0) {
        const int samples_written = bytes_written / (int)sizeof(int16_t);
        g_latest_audio_timestamp += (1000 * samples_written) / kAudioSampleFrequency;
      }
    }

    if (ret != ESP_OK && ret != ESP_ERR_TIMEOUT) {
      ESP_LOGE(TAG, "adc_continuous_read failed: %s", esp_err_to_name(ret));
      vTaskDelay(pdMS_TO_TICKS(50));
    }
  }
}

TfLiteStatus InitAudioRecording() {
  g_audio_capture_buffer = rb_init("tf_ringbuffer", kAudioCaptureBufferSize);
  if (!g_audio_capture_buffer) {
    ESP_LOGE(TAG, "Error creating ring buffer");
    return kTfLiteError;
  }

  xTaskCreate(CaptureSamples, "CaptureSamples", 1024 * 4, NULL, 10, NULL);

  while (!g_latest_audio_timestamp) {
    vTaskDelay(1);
  }

  ESP_LOGI(TAG, "Audio Recording started (ADC continuous)");
  return kTfLiteOk;
}

TfLiteStatus GetAudioSamples1(int* audio_samples_size, int16_t** audio_samples) {
  if (!g_is_audio_initialized) {
    TfLiteStatus init_status = InitAudioRecording();
    if (init_status != kTfLiteOk) {
      return init_status;
    }
    g_is_audio_initialized = true;
  }

  int bytes_read =
      rb_read(g_audio_capture_buffer, (uint8_t*)(g_audio_output_buffer), 16000, 1000);
  if (bytes_read < 0) {
    ESP_LOGI(TAG, "Couldn't read data in time");
    bytes_read = 0;
  }
  *audio_samples_size = bytes_read;
  *audio_samples = g_audio_output_buffer;
  return kTfLiteOk;
}

TfLiteStatus GetAudioSamples(int start_ms, int duration_ms,
                             int* audio_samples_size, int16_t** audio_samples) {
  if (!g_is_audio_initialized) {
    TfLiteStatus init_status = InitAudioRecording();
    if (init_status != kTfLiteOk) {
      return init_status;
    }
    g_is_audio_initialized = true;
  }

  memcpy((void*)(g_audio_output_buffer), (void*)(g_history_buffer),
         history_samples_to_keep * sizeof(int16_t));

  int bytes_read =
      rb_read(g_audio_capture_buffer,
              ((uint8_t*)(g_audio_output_buffer + history_samples_to_keep)),
              new_samples_to_get * sizeof(int16_t), pdMS_TO_TICKS(200));

  if (bytes_read < 0) {
    ESP_LOGE(TAG, "Model could not read data from Ring Buffer");
  }

  memcpy((void*)(g_history_buffer),
         (void*)(g_audio_output_buffer + new_samples_to_get),
         history_samples_to_keep * sizeof(int16_t));

  *audio_samples_size = kMaxAudioSampleSize;
  *audio_samples = g_audio_output_buffer;
  return kTfLiteOk;
}

int32_t LatestAudioTimestamp() { return g_latest_audio_timestamp; }

Issue: Missing header `esp_adc/adc_continuous.h`

After adding the include, the build failed with:

fatal error: esp_adc/adc_continuous.h: No such file or directory

Fix: Add the `esp_adc` component dependency

Edit main/CMakeLists.txt to include esp_adc to PRIV_REQUIRES (or REQUIRES):

nano ~/keyword_spotting_tflm/main/CMakeLists.txt

idf_component_register(
  SRCS ...
  INCLUDE_DIRS .
  PRIV_REQUIRES esp_adc
)

Issue: `adc_digi_output_data_t` had no `type1` on ESP32-C3

Build error:

error: 'const struct adc_digi_output_data_t' has no member named 'type1'

Fix: Use the ESP32-C3 struct layout (TYPE2)

Make the following changes in the file keyword_spotting_tflm/main/audio_provider.cc:

ADC_DIGI_OUTPUT_FORMAT_TYPE1 → ADC_DIGI_OUTPUT_FORMAT_TYPE2
p->type1.data → p->type2.data

Next, rebuild and reflash:

cd ~/keyword_spotting_tflm && idf.py build
cd ~/keyword_spotting_tflm && idf.py -p /dev/ttyUSB0 flash monitor

Issue: Toolchain version mismatch on ESP-IDF v6+

If the build fails with a toolchain mismatch (e.g., expected esp-15.2.0_20250929), install the ESP32-C3 toolchain:

python3 $IDF_PATH/tools/idf_tools.py install --targets esp32c3
. $HOME/esp/esp-idf/export.sh

Issue: `idf.py fullclean` refuses

If idf.py fullclean refuses to delete the build directory, delete it manually:

cd ~/keyword_spotting_tflm
rm -rf build
idf.py build

After switching the audio provider to ADC and aligning the ADC DMA output format for ESP32-C3, the application ran successfully and recognized the keywords “yes” and “no” over serial output. The next post will include customization for keyword spotting with additional words.

Hello World on ESP32-C3-Lyra V2.0 Using ESP-IDF

noreply@blogger.com (Maxwell Clarke) — Tue, 20 Jan 2026 19:25:00 +0000

Flashing “Hello World” to an ESP32-C3-Lyra V2.0 on Linux (ESP-IDF)

This post serves as a tutorial for how to build and flash the official ESP-IDF hello_world example to an ESP32-C3-Lyra-V2.0 from a Linux machine, then verify output over UART. This tutorial works for any ESP32-C3. Additionally, this is a precursor to the next post in this series: Keyword Spotting using ESP32-C3-Lyra V2.0

prerequisites:

Hardware

ESP32-C3 development board
USB cable
USB-to-UART bridge presented as CP2102N (common on many ESP32 boards)

Software

Ubuntu 22.04
ESP-IDF v5.2.3 (installed from source)

Step 1: Verify the board appears as a serial device

Plug the board in over USB and check the kernel log:

sudo dmesg -T | tail -n 30

You should see something indicating a USB-to-UART bridge and the assigned device node, for example:

CP2102N USB to UART Bridge Controller
... now attached to ttyUSB0

That means your UART port is likely /dev/ttyUSB0.

Exit the screen using Ctrl+A then K then y

Step 2: Install required packages

Install the typical ESP-IDF build dependencies:

sudo apt update && sudo apt install -y git python3 python3-venv python3-pip cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0

Step 3: Install ESP-IDF (v5.2.3) + ESP32-C3 toolchain

Clone ESP-IDF and install only what you need for ESP32-C3:

cd ~ && git clone -b v5.2.3 --recursive https://github.com/espressif/esp-idf.git && cd esp-idf && ./install.sh esp32c3

Then load the environment into your current terminal session:

. ./export.sh

From this point, idf.py should be available.

Step 4: Create a local `hello_world` project

Copy the example out of the ESP-IDF tree (so you can edit safely later):

cd ~ && cp -r ~/esp-idf/examples/get-started/hello_world ~/hello_world

Step 5: Set the target to ESP32-C3

cd ~/hello_world && idf.py set-target esp32c3

Step 6: Build, flash, and monitor

Flash to the detected serial port and open the monitor immediately:

idf.py -p /dev/ttyUSB0 flash monitor

If you get a permissions error on /dev/ttyUSB0, rerun with sudo:

sudo idf.py -p /dev/ttyUSB0 flash monitor

To exit the ESP-IDF monitor: press Ctrl + ].

Expected output

After flashing, the monitor should show hello_world output similar to:

Hello world!
This is esp32c3 chip with 1 CPU core(s), WiFi/BLE, silicon revision v0.3, 2MB external flash
Minimum free heap size: 328280 bytes

Troubleshooting

1) `idf.py: command not found`

Re-run:

. ./export.sh

2) Wrong serial port

Re-run:

sudo dmesg -T | tail -n 30

Look for ttyUSB0 vs ttyACM0, then update the -p argument accordingly.

3) Permission denied on `/dev/ttyUSB0`

Use sudo as shown above, or add your user to the serial group (commonly dialout) and re-login.

A Guide to Using ST Edge AI Developer Cloud

noreply@blogger.com (Maxwell Clarke) — Fri, 20 Jun 2025 13:09:00 +0000

Using ST Edge AI Developer Cloud

In a previous post, A Guide to STMicroelectronics' Edge AI Suite Tools , I provided an overview of the tools in STMicroelectronics' Edge AI Suite. In this post, we'll focus on one such tool— ST Edge AI Developer Cloud— and walk through how to use it to test machine learning models.

ST Edge AI Developer Cloud is a web-based tool that allows you to test machine learning models by remotely accessing ST boards. It enables simulation of deployment and performance testing without the need to purchase any physical boards.

This guide outlines each step required to use the Developer Cloud, with screenshots provided to show the exact process. A similar walkthrough is also available in a video on the STMicroelectronics YouTube Channel.

Walkthrough

1. Accessing Edge AI Developer Cloud

Visit the ST Edge AI Developer Cloud and click "Start Now" on the landing page.

2. Sign in or Create an Account

To use the tool, log into your myST account or create one if you haven't already.

3. Import a Model

Import a model from your device or from the ST Model Zoo. For this example, I will use the first "Hand Posture" model that appears in the Model Zoo. Once selected, click "Start" next to the imported model.

4. Choose Platform & Version

Select a version and platform to use. For this demonstration, I will use the default version, ST Edge AI Core 1.0.0, and select STM32 MPUs as the target platform.

5. Quantize the Model

Click "Launch Quantization". You may also upload a .npz file for accuracy evaluation. After quantization, click "Go Next" to move on to the Optimization stage.

6. Optimize the Model

In the Optimization section, select a .tflite file from the model selector at the top, then click "Optimize". Once the model has been optimized, click "Continue", which will appear next to the "Optimize" button.

7. Benchmark Across Boards

Click "Start Benchmark" for all available boards. This will remotely run inference on a physical board and display the inference time once complete. Afterwards, click "Go Next" above the boards.

8. View Results

Under the "Results" section, you can view metrics such as weight size, performance, and resource utilization.

The "Show details per layer" option shows the resource utilization on the selected board, and the "Show performance summary" option compares inference times across all tested boards. After reviewing the results, click "Go Next".

9. Generate Code

Based on the benchmark results, generate code tailored to the optimal board. In this example, we will select the STM32MP135F-DK board, as it showed the fastest inference time. To view the timings, refer to the "Show Performance Summary" graph from the "Results" section.

Conclusion

The ST Edge AI Developer Cloud is a powerful testing environment for optimizing AI models on ST hardware. By allowing developers to evaluate boards remotely, it streamlines the deployment process and speeds up decision-making when selecting the best platform for your machine learning applications.

Fix for KiCad Causing Windows Shutdown

noreply@blogger.com (Maxwell Clarke) — Tue, 13 May 2025 18:22:00 +0000

Recently while working with KiCad 9.0.2, my Windows 11 machine kept shutting down unexpectedly. To fix this issue I had to uninstall all versions of KiCad on my computer, then reinstall the latest version. In this instance, I had 7.0, 8.0 and 9.0 installed.

Once all versions have been removed, double check your start menu for an app called "KiCad Command Prompt". If it appears, you may need to uninstall it also. Additionally, you may need to restart your computer. Afterwards, proceed with reinstalling 9.0.2.

Now that 9.0.2 runs smoothly, I will check back upon future releases to see if similar issues arise.

How to Build the TensorFlow Lite C API from Source Inside WSL

noreply@blogger.com (Maxwell Clarke) — Fri, 14 Mar 2025 23:42:00 +0000

How to Build TensorFlow Lite C API from Source Inside WSL

TensorFlow Lite is a lightweight, efficient runtime for deploying machine learning models on edge devices. It's ideal for environments that are low-power and performance-critical such as embedded systems mobile devices, and microcontrollers.

Building the TensorFlow Lite C API from source inside Windows Subsystem for Linux (WSL) allows you to integrate AI inference into native C applications. This is useful when working on constrained devices, building low-level systems, or working with existing C/C++ codebases.

Step 1: Set Up WSL and Create a .wslconfig File (Optional)

To prevent Bazel crashes from memory exhaustion, increase the memory limit for WSL. First, Open the Terminal(Windows Powershell):

# On Windows (not WSL):
Create C:\Users\<yourname>\.wslconfig with the following content:

[wsl2]
memory=6GB
processors=4

To do this with Notepad:

Open the Start menu and type Notepad
Paste the above configuration text into the new file
Click File > Save As...
Set File name: .wslconfig
Set Save as type: to All Files
Save it to C:\Users\<yourname>\

Then from PowerShell:

wsl --shutdown

Step 2: Install Prerequisites

sudo apt update
sudo apt install -y build-essential clang git wget python3-pip

Step 3: Install Numpy

pip install numpy

Step 4: Install Bazelisk

wget https://github.com/bazelbuild/bazelisk/releases/download/v1.17.0/bazelisk-linux-amd64 -O bazelisk
chmod +x bazelisk
sudo mv bazelisk /usr/local/bin/bazelisk

Step 5: Set the Required Bazel Version

export USE_BAZEL_VERSION=5.3.0
echo 'export USE_BAZEL_VERSION=5.3.0' >> ~/.bashrc
source ~/.bashrc

Step 6: Clone TensorFlow and Check Out the Version

git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow
git checkout v2.12.0

Step 7: Build TensorFlow Lite C API

Optional but recommended: limit RAM usage to avoid crashes.

export BAZEL_BUILD_OPTS="--local_ram_resources=2048"
cd tensorflow/tensorflow/lite/c
bazelisk build -c opt $BAZEL_BUILD_OPTS --define=flatbuffer_op_resolver=false //tensorflow/lite/c:libtensorflowlite_c.so

Step 8: Install the Library and Headers

cd ~/tensorflow
sudo cp bazel-bin/tensorflow/lite/c/libtensorflowlite_c.so /usr/local/lib/
sudo ldconfig

# Copy required top-level headers
sudo mkdir -p /usr/local/include/tflite
sudo cp tensorflow/lite/c/c_api.h /usr/local/include/tflite/

# Copy all internal TensorFlow Lite C API dependencies
sudo mkdir -p /usr/local/include/tensorflow/lite/core/c
sudo cp tensorflow/lite/core/c/c_api.h /usr/local/include/tensorflow/lite/core/c/
sudo cp tensorflow/lite/core/c/c_api_types.h /usr/local/include/tensorflow/lite/core/c/

# Copy additional headers required by the C API
sudo mkdir -p /usr/local/include/tensorflow/lite
sudo cp tensorflow/lite/builtin_ops.h /usr/local/include/tensorflow/lite/

Step 9: Verify With a Simple C Program

#include "tflite/c_api.h"
#include <stdio.h>

int main() {
    TfLiteModel* model = TfLiteModelCreateFromFile("model.tflite");
    if (!model) {
        printf("Failed to load TensorFlow Lite model\n");
        return 1;
    }
    printf("TensorFlow Lite model loaded successfully!\n");
    TfLiteModelDelete(model);
    return 0;
}

Compile it with:

gcc -o tflite_test tflite_test.c -I/usr/local/include -L/usr/local/lib -ltensorflowlite_c
./tflite_test

Conclusion

Now that you’ve built the TensorFlow Lite C API from source inside WSL, you're ready to run AI inference directly in your C applications. This setup is ideal for embedded AI applications such as digital signal processing. By building from source, you gain control when integrating with systems where Python or heavy dependencies are incompatible.

Previewing NVIDIA's Cosmos, a Generative AI Worldbuilding Tool

noreply@blogger.com (Maxwell Clarke) — Wed, 15 Jan 2025 18:56:00 +0000

NVIDIA Cosmos is a new platform from NVIDIA that uses generative AI for digital worldbuilding. In this post, I will demonstrate some possible outputs from Cosmos using NVIDIA's simulation tools. Afterward, I will discuss some of the functions and libraries Cosmos uses.

To create your own digital worlds with Cosmos, follow this link to the Simulation Tools in NVIDIA Explore NVIDIA Explore - Simulation Tools. To start, we will select the cosmos-1.0-diffusion-7b.

The Preview Image for cosmos-1.0-diffusion-7b

Once we've selected cosmos-1.0-diffusion-7b, we are presented with an option to input text, an image, or a video. The default example is a robot rolling through a chemical plant, with a video as the output.

AN AI-Generated Image of a Robot Traversing a Chemical Plant

For this demonstration, I'm going to begin by inputting the following text into the input box: "A crane at a dock lifting a large cargo crate from a ship onto the dock. photorealistic" After about 60 seconds, Cosmos produces a short 5-second video as output. Here is a frame from the one it generated from my first prompt:

An AI-Generated Image of a Dock with a Crane

In this case, we used the Cosmos-1.0-diffusion-7b-Text2World model, which takes an input of up 300 words and produces an output video of 121 frames.As described in the linked documentation, it uses self-attention, cross-attention, and feedforward layers. Additionally, it uses adaptive layer normalization for denoising between each layer. Each layer is necessary and serves a unique purpose.

Starting with the self-attention layer, it is used to determine what words in the input text will be most relevant to the output image. For example, the word "crane" in our prompt is weighted higher than the word "at". While both are relevant to the output, the object of the crane is in the center of the video. Next, the cross-attention layer relates the information contained in each word and assigns it to a relevant image as a result. In our case, this is shown by the word crate and the image of a brown crate. To clarify, the word "crate" is referred to as the source, and the image is referred to as the target.

The third layer, the feedforward layer, redefines each word after the the cross-attention layer finds it relevance. For example, the crate in our example is placed on the dock in our image, because the feedforward layer related it to the phrase "onto the dock". Lastly, the adaptive layer normalization stabilizes the output, which in this case could refer to making the crane move slowly and not too jittery.

In addition to the cosmos-1.0-7b-diffusion-7b which uses the text2world model, there is also the cosmos-1.0-autoregressive-5b model.

The Preview Image for cosmos-1.0-autoregressive-5b

This model takes a picture as input and produces a 5-second video as an output. The first frame of the output video is the exact picture, and the model predicts what happens in the next 5 seconds of the scene to create the video. For this model, there are a series of 9 preselected images to choose from.

Sample Images for Video Generation

Similar to the text2world model, the autoregressive video2world model employs self-attention, cross-attention, and feedforward layers. It should be noted that while this model is referred to as video2world, it can accept text, images, or videos as input, and outputs a video from whichever input was given.

Overall, NVIDIA Cosmos is a powerful worldbuilding tool for a variety of applications including simulation software as well as game development. To learn more about the development tools NVIDIA has to offer, check out the following post: An Overview of NVIDIA's AI Tools for Developers

NVIDIA's Jetson Orin Nano Super Developer Kit

noreply@blogger.com (Maxwell Clarke) — Wed, 08 Jan 2025 16:32:00 +0000

NVIDIA recently unveiled their new Jetson Orin Nano Super Developer Kit, a powerful computer designed for using AI on edge devices. Key features of the kit include a performance of up to 67 TOPS(Trillion Operations per Second), 102 GB/s of memory bandwidth, and a CPU frequency of up to 1.7 GHz. Other technical specifications found on the datasheet are a 6-core ARM Cortex-A78AE v8.2 64-bit CPU (arm-cortex-a78ae-product-brief), an NVIDIA Ampere GPU (NVIDIA Ampere Architecture) and 8GB of memory. The kit has a cost of $249 and sold out quickly. It is currently on backorder from retailers such as Sparkfun Electronics, and Seeed Studio.

This new Super Developer Kit is an upgraded version of the previous Jetson Orin Nano Developer Kit. NVIDIA has provided the JetPack 6.1 SDK which can be used on existing Jetson Orin Nano Developer Kits to access features from the Super developer Kit. To unlock the super performance, users can select the latest release of JetPack, Jetpack 6.1 (rev. 1), when installing the latest SD Card image for the Kit. Detailed installation instructions can be found on the following page: JetPack SDK.

The following table from NVIDIA highlights the main improvements of the Super Developer Kit, including the improved operating speed of up to 67 TOPS and memory bandwidth of 102 GB/s.

In this video, NVIDIA CEO Jensen Huang presents the Jetson Orin Nano Super Developer Kit.

If you are interested in learning about software compatible with the Jetson Orin Nano, check out this post in which I summarize the key features of NVIDIA's AI tools: An Overview of NVIDIA's AI Tools For Developers

An Overview of NVIDIA's AI Tools For Developers

noreply@blogger.com (Maxwell Clarke) — Mon, 02 Dec 2024 18:18:00 +0000

Here is a quick overview of the tools NVIDIA offers for Developers! All of these tools are available through the NVIDIA Developer platform, and joining is free. NVIDIA also offers the Jetson Orin Nano Super Developer kit which I review in the post: NVIDIA's Jetson Orin Nano Super Developer Kit

NIM APIs - designed for developers who need to perform AI inference in various scenarios. Notable examples include the Deepfake Image Detection Model by Hive, and the llama-3.2-90b-vision-instruct model by Meta. The following video is an excellent tutorial for getting started with an NVIDIA NIM.

LLMs - NVIDIA provides Large Language Models(LLMs) for tasks such as data generation and classification. These models, including the latest Llama 3.1, are valuable for AI applications like deep reasoning. Customize Generative AI Models for Enterprise Applications with Llama 3.1

Sample uses for LLMs (Transformer Models)

Generative AI Solutions - NVIDIA's Generative AI Solutions offer a comprehensive set of tools for developers. A great starting point is the Full-Stack Generative AI Platform for Developers. This platform provides an overview of NVIDIA's software. hardware, and services for building with Generative AI. Its "full-stack" approach enables developers to complete entire builds using NVIDIA products.

Full-Stack Overview

NVIDIA Documentation Hub - a centralized resource for accessing technical documentation for all NVIDIA products and services.

AI Inference Tools and Technologies - NVIDIA offers tools specifically designed to perform inference, which involves using AI to generate or process data. Three notable tools included in this page are sample applications for Building a Digital Human (Avatar), Designing Molecules, and Data Extraction.

Building a Digital Human Demo

RAPIDS Suite - contains AI Libraries that improve the performance of other open-source tools. It includes support for libraries such as Apache Spark, PyData, Dask, Python, and C++.

Full-Stack Using RAPIDS Libraries

Riva Speech AI SDK - a collection of software development kits (SDKs) useful for speech-related tasks. It offers starter guides for key use cases, including Text-To-Speech, Automatic Speech Recognition, and Neural Machine Translation. A tutorial for getting started with RIVA can be found in the video below.

DeepStream SDK - a tool useful for vision-based AI applications. It supports tasks such as video encoding, decoding, and rendering, enabling real-time video analysis.

DeepStream SDK

NVIDIA Developer Forums - an excellent resource for developers to ask questions and find answers to technical issues related to NVIDIA's AI tools.

Developer Forums

In conclusion, NVIDIA offers an extensive library of tools for a wide range of AI applications. Many of these tools can be accessed for free through the NVIDIA developer program, making it a valuable resource for developers at any level.

Black Friday 2024 Deals on Maker Electronics!

noreply@blogger.com (Maxwell Clarke) — Fri, 29 Nov 2024 14:13:00 +0000

Here are some of the deals major online electronic retailers are offering this Black Friday!

Adafruit Industries -15% Off with code acidburn15 Hack the Holidays – Use Code acidburn15 for 15% Off « Adafruit Industries – Makers, hackers, artists, designers and engineers!

Pololu is offering discounts (up to 50% off!) on many electronic components including Robot Chassis, motors, sensors, and much, much more! Check out the link for all details and a categorized list of all products on sale: Pololu Black Friday Sale 2024.

seeed studio Thanksgiving Sale While Thanksgiving day may be over, the seeed studio 2024 Thanksgiving Sale is happening right now! The sale includes Buy 2 get 1 Free on select products, special offers, and, for 11/29 (Black Friday!) only, a flash sale. Link to entire sale: Exclusive Sale: Up to 90% Off & Buy 2 Get 1 Free & Flash Sale & Pre-Sale Offers

Black Friday Sale on Tindie has many devices from various shops, including wifi adapters, battery testers, and much more!

Arduino Online Shop has plenty of devices on sale, including the Arduino Nano, Arduino Uno, and Nicla.Arduino Online Shop

STMicroelectronics is offering free worldwide shipping through November 30th! Additionally, select Motor Control Evaluation Boards have discounts with code DV-ASP-BOGO-11 at checkout. eStore - STMicroelectronics - Buy Direct from ST

Raspberry Pi While not a Black Friday deal, the Raspberry Pi Compute Module is now available for $45 from the Raspberry Pi Store. It is a modular version of the Raspberry Pi 5.

2024 Black Friday Sale - BC Robotics Receive free gifts with purchases and save on select items. Additionally, there are door crasher deals.

Parts Express save 12% sitewide with code BLACKFRI24

RobotShop has up to 70% off on select products

JLCPCB:Black Friday 2024 Deals and Coupons- Coupons on PCB orders

Find any other great deals on electronics? Let me know in the comments below!

A Guide to STMicroelectronics' Edge AI Suite Tools

noreply@blogger.com (Maxwell Clarke) — Sat, 02 Nov 2024 19:30:00 +0000

STMicroelectronics has plenty of Edge AI Tools that are useful in a variety of applications. They collectively form the ST Edge AI Suite - STMicroelectronics. All of these tools can be used free of charge with a myST user account. To create an account, click on the avatar in the top right corner of the STMicroelectronics Website. You are now ready to access the ST Edge AI Suite!

1. NanoEdge AI Studio

The first tool is NanoEdge AI Studio, it is a good place to get started with Machine Learning.

It is also compatible with Arduino Devices.

For getting started, the following video is a good example - Anomaly Detection Demo Using Edge AI. It uses the Accelerometer on the STEVAL-STWINKT1B board. In a future blog post I will try this demo using some other ST Micro boards that have an accelerometer. Additionally, the Nano Edge AI libraries can be used with any type of sensor.

To use the Datalogger Feature in NanoEdge AI Studio, a Nucleo-F411RE, ST-EVAL or similar type of board is necessary. Here are all the Board options shown:

2. ST Edge AI Developer Cloud

ST Edge AI Developer Cloud is a tool that allows users to do on remotely stored devices. You can import your own ML model or choose from ST Edge AI Model Zoo for optimization and testing.

The following video is a getting started guide for STM32Cube.AI Developer Cloud, the predecessor of ST Edge AI Developer Cloud. While there are some differences between the two, this getting started video does have some helpful steps. I'm also working on a blog post about getting started with the updated ST Edge AI Developer Cloud.

3. MEMS-Studio

MEMS-Studio is a software solution for using AI in embedded systems development. It can be downloaded from the linked webpage (MEMS-Studio). Once installed an opened, you will have access to Getting Started with MEMS-Studio - User manual and Introduction-to-MEMS-studio Video.

4. STEdgeAI-Core

STEdgeAI-Core is used for compiling edge AI models on various ST MCUs MPUs, and Smart Sensors - all in one tool. The following video gives an overview.

5. ST Edge AI Model Zoo

The ST Edge Ai Model Zoo is a collection of AI models that can be used for running on the other tools in the ST Edge AI Suite. It is comprised of three GitHub repositories: AI Model Zoo for STM32 devices, GitHub - STMicroelectronics/STMems_Machine_Learning_Core, and GitHub - STMicroelectronics/st-mems-ispu: Examples, tutorials, and other development resources for the ISPU.

6. X-CUBE-AI - AI expansion pack for STM32CubeMX

The AI expansion pack for STM32CubeMX is a tool used for optimizing and profiling Neural Networks and Machine Learning models for STM32. To install this tool, open the STM32CubeIDE, and go to Help > Manage Embedded Software Packages > STMicroelectronics, then scroll down to X-CUBE-AI and click on the newest available version (9.1.0 in this case), then click "Install." You may need to log-in on the pop up that appears for the expansion to install successfully. Here is a screenshot of the installation screen.

7. ST High Speed Datalog

One of the newer tools in the suite, ST High Speed Datalog is a data acquisition and visualization toolkit. It is specifically designed for applications in embedded systems and data science. The following Datalog Quick Start Guide is designed to get started with the Datalog along with the STEVAL-STWINBX1 .

8. StellarStudioAI

The next tool, StellarStudioAI is an AI tool for StellarStudio - development environment for Stellar automotive MCUs. StellarStudioAI generates neural networks and converts them into C libraries for use by MCUS.

9. X-Linux-AI

X-Linux-AI is an expansion for the STM32 MPU OpenSTLinux Distribution. X-LINUX-AI provides LINUX AI frameworks and application examples for use with STM32 MPUs.

10. ST ToF Hand Posture

The final tool, ST ToF Hand Posture, is used for detecting hand gestures on STM32 MCUs using a Time-of-Flight sensor. It requires a free login to MyST to access.

This completes the summary of the ST Edge AI Tools. The first four tools listed are the best place to get started when doing AI projects for ST MCUs and MPUs. Helpful example videos are also available for NanoEdgeAI Studio, and MEMS Studio. For example, NanoEdge AI Studio V3 - Anomaly Detection demo and Introduction To MEMS Studio.

Introduction to Machine Learning on Arduino - Micro Speech - Part 2

noreply@blogger.com (Maxwell Clarke) — Mon, 26 Aug 2024 14:56:00 +0000

Part 2: Changing Words

In this part, we will use the words "On" and "Off" to control the LED and turn it on and off. This is a continuation from the steps in Part 1.

To do this, we can use the Google Commands dataset which has data for many short words. The following video by Digi-Key includes instructions for Downloading the dataset. Although it is not labeled as machine learning in the title, it is similar to what is being done in this ML guide.

2.1 Downloading Python Script

The Dataset Curation Python Script is available to download. Here are the steps to extract it properly, as the video instructions are shown too quickly.

2.1.1 Install 7-Zip

For extracting the file. On the 7-Zip Download Page select the download for your machine. (I'm using Windows 7 64-Bit)

Once downloaded, open the .exe file. (You may need to click "Allow Changes" in the pop-up window) Next, Install.

2.1.2

Click on the green "Code" box and select the "Download ZIP" option that appears

2.1.3

Left Click on the Downloaded zip file, then click on "Show more options".

2.1.4

Under "Show more options" Hover over "7-zip", then click on "Extract Here"

2.1.5

Once it's extracted, open File Explorer and search for "ei-keyword". Open the "ei-keyword-spotting-master" folder.

2.1.6

Copy and paste the "dataset-curation" and "utils" Python files into a directory where they can be accessed.

2.2 Downloading Speech Commands

As shown in the video description, here is a download link for the Google Speech Commands Dataset. If a page appears when following this link, click on "Go to Site" to start the download.

2.2.1

Left Click on the Downloaded zip file of the Dataset, then click on "Extract All"

2.2.2

Create a folder for housing the necessary files. In this example I named it "Speech_Rec".

2.2.3

Cut and paste the _background_noise_ into the created folder. The _background_noise_ must be stored separately from the individual keywords.

2.2.4

Create a folder for housing keywords such as "on" and "off". (The video example includes using custom keywords as well)

2.2.5

Copy and paste the "on" and "off" keywords into the newly created "keywords" folder

2.3 Using the Program

2.3.1

Python will need to be installed to run the speech Recognition Code. Download Python — click on the yellow box to download the latest version.

2.3.2

If you already have Python installed, it may ask to upgrade to the latest version. Otherwise, there should be an option to install.

2.3.3

An option to close the installer will appear once setup is successful

2.3.4

Open Windows PowerShell (Terminal) and run the following command. This installs all the necessary Python libraries for the speech recognition.

PowerShell — install Python libraries

pip install librosa numpy soundfile

2.3.5

While in PowerShell, go to the folder that contains all the speech recognition files ("Speech_Rec"). To find the path, click on the folder name in File Explorer, then left click and select "Copy Address".

Use the cd command followed by the path to the directory, for example:

PowerShell — navigate to Speech_Rec folder

cd C:\Users\maxcl\Downloads\Speech_Rec

2.3.6

Next, enter the following line into the terminal. It runs the dataset-curation.py script and sets all the necessary parameters.

PowerShell — run dataset curation script

python dataset-curation.py -t "on, off" -n 1500 -w 1.0 -g 0.1 -s 1.0 -r 16000 -e PCM_16 -b "..\Speech_Rec\_background_noise_" -o "Speech_Rec\keywords" "..\Speech_Rec\data_speech_commands_v0.02" "..\datasets\keywords"

The script may take a few minutes to run.

2.3.7

Once it's finished, a set of folders named after the input, output, and keywords can be found in the directory. (Note: In this case, the folder name "Speech_Rec" appeared twice so I changed the second one to "results")

2.3.8

Open the "on" folder and select one of the audio samples to play.

2.4 Edge Impulse

2.4.1

Next, we will go to Edge Impulse. Click on the "Login" option in the top left corner of the home screen, as this will give you the option to either log into your account or create a new account.

2.4.2

Once you've created an account and/or logged in, click on the icon in the top left, then click on "Projects"

2.4.3

Click on "Create New Project"

2.4.4

Name your project then click on the green "Create New Project". I chose to leave this one as public.

2.4.5

In the new project, click on "Data Acquisition"

2.4.6

Next, click on "Add Data"

2.4.7

Select "Upload Data"

2.4.8

Leave the default selections for category and label, and click on "Select Files"

2.4.9

Start by uploading all the files in the noise folder. (I selected all by clicking on the box next to "name"). Click "Open"

2.4.10

Afterwards, click on the purple "Upload Data" button

Edge Impulse will then process and add the files to the project.

2.4.11

Repeat steps 2.4.8 through 2.4.10 for the "on", "off", and "unknown" categories.

2.4.12

The pie charts at the top of the Dashboard should show that the data is split amongst the four categories, and 80% of the data was assigned to training and 20% was assigned to testing.

2.4.13

On the left side, click on Impulse Design, then Create Impulse. Click on "Add Processing Block".

2.4.14

Add the Audio (MFCC) Processing Block

2.4.15

Select the "Add a Learning Block" option, then choose the default "Classification" block

2.4.16

Click on the Green "Save Impulse"

2.4.17

From the MFCC on the left, select the Generate Features tab, then click on "Generate Features"

2.4.18

Next, under Classifier on the left, choose Save & Train.

2.4.19

Now that the model is trained, the loss and accuracy of the model will appear. The accuracy refers to the percentage of the data that was correctly classified. As for loss, it is based upon a predefined function, and a smaller loss value indicates better ML model accuracy.

2.4.20

To test the model, go to Model Testing on the left. Select "Classify All".

2.4.21

Once the testing is done, the results will be shown.

2.4.22

Next, we will deploy the model for Arduino. Click on the Search Bar and select the Arduino option. (Note: There are other options that can be used for other microcontrollers such as STM32 devices)

2.4.23

Select "Build" to create the Arduino Library, which will download automatically.

Once it's done building, a dialogue will pop up showing the path for using the Arduino Library

2.5 Arduino

2.5.1

In the Arduino IDE, open a new Arduino Sketch (File > New Sketch)

2.5.2

Once you're in a new sketch, select Sketch > Include Library > Add .ZIP Library and open the library downloaded from Edge Impulse.

2.5.3

Next, go to File > Examples and scroll down to find the library you just installed. It should have the same name as the project in Edge Impulse. Go to "Speech_Rec_inferencing" (may differ based on your project name) > "nano_ble33_sense" > "nano_ble33_sense_microphone_continuous". Note: the Arduino IDE might need to be restarted for it to appear.

2.5.4

Open File Explorer and go to the newly installed Speech_Rec library and locate the Model variables file. The path will likely be: User > Arduino > libraries > Speech_Rec_Inferencing > src > model_parameters > model_variables

2.5.5

Open the model_variables.h file in a code editor such as VS Code, and search for where the keywords are listed. The line will likely be:

model_variables.h — keyword categories array

const char* ei_classifier_inferencing_categories[] = { "_noise", "_unknown", "off", "on" };

In this case "_noise" is in position 0, "_unknown" is in 1, "off" is 2, and "on" is 3.

2.5.6

Return to the Arduino IDE. In the nano_ble33_sense_continuous example, create a new line under line 59 (static int print_results = -(EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW);).

The new line will read:

Arduino sketch — declare LED pin variable

static const int led_pin = LED_BUILTIN;

2.5.7

Next, go a few lines down to be within the void setup() { block of code. Type the following line:

Arduino sketch — set LED pin as output (inside void setup)

pinMode(led_pin, OUTPUT);

2.5.8

Even further down in the code, above line 114 (if (++print_results >= (EI_CLASSIFIER_SLICES_PER_MODEL_WINDOW)) {), enter the following block of code:

Arduino sketch — LED on/off control block

    //turn on LED if "On" value is above a threshold
    if (result.classification[3].value > 0.7) {
      digitalWrite(led_pin, HIGH);
    }
    //turn off LED if "Off" value is above a threshold
    else if (result.classification[2].value > 0.7) {
      digitalWrite(led_pin, LOW);
    }
    else {
      digitalWrite(led_pin, LOW);
    }

This code controls whether the LED is "on" or "off" based on the keyword recognized. The result.classification[2] and result.classification[3] values are consistent with the labels for "Off" and "On" found in the model_variables.h file.

2.5.9

Next, click the arrow to upload the code. It may take a while to compile, as it is a lengthy code segment.

2.5.10

Once the code is compiled, speak the keywords within a couple inches of the board. The voice needs to be very close for the recognition to work. If the board recognizes a word with a confidence value of 0.7, the LED next to the USB will turn yellow.

2.5.11

To check the readings for the speech recognition values on the board, you can look at the serial monitor in the Arduino IDE. To do this click on Tools > Serial Monitor (or type Ctrl + Shift + M). The serial monitor continually shows the speech rec values for all the keywords, as we are in the continuous example.

We now have a working Arduino program that uses machine learning to recognize the words "on" and "off". In the future, the code can be adjusted so that the LED stays on for longer, thus making the "off" command more effective.

Introduction to Machine Learning on Arduino - Micro Speech - Part 1

noreply@blogger.com (Maxwell Clarke) — Wed, 21 Aug 2024 15:46:00 +0000

The beginning of this guide is based upon Get Started With Machine Learning on Arduino | Arduino Documentation and Wake-Word Detection - TinyML Notes. Up through the speech recognition example, it follows mostly the same steps. I've added steps for clarity and detail, for anyone starting from scratch. Additionally, I encountered an issue with the Windows Machine I'm using crashing due to the GameInput Application, which will be addressed at the end.

The examples in this guide are part of the field of TinyML. TinyML is machine learning done on low-power devices such as the Arduino Nano we are using. One potential use case for TinyML is wake-word detection.

I have written this guide so that the program can be run and tested in the Arduino IDE as of June 2024.

This is Part 1 of 2 in this series of posts. In part 2, I will show how to change the keywords using Edge Impulse.

1 Creating a New Arduino Sketch

1.1 Prerequisites

The Arduino IDE is required for this operation. The host machine should have a 64-bit processor. You will also need about 16GB of hard drive space and a stable network connection.

1.2 Install Arduino IDE

Arduino IDE 2.3.2 is current at time of writing. Here's the link to Download the installer for your machine: Software | Arduino.

When you click on the installer, you can select "Just Download" on the following two pages that pop up.

Once the .exe file is downloaded, open it and click on "I agree" to the installer.

Continue and Click "Next" after selecting which users to install the IDE for.

A system dialogue screen may appear (not pictured in this guide). Click "Yes" to "Allow Changes" then click on "I agree" once more.

Afterwards, click "Install"

The installation will take a minute. Once it is done, click "Finish".

1.3 Install Git

Download the correct version of Git for your machine. The latest Windows Release as of writing is 2.45.2. Click on the "Download For Windows" Button. Here is the link to the site to download: Git (git-scm.com)

Next, click on "Click here to download" the latest version

Open the .exe file and click "Next".

I chose all of the default selections through the rest of the installation. Feel free to make any changes necessary for your installation. For each screen, Click "Next" until you get to the "configuring experimental options" and can click "Install"

Click "Install". The installation will run for a few minutes.

Click "Finish". The Release notes will open in a browser and can be disregarded for now

1.4 Install TensorFlow Arduino Libraries

Open the 'Terminal' (AKA Windows Powershell) application. There may be an icon on the start ribbon at the bottom of your desktop, if not search for 'Terminal' in the start menu.

Open the file explorer and locate your Arduino Libraries Folder. Click on it so that its highlighted, then press Ctrl+Shift+C to copy as path. (Alternatively, you can also left click on it then click "Copy as Path".)

Once you've copied the path, return to the terminal and paste the path in the command line:

C:\Users\maxcl>:~# "C:\Users\maxcl\Arduino\libraries"

Next: Use the following command to install the TensorFlow Lite library

C:\Users\maxcl>:~# git clone https://github.com/tensorflow/tflite-micro-arduino-examples Arduino_TensorFlowLite

Once the library is installed, use the following command to change directory to the new TensorFlow Library

C:\Users\maxcl>:~# cd Arduino_tensorFlowLite

Lastly use the git pull command to update the library

1.5 Working in the Arduino IDE

Open the Arduino IDE, either by searching for it from the Windows start menu ("Arduino IDE"), or clicking on a desktop icon. (It also may be open from the installation earlier.)

When it opens, there will be a message that appears in the bottom left of the screen prompting you to install updates. Click "Install all". There also may be an update message for boards. Click "Install All" to this too.

After everything is installed, verify that the Arduino_TensorFlow Lite library is installed by looking under "File" Then "Examples" in the top right-hand corner of the IDE. You might have to scroll down in the "Examples" section to find it because it will be found under "Examples from Custom Libraries"

1.6 Attaching the Arduino Nano

Connect the Arduino Nano 33 BLE to your computer using a USB A to Micro USB cable. Note: if you have the Arduino Tiny Machine Learning kit, the Arduino Nano 33 BLE is in foam (see picture below).

Also, a USB A to Micro USB cable is provided in the kit.

In the Windows start menu, search for the "device manager". Open it and click on "USB Serial Device(COMX)" under Ports. The Com number may differ; it is COM7 in this example.

When you click on it, it should say "This Device is working properly" under General.

When the board is Connected for the first time, A pop-up will appear in the bottom right corner of the Arduino IDE, prompting you to manually install the board package. Click "Install" then "Next". A banner will appear on the right side of the screen. Click the green "Install" button on the one that says, "Arduino Mbed OS Nano Boards."

Once the board package is installed, click on the white text box in the top right of the screen. Note: it may say "None" or "Unknown" at first. Click on the pen icon to open the board/port selector.

When the Board Selector Opens, search for "Arduino Nano 33 BLE" and select the one that reads 'Arduino Nano 33 BLE - Arduino Mbed OS Nan...'. Under 'Ports' Select the COM device that was listed for 'USB SERIAL DEVICE' in the Device Manager. Click the Green 'OK' Button.

1.7 The Micro Speech Example

In the Arduino IDE, under File>Examples>Arduino_TensorFlowLite, click on 'micro_speech'. This will open the "micro_speech" example which does basic speech recognition. Note: Make sure to Click on 'Examples' as opposed to just hovering over it.

Once micro_speech is open, check to make sure that the proper board and port are selected. (You may need to reselect the correct COM port and the board.)

Next, click the checkmark in the top right of the screen. (The "Verify" button.) This will compile the code. (Which may take a few minutes and uses a lot of processing power).

After the board is verified, click the arrow next to it, the 'Upload' Button. This programs the board with the code.

The board is now programmed with speech recognition. You can test it by saying "Yes" aloud and watching the LED turn green, then saying "No" for it to turn red.

While verifying and uploading the code, my computer kept crashing. After looking into the event viewer, the shutdowns were due to the 'GameInput' Application. To address this I followed the steps in the following video up to 1:59. At this point, I had to have the GameInput startup to 'Manual' instead of 'Automatic' which was shown in the video.

The "Manual" startup should be the default setting. If you need to change startup mode, the way to open the GameInput properties differs than in the video.

To access the properties, first, right click on GameInput in Task Manager, and click on the 'Open Services' Option. Note: Tasks are sorted alphabetically

Next, right click on the GameInput Service and choose 'Properties'. Note: Sorting is again alphabetical

Once you are in the properties, select 'Manual' for Startup Type. If it is running, select 'Stop'. Afterwards, click on 'OK'. Now that the GameInput is not running, you should be able to run the micro_speech example without crashing.

In part 2, I will show how to change the words being used for speech recognition using Edge Impulse.

ST Micro eStore Free Global Delivery for August

noreply@blogger.com (Aaron Clarke) — Mon, 19 Aug 2024 14:06:00 +0000

The STMicroelectronics eStore is offering free worldwide delivery for the remainder of August. No code or minimum purchase is required. There are also a few buy-one-get-one-free offers with a code.

Triple half-bridge gate driver - STDRIVE101 and STDRIVE101TR with code EV-FREEDRIVE101
250 V three-phase BLDC controller with embedded STM32 MCU - STSPIN32G0251Q with code EV-FREE-STSPIN32
3-phase inverter based on STSPIN32G0251Q - EVSPIN32G02Q1S1 with code EV-BOGO-EVSPIN32

Ezurio Development Board Kit Giveaway

noreply@blogger.com (Aaron Clarke) — Wed, 17 Jul 2024 10:28:00 +0000

Ezurio (formerly Laird Connectivity) is offering a development kit giveaway that includes a System on Module (SOM), a carrier board, a wireless module, and an antenna. Ezurio specializes in SOMs featuring NXP i.MX processors, including the i.MX 8M Mini, 8M Plus, and i.MX 93, with support for both Android and Linux. The giveaway also includes Sona WiFi 6/6E and Bluetooth LE modules.

Machine Learning Development Boards

noreply@blogger.com (Maxwell Clarke) — Mon, 15 Jul 2024 19:02:00 +0000

My name is Maxwell Clarke, I am an embedded systems engineer at Aaron Clarke where I am researching machine learning application development. An easy way to get started with machine learning development is to use an off-the-shelf development board. Here are some development boards I've identified that support machine learning. I will be posting tutorials for some of these boards in the upcoming weeks.

Arduino Tiny Machine Learning Kit

This kit's main purpose is to run TinyML projects by using TensorFLowLite on the board included. It contains an Arduino Nano 33 BLE Sense board, which features the 32-bit Arm Cortex-M4 microcontroller in an nRF52840 processor from Nordic Semiconductors. The board also has sensors for proximity, motion, and many other measurements. The kit costs $60 from the Arduino Store.

BeagleBone® AI-64

The BeagleBone AI-64 has great computing power as it contains a Texas Instruments TDA4VM SoC with a dual 64-Bit Arm Cortex-A72 microprocessor. It interfaces via a M.2E-key PCIe USB 3.0 connector for WiFi, Ethernet, and USB 3.0. Also included are 4-Lane CSI and DSI connectors for cameras and displays, respectively. It is available for about $190 from online distributors such as Digi-Key.

BeagleY-AI

This board uses a TI AM67A Arm- based vision processor, which contains a 64-bit Arm Cortex-A53 Microcontroller. It also has a BM3301 BLE module for networking over BLE. Other features include 3 displays including touchscreen support, a 10-pin Tag-Connect and 4 USB3 ports. It is less expensive than the AI-64 as it has a cost of around $70.

NVIDIA Jetson Orin Nano Developer Kit

This kit contains the Jetson Orin Nano 8GB, which features an Ampere GPU and a 6-core ARM CPU. It has a processing ability of up to 40 TOPS, making it the most powerful board on this list. Due to its top performance capabilities, it cost $500 from online distributors such as SparkFun.

In December 2024, NVIDIA unveiled a new and improved version of this Jetson Kit, the Jetson Orin Nano Super Developer Kit with an improved processing power of up to 67 TOPS. This board retailed for $250 and quickly sold out. Owners of the previous Jetson Orin Nano Developer Kit can also access the super capabilities of the new kit by upgrading the SDK, more instructions can be found on this page: Jetpack SDK For more information on Super developer kit, check out my post reviewing its key features, NVIDIA's Jetson Orin Nano Super Developer Kit The following video is from NVIDIA CEO Jensen Huang, showcasing the kit's small size and high performance capabilities.

Renesas Microcontrollers & Microprocessors (MCUs, MPUs)

Renesas currently has a Reality AI Software which is useful for all sorts of ML applications. They also have a wide range of microcontrollers which are compatible with their software. For example, there are RH850 Automotive MCUs for automotive applications, and RL78 Low Power 8 & 16-bit MCUs |which are good for getting started. Many of the microcontrollers such as the R7F7015813AFP-C#AA3 , which has a cost of $17, can be purchased from online distributors.

STM32 Microcontrollers (MCUs)

Currently, St Microelectronics has a vast library of Edge AI tools including MEMS Studio. These software libraries can be used for developing ML applications with STM32 boards. For example, the X-NUCLEO-IKS4A1 is a sensor kit forSTM32 boards that can be used with MEMS Studio. Both the kit and development boards can be purchased directly from STMicro. The X-NUCLEO-IKS4A1 costs about $33, and a new compatible board, the NUCLEO-G491RE costs about $16.

Edge Impulse

Edge Impulse is a suite of tools for conducting machine learning on edge devices. It has different ways to get started for beginners, ML practitioners, and embedded engineers. The getting started for beginners is a good way to begin with machine learning, without much coding required.

Black Friday 2021 Electronics Deals

noreply@blogger.com (Aaron Clarke) — Fri, 26 Nov 2021 13:07:00 +0000

I haven't found as many Black Friday promotions this year for electronics - probably due to the ongoing supply chain issues. If you see a good deal that is not in the list, please leave a comment.

Adafruit has a code for 15% off all weekend here. It's also Feather Friday which means 20% off all Feather (mainboards) and wing (daughterboards) with the code shown in the site header today.

Parts Express (audio components) has 10% of sitewide.

Pololu Robotics & Electronics Black Friday and Cyber Monday deals on select products are here.

RobotShop has a sale on some items until November 30 here.

Jameco Electronics has free shipping with a code on all orders.

Xilinx Adaptive Computing Challenge 2021 at hackster.io

noreply@blogger.com (Aaron Clarke) — Fri, 10 Sep 2021 13:43:00 +0000

The 2021 Adaptive Computing Challenge at hackster.io will reward top Xilinx development projects up to $10,000 (prizes are all Visa gift cards). You can apply for free hardware to compete in the contest until October 15, 2021. Final submissions are due February 28, 2022. The Xilinx University Program is sponsoring an additional $2500 prize for college students.

There are three main categories in the contest: Edge Computing, Data Center AI and Big Data Analytics. The edge computing requires a solution running on the Xilinx Kria KV260 Vision AI Starter Kit. Data center AI must use the Xilinx VCK5000 development card. Big data analytics requires the Xilinx Big Data Analytics card (not much information on this board).

The Kria KV260 Vision AI Starter Kit is listed at $199 but currently out of stock at most distributors. It is meant for vision application development for Edge computing. It has multiple camera interfaces including support for the Raspberry Pi camera module. Multiple cameras are supported by using additional interfaces such as IAS or USB. A power supply is also required. One is also included in the basic accessory pack which also includes a 13MP camera module, available from Xilinx for $59.

KV260 Vision AI Starter Kit with Basic Accessory Pack

MIDI Hack Competition at NAMM 2021

noreply@blogger.com (Aaron Clarke) — Thu, 03 Dec 2020 14:12:00 +0000

The MIDI Innovation Awards, a design competition for MIDI hacks, DIY projects, or innovative MIDI products is now open with final judging at NAMM 2021. This competition was announced by Music Hackspace, the MIDI Association and NAMM.

The first prize includes an exhibition booth and free advertising at the 2022 NAMM show. Other prizes to be announced. Submissions of existing commercialized products and art installations are also eligible. For more details see the Music Hackspace Innovation Awards page.

Arduino Sensor Kit

noreply@blogger.com (Aaron Clarke) — Tue, 01 Dec 2020 16:40:00 +0000

Arduino and Seeed have released the Arduino Sensor Kit for $23 (USD). The kit includes 10 modules and sensors including light, sound, air pressure, temperature, an accelerometer, buzzer, button, and a small OLED screen. The Arduino sensor kit base is an Arduino Shield (daughterboard) so you will need an Arduino UNO or compatible board to use it. If you buy both from Arduino you can get them bundled for a discounted price ($38.70). The Arduino Sensor Kit can be purchased directly from Arduino or Seeed and Arduino has developed a step-by-step tutorial to go with it and the Arduino UNO.

Seed has a very similar all-in-one board, the Seeed Grove Beginner Kit for Arduino ($19.90), which includes an Arduino compatible base board.

Electronics Deals Cyber Monday 2020

noreply@blogger.com (Aaron Clarke) — Mon, 30 Nov 2020 16:50:00 +0000

Cyber Monday electronics deals are here. Many of the deals listed in my blog post on Black Friday are still available from Adafruit, Polulu, SparkFun, and Tindie.

1BitSquared has a 15% discount off any piece of hardware with the checkout code BLACKFRIDAY2020 through Midnight (US PST). They have a few unique development boards and JTAG accessories. They have hard to find JTAG adapters and cables that are useful for ARM processor debugging. I have used the Black Magic 0.1in Pin Header JTAG cable for STM32 debugging. They also have STM32 and FPGA development boards and the Black Magic Probe for ARM Cortex MCUs.

MikroE is having deals until Tuesday December 1st on many products. This includes 25% off most Click Boards. These are small accessory boards that use the microBUS socket available on many different development boards including the low cost Microchip Curiosity Development Boards. MicroBus Click Boards are great for prototyping or general DIY projects. You can add just about any sensor, analog or digital interface, or communications module you would need.

RobotShop is discounting specific products from 5% to 50% for it's Cyber Monday sale. Of course this includes robot kits and parts, but you can also find Arduino boards, electronic learning kits, drones and Raspberry Pi development boards.

Electronics Deals Black Friday 2020

noreply@blogger.com (Aaron Clarke) — Fri, 27 Nov 2020 16:42:00 +0000

Black Friday is a great time to get a new development board or some parts for a DIY electronics project. Here are a few great online electronics stores where you can find deals.

Adafruit is having two sales, one is for 15% all items in stock with the code ADATHX see more details in the Adafruit Thanksgiving Weekend Sale blog post. The Adafruit Feather line of development boards is 20% off today only with the same code, see Adafruit Feather Friday Sale for details.

Polulu has a few different Black Friday deals. There are coupons for individual products, discounts on your entire order and the unique design your own deal sale. Follow this link to Pololu Black Friday/Cyber Monday Sale 2020 for the details.

The design your own deal allows you to propose propose a discount for up to three products. Last year I was able to get even bigger discounts than I asked for. I don't recall the original prices but I was pleasantly surprised that they were all approved and discounted further. Here are my three wishes and the deals I was offered.

Pololu item #3103: A-Star 32U4 Mini LV
You requested: 1 at $12.00 each.
We can do even better! Up to 3 at $8.88 each.

Pololu item #3110: A-Star Prime Accessory Pack
You requested: 1 at $3.00 each.
We can do even better! Up to 3 at $2.75 each.

Pololu item #3113: A-Star 32U4 Prime SV
You requested: 1 at $15.00 each.
We can do even better! Up to 3 at $14.00 each.

SparkFun Black Friday Deals are discounts from 15 to 60 percent off select products. There are not as many deals here but you might get lucky and find something you want. You can also check out this sneak peak of Cyber Monday Deals.

At the Black Friday Sale on Tindie many development boards are discounted. Tindie is a unique online store featuring independent sellers of all kinds of development boards and accessories for unique applications. If you haven't been to this site before it's a great time to look around.

SparkFun A La Carte Custom Board Designer

noreply@blogger.com (Aaron Clarke) — Fri, 30 Oct 2020 14:19:00 +0000

SparkFun Electronics is offering a web based custom board design service called SparkFun A La Carte (ALC). You can choose from a few different microcontrollers that can run Arduino programs. You also have a choice of I/O, connectors and power options. Communications options include RFID, WiFi and Bluetooth. There is a one-time design fee for each board and a per unit cost calculated as you add the components. You can get a 50% discount on the design fee until 12/31/2020 with the promocode ALCSPARKFUN50. For more information see this ALC post on the SparkFun Blog

This YouTube video on the custom board designer goes over an example design.

WIZnet IoT iOffload Contest

noreply@blogger.com (Aaron Clarke) — Mon, 08 Jul 2019 18:10:00 +0000

WIZnet is running a contest this summer for IoT designs that use either the W6100 Internet controller chip or WIZ610io Ethernet module. They are looking for 'IoT innovation' and will award five $1000 and ten $500 prizes. For more details see the IoT iOffload Contest.

WIZnet products include chips and modules to enable networking for microcontrollers. For example, the Arduino Ethernet Shield 2 uses the W5500 Ethernet Controller.

Lattice Development Board Sale and New Crowd Funded Board

noreply@blogger.com (Aaron Clarke) — Mon, 17 Dec 2018 18:45:00 +0000

Lattice has three boards on sale until December 31, 2018, click on the board links to learn more.
MachXO3-9400 Development Board - MachXO3LF-9400C PLD - includes Arduino and Raspberry Pi connectors - $39

Embedded Vision Development Kit - FPGA for embedded vision systems - $199

iCE40 UltraPlus Breakout Board - FPGA for mobile and IoT - $49

If you are interested in learning how to program the iCE40 with open source tools and hardware see the recently funded iCEBreaker FPGA project by 1BitSquared.

Atmel-ICE and Development Board Sale at Microchipdirect.com

noreply@blogger.com (Aaron Clarke) — Thu, 08 Feb 2018 16:58:00 +0000

The Atmel-ICE Full Kit is half off with coupon code TP1747 until February 28, 2018. A few other development boards are discounted as well. See details at http://www.microchipdirect.com/product/DevToolDeals

I have used the Atmel-ICE with a few different AVR microcontrollers along with the Atmel Studio 7. It works well but I have had a few small issues with JTAG debugging from Atmel Studio, usually I just restart the debug session and things work. Definitely the best tool I have found for AVR JTAG debugging. For more information see the the Atmel-ICE product page. The link to the online user's guide on that page is currently broken, I will add it to the comments if I can find a link. Here is the Atmel-ICE User Guide at DigiKey. The Youtube video Atmel Edge With Paul Rako: Debug 102 shows and unboxing and goes over the basics of the kit. The video is embedded below (no pun intended).

Embedded Coding

Keyword Spotting on ESP32-C3-Lyra V2 Using ESP-IDF

ESP32-C3 Keyword Spotting (“Yes/No”) with TFLite Micro micro_speech using the Onboard Mic (ADC)

Environment / Versions

Command Log

1) Load the ESP-IDF environment

2) Create a new project from the micro_speech example

3) Set the target to ESP32-C3

4) Install/verify ESP-IDF tools for ESP32-C3 (v6+ toolchain)

5) Build

6) Flash and open the serial monitor

Troubleshooting

Issue: Default example tried I2S and failed

Fix: Switch audio capture from I2S to ADC continuous sampling

Issue: Missing header esp_adc/adc_continuous.h

Fix: Add the esp_adc component dependency

Issue: adc_digi_output_data_t had no type1 on ESP32-C3

Fix: Use the ESP32-C3 struct layout (TYPE2)

Issue: Toolchain version mismatch on ESP-IDF v6+

Issue: idf.py fullclean refuses

Hello World on ESP32-C3-Lyra V2.0 Using ESP-IDF

prerequisites:

Hardware

Software

Step 1: Verify the board appears as a serial device

Step 2: Install required packages

Step 3: Install ESP-IDF (v5.2.3) + ESP32-C3 toolchain

Step 4: Create a local hello_world project

Step 5: Set the target to ESP32-C3

Step 6: Build, flash, and monitor

Expected output

Troubleshooting

1) idf.py: command not found

2) Wrong serial port

3) Permission denied on /dev/ttyUSB0

A Guide to Using ST Edge AI Developer Cloud

Walkthrough

1. Accessing Edge AI Developer Cloud

2. Sign in or Create an Account

3. Import a Model

4. Choose Platform & Version

5. Quantize the Model

6. Optimize the Model

7. Benchmark Across Boards

8. View Results

9. Generate Code

Conclusion

Fix for KiCad Causing Windows Shutdown

How to Build the TensorFlow Lite C API from Source Inside WSL

Step 1: Set Up WSL and Create a .wslconfig File (Optional)

Step 2: Install Prerequisites

Step 3: Install Numpy

Step 4: Install Bazelisk

Step 5: Set the Required Bazel Version

Step 6: Clone TensorFlow and Check Out the Version

Step 7: Build TensorFlow Lite C API

Step 8: Install the Library and Headers

Step 9: Verify With a Simple C Program

Conclusion

Previewing NVIDIA's Cosmos, a Generative AI Worldbuilding Tool

NVIDIA's Jetson Orin Nano Super Developer Kit

An Overview of NVIDIA's AI Tools For Developers

Black Friday 2024 Deals on Maker Electronics!

A Guide to STMicroelectronics' Edge AI Suite Tools

1. NanoEdge AI Studio

3. MEMS-Studio

MEMS-Studio is a software solution for using AI in embedded systems development. It can be downloaded from the linked webpage (MEMS-Studio). Once installed an opened, you will have access to Getting Started with MEMS-Studio - User manual and Introduction-to-MEMS-studio Video.

4. STEdgeAI-Core

STEdgeAI-Core is used for compiling edge AI models on various ST MCUs MPUs, and Smart Sensors - all in one tool. The following video gives an overview.

5. ST Edge AI Model Zoo

6. X-CUBE-AI - AI expansion pack for STM32CubeMX

7. ST High Speed Datalog

8. StellarStudioAI

9. X-Linux-AI

10. ST ToF Hand Posture

Introduction to Machine Learning on Arduino - Micro Speech - Part 2

Part 2: Changing Words

2.1 Downloading Python Script

2.1.1 Install 7-Zip

2.1.2

ESP32-C3 Keyword Spotting (“Yes/No”) with TFLite Micro `micro_speech` using the Onboard Mic (ADC)

2) Create a new project from the `micro_speech` example

Issue: Missing header `esp_adc/adc_continuous.h`

Fix: Add the `esp_adc` component dependency

Issue: `adc_digi_output_data_t` had no `type1` on ESP32-C3

Issue: `idf.py fullclean` refuses

Step 4: Create a local `hello_world` project

1) `idf.py: command not found`

3) Permission denied on `/dev/ttyUSB0`