🧠 RZ/G Artificial Intelligence and Machine Learning

RZ/G FOSS AI BSP

Renesas has created an "AI BSP" for the RZ/G family of SoCs and reference platforms.

This "BSP" adds support for a number of popular machine learning frameworks allowing users to run AI inference natively on an embedded platform.

Currently the following frameworks are supported:

ArmNN SDK
ONNX Runtime
TensorFlow Lite

Support is added to the RZ/G BSP using Yocto/OpenEmbedded meta-layers. The source code can be found on GitHub: https://github.com/renesas-rz/meta-renesas-ai.

A spreadsheet has been created showing benchmark results from using a number of popular models across all of the supported platforms: 20220728_FOSS_Deep_Learning_Object_Recognition_Renesas_RZG_v5.0.0.xlsx

RZ/G FOSS AI Demos

Renesas have created a number of simple demo applications that make use of the RZ/G AI BSP.

Each demo can be added to the RZ/G BSP by using an Yocto/OpenEmbedded meta-layer. These meta-layers can be found on GitHub.

RZ Edge AI Demo

This demo is to showcase several Machine Learning use cases in a single easy-to-use application, supported on a range of Renesas RZ/G2 evaluation platforms.

v5.0 of the demo supports five "modes":

Audio Command mode

Detects audio commands on wav files or a USB microphone feed. User can steer an arrow around the screen with their voice.

Face Estimation mode

Performs face or iris landmark detection on image files, video file or live camera stream. Facial landmarks or irises are highlighted and mapped. Inference times and FPS are recorded.

Click on the images/GIFs for animation

Pose Estimation mode

Performs pose estimation on image files, video file or live camera stream. Limb, facial and/or hand points of interest are marked with interconnecting lines. Inference times and FPS are recorded.

Click on the images/GIFs for animation

Object Detection mode

Performs object detection on a range of inputs (image file, video file and camera feed) and collates the inference and total FPS results.

Click on the images/GIFs for animation
	]

Shopping Basket mode

Mimics a supermarket use case by identifying items in a shopping basket and providing an itemised price list.

Click on the images/GIFs for animation

📑 Source code: https://github.com/renesas-rz/meta-rz-edge-ai-demo

🎬Videos: RZ/G Edge AI Demo v2.0 | RZ/G Edge AI Demo v4.0

Object Detection Demo

Note: This demo is no longer supported. Please use the RZ Edge AI Demo

This demo showcases simple Object Detection using TensorFlow Lite and mobilenet SSD models.

It will draw boxes around detected objects and identify them. Image or video files can be loaded from storage, or a USB webcam can be used.

📑 Source code: meta-object-detection-demo

Shopping Basket Demo

Note: This demo is no longer supported. Please use the RZ Edge AI Demo

This demo builds upon the Object Detection demo, but has a specific use case - an automated shopping checkout.

A custom model has been trained to recognise a number of common food items such as a can of Coke. A whole shopping basket can be placed in front of a camera and the application will identify the items and total up the cost of the shopping.

📑 Source code: meta-shopping-basket-demo

🎬Video: RZ/G FOSS AI Shopping Basket Demo