RZ-V/RZ-V2H EVK: Difference between revisions

From Renesas.info
(→‎Getting Started: Added Japan Video to Wiki)
 
(7 intermediate revisions by the same user not shown)
Line 24: Line 24:
To help expedite evaluation of the RZ/V2H we provide Pre-build Images of the RZ/V2H VLP.  
To help expedite evaluation of the RZ/V2H we provide Pre-build Images of the RZ/V2H VLP.  


* RZ/V2H VLP ( Coming Soon )
* [https://www.renesas.com/us/en/document/sws/rzv2h-ai-sdk-v300?r=25470141 '''RZ/V2H AI SDK''']  
* [https://www.renesas.com/us/en/document/sws/rzv2h-ai-sdk-v300?r=25470141 '''RZ/V2H AI SDK''']  
== AI SDK Contents ==
=== eSD Card Image ===
The eSD Card Image is a prebuilt yocto image that contains the following yocto packages
* VLP Linux
*ARM Mali Graphics ( limited license )
*Codec ( limited license )
*DRP-AI Driver
*OpenCVA ( DRP Accelerated  )
The eSD Image '''DOES NOT''' include the following
* e-CON MIPI Camera ( '''Contact [https://www.e-consystems.com/renesas/sony-starvis-imx462-ultra-low-light-camera-for-renesas-rz-v2h.asp e-CON Systems]''' )
* AI SDK Demos ( Coming Soon )
=== TVM Translator Environment ===
The TVM Translator is the recommended translator to translate AI models for the DRP-AI Driver. All the AI SDK demos use the TVM Translator. Included in the AI SDK is the DRP-AI Translator. This translator is needed for TVM installation and does not need to be installed. Installation using Docker is recommended because the TVM Environment requires a PC running Ubuntu 20.04. For information about Docker setup see '''[https://github.com/renesas-rz/docker_setup here]'''.
For information about using the TVM Translator see the [https://github.com/renesas-rz/rzv_drp-ai_tvm/tree/main '''Official Site here'''].
NOTE: Once the AI SDK Environment is setup the TVM will be installed in the the root directory /'''drp_tvm_sdk'''


= Available Demos =
= Available Demos =
Line 41: Line 63:


= DRP-AI Tools =
= DRP-AI Tools =
[[File:image 2024-03-28 192101378.png|alt=AI Model to DRP-AI Hardware Translation Flow.|center|thumb|850x850px|AI Model to DRP-AI Hardware Translation Flow. For Sparse Model Creation the recommended path after ML Framework is S1, S2, S3. For Dense Model Creation after ML Framework is D1, D2.]]
⚠️ The DRP-AI Tools will only work in Ubuntu 20.04
⚠️ The DRP-AI Tools will only work in Ubuntu 20.04


== RZ/V2H DRP-AI Translator and Quantization Tool ==
⚠️ the  flow S1 is using the pruning CLI tool. The Pruning tool also includes python expansion pack, so S1 can be integrated into the ML Framework training step.
 
== (S1) RZ/V2H Expansion Pack (Pruning Tool ) ==
The DRP-AI MAC supports '''sparse arrays''' (unstructured pruning). The expansion pack includes CLI and python libraries (expansion pack). The expansion pack allows for integration of the Pruning process into the ML Frameworks for training. The Expansion Pack supports Pytorch, Tensorflow, and Tensorflow-lite. 
 
'''[https://www.renesas.com/us/en/software-tool/drp-ai-extension-pack-pruning-tool Official Site]'''
 
== (S2,3 or D1,2 ) RZ/V2H DRP-AI TVM ==
'''[https://github.com/renesas-rz/rzv_drp-ai_tvm TVM Repository]'''
 
'''[[DRP-AI TVM|DRP-AI TVM wik]]<nowiki/>i'''
 
The Renesas TVM is the DRP-AI translator tool that converts standard ML Frameworks ( Pytorch, Tensorflow, ONNX, .. ) to DRP-AI Hardware object files. The TVM for RZ/V2H includes the following tools. This is the recommended tool for '''AI Model Translation to the DRP-AI'''.
 
* Inference Pre-processing Configuration
* Quantization Tool
* Translation Compiler
 
=== RZ/V2H DRP-AI Translator and Quantization Tool ===
'''[https://www.renesas.com/us/en/software-tool/drp-ai-translator-i8 Official Site]'''
'''[https://www.renesas.com/us/en/software-tool/drp-ai-translator-i8 Official Site]'''


This tool quantize and translates AI Models exported in the ONNX format to the RZ/V2H DRP-AI (INT8) hardware. This tool includes the following
This tool is used by the '''TVM''' for translation of AI Models. It is needed to to install the TVM. ( AI SDK includes this )
 
For Advance Translation this tool can also be used for AI Model translation instead of the Renesas TVM. Only AI Model operations listed in the RZ\V2H DRP-AI Translator Manual ( section 4.1) can be used with this translator. For AI models that have un-supported operations please use the Renesas DRP-AI TVM Tool. The tool supports quantization and translation of AI Models exported in the ONNX format to the RZ/V2H DRP-AI (INT8) hardware. This tool includes the following


* '''RZ/V2H DRP-AI Translator'''
* '''RZ/V2H DRP-AI Translator'''
* '''DRP-AI Quantization Tool'''  
* '''DRP-AI Quantization Tool'''  


The RZV2H DRP-AI MAC only performs INT8 operations, so AI Models must be quantized before translation.  
The RZV2H DRP-AI MAC only performs INT8 operations, so AI Models must be quantized before translation.
 
AI Models must use only the supported AI operations listed in the RZ\V2H DRP-AI Translator Manual ( section 4.1). For AI models that have un-supported operations please use the Renesas DRP-AI TVM Tool.


====== Retraining Quantized Models ======
====== Retraining Quantized Models ======
Line 63: Line 104:
* Function to parse the mode
* Function to parse the mode


== RZ/V2H DRP-AI TVM ==
= Getting Started =
'''[https://github.com/renesas-rz/rzv_drp-ai_tvm Official Site]'''
The [https://renesas-rz.github.io/rzv_ai_sdk/latest/ '''RZV2H AI SDK page'''] is the recommended place to start evaluating the RZ/V2H EVK. The page includes detailed information about setup of the EVK, and of the AI SDK Development Environment.  Building a Yocto BSP,. links to Yocto Packages and information is [https://renesas-rz.github.io/rzv_ai_sdk/3.00/howto_build_aisdk_v2h.html here]..


'''[[DRP-AI TVM|DRP-AI TVM wik]]<nowiki/>i'''
[https://www.renesas.com/us/en/video/rzv2h-ai-applications-tutorial-getting-started-v300 '''RZ/V2H AI Applications Tutorial Getting Started V3.00 | Renesas''']
 
== Setup AI SDK Environment and Evaluate Prebuilt Image ==
 
* '''Step 1''' [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started.html#step1Obtain&#x20;EVK&#x20;Board Obtain EVK Board]
** Links to the AI SDK and EVK
* '''Step 2''' [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started.html#step2 Obtain Necessary Environmen]t
** List Software and hardware requirments
* '''Step 3''' [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started.html#step5 Obtain the AI SDK]
** Download the [https://www.renesas.com/us/en/document/sws/rzv2h-ai-sdk-v300 '''RZ/V2H AI SDK''']
** AI SDK Source not needed for initial evaluation
** See [[RZ-V/RZ-V2H EVK#Prebuilt Images|Prebuilt Images]] for more information
* '''Step 4''' [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started.html#step5 Setup SDK]
* '''Step 5'''  [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started.html#step6 Build AI Application]
** Requires TVM Runtime Library to run applications
*** Instructions steps 5 - 7 show how to get the library from AI SDK
*** TVM Runtime for RZV2H can be found [https://github.com/renesas-rz/rzv_drp-ai_tvm/tree/main/obj/build_runtime/V2H here] too.
*** [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started_v2h.html Deploy TVM Runtime to Board]
**** Section 2 "Deploy Application to the Board"
** [https://github.com/renesas-rz/rzv_ai_sdk/tree/v3.00/R01_object_detection Link] to Demo Applications mentioned in this step
*** The link included information on how to build, deploy and run the application
*** More Demos see [[RZ-V/RZ-V2H EVK#Available Demos|here]].
* '''Step 6''' Deploy and Run Applications
** Follow the [https://renesas-rz.github.io/rzv_ai_sdk/3.00/getting_started_v2h.html RZ/V2H EVK Getting Started page]
 
== Yocto Build ==
The Renesas  AI SDK includes information about building the AI SDK using the Yocto Build Framework. This is recommended to customize the Linux BSP.
 
* Download the '''[https://www.renesas.com/us/en/document/sws/rzv2h-ai-sdk-v300-source-code AI SDK Source]'''  
* Follow the direction "[https://renesas-rz.github.io/rzv_ai_sdk/3.00/howto_build_aisdk_v2h.html How to build RZ/V2H AI SDK Source Code]"


== RZ/V2H Expansion Pack (Pruning Tool ) ==
The BSP <u>'''includes'''</u> the following packages
The DRP-AI MAC supports sparse arrays (unstructured pruning). The expansion pack tool are python libraries that can be imported into the training environment.


'''[https://www.renesas.com/us/en/software-tool/drp-ai-extension-pack-pruning-tool Official Site]'''
* RZ/V2H Linux VLP
* DRP-AI Drivers
* DRP OpenCV Accelerator
* Mali Graphics ( Restricted License )
* Renesas Codec ( H.265/H.264 ) ( Restricted License )


== RZ/V2H Implementation Guide ==
'''<u>Not included</u>'''
This is included in the '''RZ/V2H DRP-AI Yocto Package'''.  The guide provides step by step examples of how to use the RZ/V2H DRP-Tools on trained AI models .


Coming Soon
* Demos ( see here for [[RZ-V/RZ-V2H EVK#Available Demos|information]] )
* e-Con MIPI Driver ( see [[RZ-V/RZ-V2H EVK#MIPI Camera|link]] )
** follow step 5 to install e-CON Camera
* Unrestricted  Mali Graphics and Codec License
** Request for [https://www.renesas.com/us/en/request-access/1823516 Unrestricted Graphic Mali License]
* CM33 and CR7 Multi-OS Support and FSP ( Coming Soon )


= Getting Started =
== CM33 and CR7 Multi-OS and FSP ==
This has not been released for RZ/V2H yet.


= Hardware =
= Hardware =

Latest revision as of 15:32, 9 April 2024

RZ/V2H EVK Board by Renesas

General Information

  • Official RZ/V2H Device Website *
    • Product Page
    • Please review the **Documentation & Downloads** section
  • Software Downloads
    • RZ/V2H Linux Package
    • Mali Graphic Library
    • Video Codec Library
    • ARM Mali Full ISP
    • RZ/V2L DRP-AI Support Package
    • RZ/V2H Cortex-M33 and Cortex-CR8 Multi-OS Package (optional)
    • Robotic Operating System (ROS)
    • RZ/V2H Cortex CM33 FSP

Prebuilt Images

To help expedite evaluation of the RZ/V2H we provide Pre-build Images of the RZ/V2H VLP.

AI SDK Contents

eSD Card Image

The eSD Card Image is a prebuilt yocto image that contains the following yocto packages

  • VLP Linux
  • ARM Mali Graphics ( limited license )
  • Codec ( limited license )
  • DRP-AI Driver
  • OpenCVA ( DRP Accelerated )

The eSD Image DOES NOT include the following

  • e-CON MIPI Camera ( Contact e-CON Systems )
  • AI SDK Demos ( Coming Soon )

TVM Translator Environment

The TVM Translator is the recommended translator to translate AI models for the DRP-AI Driver. All the AI SDK demos use the TVM Translator. Included in the AI SDK is the DRP-AI Translator. This translator is needed for TVM installation and does not need to be installed. Installation using Docker is recommended because the TVM Environment requires a PC running Ubuntu 20.04. For information about Docker setup see here.

For information about using the TVM Translator see the Official Site here.

NOTE: Once the AI SDK Environment is setup the TVM will be installed in the the root directory /drp_tvm_sdk

Available Demos

DRP-AI

  • vSLAM (Contact Sales Representative for demo)
  • DRP-AI Demos
    • Included with the DRP-AI Package. AI models include
      • Classification (Resnet50)
      • Object Detection ( YoloX and Yolov2 )
      • Segmentation ( DeepLabv2 )
  • Renesas RZ/V2H AI SDK

DRP-AI Tools

AI Model to DRP-AI Hardware Translation Flow.
AI Model to DRP-AI Hardware Translation Flow. For Sparse Model Creation the recommended path after ML Framework is S1, S2, S3. For Dense Model Creation after ML Framework is D1, D2.

⚠️ The DRP-AI Tools will only work in Ubuntu 20.04

⚠️ the flow S1 is using the pruning CLI tool. The Pruning tool also includes python expansion pack, so S1 can be integrated into the ML Framework training step.

(S1) RZ/V2H Expansion Pack (Pruning Tool )

The DRP-AI MAC supports sparse arrays (unstructured pruning). The expansion pack includes CLI and python libraries (expansion pack). The expansion pack allows for integration of the Pruning process into the ML Frameworks for training. The Expansion Pack supports Pytorch, Tensorflow, and Tensorflow-lite.

Official Site

(S2,3 or D1,2 ) RZ/V2H DRP-AI TVM

TVM Repository

DRP-AI TVM wiki

The Renesas TVM is the DRP-AI translator tool that converts standard ML Frameworks ( Pytorch, Tensorflow, ONNX, .. ) to DRP-AI Hardware object files. The TVM for RZ/V2H includes the following tools. This is the recommended tool for AI Model Translation to the DRP-AI.

  • Inference Pre-processing Configuration
  • Quantization Tool
  • Translation Compiler

RZ/V2H DRP-AI Translator and Quantization Tool

Official Site

This tool is used by the TVM for translation of AI Models. It is needed to to install the TVM. ( AI SDK includes this )

For Advance Translation this tool can also be used for AI Model translation instead of the Renesas TVM. Only AI Model operations listed in the RZ\V2H DRP-AI Translator Manual ( section 4.1) can be used with this translator. For AI models that have un-supported operations please use the Renesas DRP-AI TVM Tool. The tool supports quantization and translation of AI Models exported in the ONNX format to the RZ/V2H DRP-AI (INT8) hardware. This tool includes the following

  • RZ/V2H DRP-AI Translator
  • DRP-AI Quantization Tool

The RZV2H DRP-AI MAC only performs INT8 operations, so AI Models must be quantized before translation.

Retraining Quantized Models

It is recommended to retrains a model after quantization to reduce changes to the inference accuracy . For that purpose the Quantization tool includes optional parameters that point to the data set and dataset parsing function.

Beside the AI Model below are additional items needed to properly quantize a model.

  • Customer Dataset ( does not need to be the full dataset )
  • Function to parse the mode

Getting Started

The RZV2H AI SDK page is the recommended place to start evaluating the RZ/V2H EVK. The page includes detailed information about setup of the EVK, and of the AI SDK Development Environment. Building a Yocto BSP,. links to Yocto Packages and information is here..

RZ/V2H AI Applications Tutorial Getting Started V3.00 | Renesas

Setup AI SDK Environment and Evaluate Prebuilt Image

Yocto Build

The Renesas AI SDK includes information about building the AI SDK using the Yocto Build Framework. This is recommended to customize the Linux BSP.

The BSP includes the following packages

  • RZ/V2H Linux VLP
  • DRP-AI Drivers
  • DRP OpenCV Accelerator
  • Mali Graphics ( Restricted License )
  • Renesas Codec ( H.265/H.264 ) ( Restricted License )

Not included

  • Demos ( see here for information )
  • e-Con MIPI Driver ( see link )
    • follow step 5 to install e-CON Camera
  • Unrestricted Mali Graphics and Codec License
  • CM33 and CR7 Multi-OS Support and FSP ( Coming Soon )

CM33 and CR7 Multi-OS and FSP

This has not been released for RZ/V2H yet.

Hardware

Power Supply

Recommended Power Supplies ( Recommend 100Ws )

Description Link Comments
🙂 Amazon Basics Link 2xUSB-C, 2xUSB,

1x65-100W, 1x18W, 2x12W

🙂 Nekteck Link 1x USBC 100W

+LCD Monitors

LCD Monitors
Resolution Touchscreen Description Link Comments
🙂 1920x1080 Yes Wimaxit 14 Inch 🛒 buy This is a very nice screen. Good color and easy to press the icons. The LCD input is mini-HDMI and comes with a mini-HDMI to HDMI cable. The RZ board is micro-HDMI. You will also need to buy a HDMI to micro-HDMI adapter. Or, buy a mini-HDMI to HDMI-micro cable. Note the the micro-USB connection on the LCD for touchscreen does not work, but the USB-C connector does work for touchscreen. You will simply need a USB-C to USB-A cable or adapter.
🙂 1920x1080 Yes ELECROW 10.1 Inch 🛒 buy This is a very nice screen. You will also need to buy a HDMI to micro-HDMI cable.
🙂 1280x800 Yes SunFounder 10.1 inch 🛒 buy All the cables that you will need are included.
🙂 1024x600 Yes Ingcool 7 inch
7IP-CAPLCD		 🛒 buy If you attach the 'Touch' USB connection on the LCD to the RZ/G2L board, it will both power the LCD board as well as enable touch support. This kit comes with HDMI cables.
🙂 800x480 Yes GeeekPi 5 Inch
EP-0081		 🛒 buy HDMI cable not included.

MIPI Camera

Supported MIPI Cameras

Resolutions Description Link Comments
🙂 VGA ( 640x480) , HD ( 1270x720), FHD(1920,1080)

Format YUYV FPS 30 and 60

e-CAM22_CURZH Link This Camera was designed for the EVK MIPI Connector Interface. Uses the Sony Starvis I IMX462.
Retrieved from "https://renesas.info/w/index.php?title=RZ-V/RZ-V2H_EVK&oldid=2928"