DRP-AI TVM
The Renesas TVM is the Extension package of Apache TVM Deep Learning Complier for Renesas DRP-AI accelerators powered by EdgeCortix MERA™. The TVM is a software framework that translates Neural Networks to run on the Renesas MPUs. While the AI Translator (see section above) can translate ONNX models to the DRP-AI hardware, it is restricted by the supported AI operations. This can restrict the number of supported AI Models. The TVM Translator expands the number of supported AI models for the RZV processors (currently RZV2MA). The TVM translates ONNX models by delegating the generated output between the DRP-AI and CPU.
This is the TVM Software framework based on the Apache TVM. The TVM includes python support libraries, and sample scripts. The python scripts follow the Apache TVM framework API found here.
- The TVM Provides the following
- Wider supported range of AI Networks that can run on the DRP-AI and CPU.
- Translate AI models from ONNX files
- Translate AI models from PyTorch PT saved models. ( For other supported AI Software Frameworks see Apache TVM)
- Translate models to run on CPU only. This allows models to run on RZG.
Official RZ\V2MA TVM Translator Github repo
Installation Directions Readme
Current Supported Renesas MPUs
- RZV2MA
- RZV2M
Generated Output
- DRP-AI + CPU
- CPU only
Application Files
A TVM application need the following files.
- Apache TVM Runtime Library
- Generated TVM files
- Preprocessor Binaries
- Preprocessing CPP files
- EdgeCortix MERA Wrapper CPP files
Apache TVM Runtime Library
This is the TVM runtime library. This a pre-compiled library for the RZV2M and RZV2MA BSP. It is included in the Renesas TVM repository.
Generated TVM files
The file that are generated using the tutorial scripts included in the repository.
Preprocessor Binaries
Neural Network require preprocessing of the input images before an inference can be run. This can involve resize, crop, and format conversion to match the input source to the expected inference input. In addition inference do not process RGB pixels instead the images must be converted to float and normalized. These preprocessing operation will increase the total inference time when done on the CPU. The Renesas TVM provides DRP-AI binaries to accelerate this process.
The DRP-AI Preprocessing library must operate as follows
- All preprocessing operations ( Total 6 ) need to be executed in sequence
- input image must be follows
- YUV Format
- Max Resolution : 4096x2160
NOTE: This library is optional. It is provided to accelerate preprocessing.
Preprocessing CPP files
This is the CPP Wrapper files that utilize the Preprocessor Binaries.
EdgeCortix MERA Wrapper CPP files
These files contain the application CPP API functions for loading and running the TVM.
Implementation examples
AI Model Performance
The link below shows the performance of several AI Vision Models converted using the TVM and Tested on the RZV2MA. The TVM is capable of converting modesl from exported trained models in the standard ONNX format, pyTorch IMage Models TIMM, and Pytorch Class Models.