RZ/V ISP: Difference between revisions
No edit summary |
|||
(One intermediate revision by one other user not shown) | |||
Line 55: | Line 55: | ||
''<small>2 ) Due to the BSP v1.0.1 ISP resaved area size ( 48MB ) this resolution is not supported. The reserved memory needs to be increased.</small>'' | ''<small>2 ) Due to the BSP v1.0.1 ISP resaved area size ( 48MB ) this resolution is not supported. The reserved memory needs to be increased.</small>'' | ||
= '''RZ/V2L ISP Monitor Demo''' = | = '''RZ/V2L Simple ISP Monitor Demo''' = | ||
[https://github.com/renesas-rz/rzv2l_camera_mipi_isp RZ/V2L ISP Sample Camera Demo] | [https://github.com/renesas-rz/rzv2l_camera_mipi_isp RZ/V2L ISP Sample Camera Demo] | ||
This demo show how to implement the RZ/V2L Simple ISP with OV5645 Cameras. This expands on the RZ/V2L ISP Sample Application that is included with the RZ/V2L ISP Package with all the resolutions listed in the section above. | This demo show how to implement the RZ/V2L Simple ISP with OV5645 Cameras. This expands on the RZ/V2L ISP Sample Application that is included with the RZ/V2L ISP Package with all the resolutions listed in the section above. | ||
=Run RZV2L Simple ISP Demos= | |||
======Requirements ====== | |||
*Linux BSP SDK : The SDK must include the DRP-AI and ISP packages. Refer to Linux BSP Release Note instructions (r01an6221ej<version>-rzv2l-linux.pdf) Section 6. | |||
*RZV2L DRP-AI Sample Application : ( rzv2l_drpai-sample-application_ver5.00.tar.gz ) | |||
*Power : A powered hub or PC may not provide sufficient power. A USB Power Block is suggested. | |||
======Build ISP Monitoring Application====== | |||
Detailed instructions for setup and building are located in the '''Application Note Revision''' documents in "Simple ISP Sample Application Note Revision" package. | |||
*⚠️ You must first follow the Simple ISP Sample Application Note Revision instructions (r11an0563ej<version>-rzv2l-simple-isp.pdf) Section 2.0 and 2.6 | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
|Simple build instructions for ISP Monitioring Application | |||
|- | |||
| | |||
These instructions are a summary of what is explained in the package Release Notes. | |||
<pre> | |||
export ISP_FILE=r11an0561ej0100-rzv2l-isp-sp.zip | |||
# Step 1. Extract the ISP package and ISP sample application files. Enter the ISP Montitoring source directory. | |||
unzip $ISP_FILE -d isp_work | |||
cd isp_work | |||
tar zxvf rzv2l_isp-sample-application_ver1.00.tar.gz | |||
cd app_isp_monitoring/src | |||
# Step 2. Start the SDK | |||
source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux | |||
# Step 3. Build the source with make | |||
make | |||
</pre> | |||
|} | |||
======Build ISP Tiny YoloV2 Application====== | |||
Detailed instructions for setup and building are located in the '''Application Note Revision''' documents in "Simple ISP Sample Application Note Revision" package. | |||
*⚠️ You must first follow the Simple ISP Sample Application Note Revision instructions (r11an0563ej<version>-rzv2l-simple-isp.pdf) Section 2.0 and 2.7 | |||
This build requires the DRP-AI Tiny Yolo V2 Cam source is needed for this build ( /rzv2l_drpai-sample-application/app_tinyyolov2_cam/src ). | |||
This build requires the DRP-AI Tiny Yolo V2 Cam directory "tinyyolov2_cam" is needed for this build ( /rzv2l_drpai-sample-application/app_tinyyolov2_cam/exe ). | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
| Simple build instructions for ISP Tiny Yolo V2 Application | |||
|- | |||
| | |||
These instructions are a summary of what is explained in the package Release Notes. | |||
<pre> | |||
mkdir drp-ai-supporrt | |||
export DRP_WORKDIR=$PWD/drp-ai-support | |||
mkdir isp-support | |||
export ISP_ WORKDIR=$PWD/isp-support | |||
export APP_WORKDIR=$PWD | |||
# Step 1. Extract the DRP Appicatiion Code to $DRP_WORK | |||
unzip r11an0549ej0500-rzv2l-drpai-sp.zip -d $DRP_WORK | |||
cd $DRP_WORK/rzv2l_drpai-sample-application/ | |||
tar -xvf rzv2l_drpai-sample-application_ver5.00.tar.gz -C . | |||
cd $APP_WORKDIR | |||
# Step 2. Extract the ISP Application Code to $ISP_WORK | |||
unzip r11an0561ej0100-rzv2l-isp-sp.zip -d $ISP_WORK | |||
cd $ISP_WORK | |||
tar -xvf rzv2l_isp-sample-application_ver1.00.tar.gz -C . | |||
cd $APP_WORKDIR | |||
# Step 3. Copy TinyYoloV2 source code to ISP TinyYoloV2 application source directory | |||
cp -r drp-ai-support/rzv2l_drpai-sample-application/app_tinyyolov2_cam/src/ ./isp-support/app_tinyyolov2_cam/src/ | |||
# Step 4. Apply Patch | |||
cd $ISP_WORKDIR/app_tinyyolov2_cam/src/ | |||
patch -p 1 < rzv2l_app_tinyyolov2_cam_usb2mipi.patch | |||
# Step 5. Setup SDK | |||
source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux | |||
# Step . Build Source | |||
make | |||
</pre> | |||
|} | |||
= '''RZ/V2M ISP usage''' = | |||
======H.265/H.264 Multi Codec (VCD): How to start encoding after only 1 raw frame from the CSI camera====== | |||
instead of using B_IMAGE_StartMovieStream, use these functions in the following order: | |||
<pre> | |||
B_IMAGE_StartObjectToMovieStream | |||
B_IMAGE_Imread | |||
B_OBJ_BeginReadingData | |||
B_IMAGE_EncodeRequestStream | |||
</pre> |
Latest revision as of 12:36, 29 November 2023
The RZ/V Series has two version of the Image Signal Processor (ISP)
RZ/V2L Simple ISP Support Package
RZV/2M Full ISP Support Package
RZ/V2L Simple ISP (Memory)
The Simple ISP uses a working memory area for processing camera streams. This working memory area is defined in the device memory space define ISP Reserved memory area ( see Figure ). In the Application Layer uses the Video 4 Linux API (VIDIOC_REQBUFS, VIDIOC_QUERYBUF) to allocated memory. How much of this memory is used is defined by the camera device resolution, pixel color depth, and capture frame buffer count. The ISP revered memory space is defined in the RZV2L Linux Device Tree File ( r9a07g04512-smarc.dts ).
BSP v1.01 file : r9a07g054l2-smarc.dts isp_work0: SIMPLE_ISP { reusable; reg = <0x0 0xBD000000 0x0 0x03000000>; };
RZV2L BSP v1.0.1 Memory Area
OV5645 Supported Resolutions | ||||
Width | Height | Pixel Depth | Number Frames | Total Memory Usage |
640 | 480 | 2 | 6 | 3,686,400 |
1280 | 720 | 2 | 6 | 11,059,200 |
1920 | 1080 | 2 | 6 | 24,883,200 |
2592 | 1944 *1 | 2 | 6 | 60,466,146 *2 |
1 ) Maximum Resolution for OV5645. For AI inference please uses smaller resolution.
2 ) Due to the BSP v1.0.1 ISP resaved area size ( 48MB ) this resolution is not supported. The reserved memory needs to be increased.
RZ/V2L Simple ISP Monitor Demo
This demo show how to implement the RZ/V2L Simple ISP with OV5645 Cameras. This expands on the RZ/V2L ISP Sample Application that is included with the RZ/V2L ISP Package with all the resolutions listed in the section above.
Run RZV2L Simple ISP Demos
Requirements
- Linux BSP SDK : The SDK must include the DRP-AI and ISP packages. Refer to Linux BSP Release Note instructions (r01an6221ej<version>-rzv2l-linux.pdf) Section 6.
- RZV2L DRP-AI Sample Application : ( rzv2l_drpai-sample-application_ver5.00.tar.gz )
- Power : A powered hub or PC may not provide sufficient power. A USB Power Block is suggested.
Build ISP Monitoring Application
Detailed instructions for setup and building are located in the Application Note Revision documents in "Simple ISP Sample Application Note Revision" package.
- ⚠️ You must first follow the Simple ISP Sample Application Note Revision instructions (r11an0563ej<version>-rzv2l-simple-isp.pdf) Section 2.0 and 2.6
Simple build instructions for ISP Monitioring Application |
These instructions are a summary of what is explained in the package Release Notes. export ISP_FILE=r11an0561ej0100-rzv2l-isp-sp.zip # Step 1. Extract the ISP package and ISP sample application files. Enter the ISP Montitoring source directory. unzip $ISP_FILE -d isp_work cd isp_work tar zxvf rzv2l_isp-sample-application_ver1.00.tar.gz cd app_isp_monitoring/src # Step 2. Start the SDK source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux # Step 3. Build the source with make make |
Build ISP Tiny YoloV2 Application
Detailed instructions for setup and building are located in the Application Note Revision documents in "Simple ISP Sample Application Note Revision" package.
- ⚠️ You must first follow the Simple ISP Sample Application Note Revision instructions (r11an0563ej<version>-rzv2l-simple-isp.pdf) Section 2.0 and 2.7
This build requires the DRP-AI Tiny Yolo V2 Cam source is needed for this build ( /rzv2l_drpai-sample-application/app_tinyyolov2_cam/src ).
This build requires the DRP-AI Tiny Yolo V2 Cam directory "tinyyolov2_cam" is needed for this build ( /rzv2l_drpai-sample-application/app_tinyyolov2_cam/exe ).
Simple build instructions for ISP Tiny Yolo V2 Application |
These instructions are a summary of what is explained in the package Release Notes. mkdir drp-ai-supporrt export DRP_WORKDIR=$PWD/drp-ai-support mkdir isp-support export ISP_ WORKDIR=$PWD/isp-support export APP_WORKDIR=$PWD # Step 1. Extract the DRP Appicatiion Code to $DRP_WORK unzip r11an0549ej0500-rzv2l-drpai-sp.zip -d $DRP_WORK cd $DRP_WORK/rzv2l_drpai-sample-application/ tar -xvf rzv2l_drpai-sample-application_ver5.00.tar.gz -C . cd $APP_WORKDIR # Step 2. Extract the ISP Application Code to $ISP_WORK unzip r11an0561ej0100-rzv2l-isp-sp.zip -d $ISP_WORK cd $ISP_WORK tar -xvf rzv2l_isp-sample-application_ver1.00.tar.gz -C . cd $APP_WORKDIR # Step 3. Copy TinyYoloV2 source code to ISP TinyYoloV2 application source directory cp -r drp-ai-support/rzv2l_drpai-sample-application/app_tinyyolov2_cam/src/ ./isp-support/app_tinyyolov2_cam/src/ # Step 4. Apply Patch cd $ISP_WORKDIR/app_tinyyolov2_cam/src/ patch -p 1 < rzv2l_app_tinyyolov2_cam_usb2mipi.patch # Step 5. Setup SDK source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux # Step . Build Source make |
RZ/V2M ISP usage
H.265/H.264 Multi Codec (VCD): How to start encoding after only 1 raw frame from the CSI camera
instead of using B_IMAGE_StartMovieStream, use these functions in the following order:
B_IMAGE_StartObjectToMovieStream B_IMAGE_Imread B_OBJ_BeginReadingData B_IMAGE_EncodeRequestStream