RZ-G/RZG GStreamer: Difference between revisions

From Renesas.info
(Added section for GStreamer C Level API Demos. This is helpful for working with applications that require c programming. Will be adding code for interection with DRP-AI algorithms.)
No edit summary
Line 66: Line 66:
<pre>
<pre>
gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA ! waylandsink
gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA ! waylandsink
</pre>
== Camera Setup ==
=== MIPI Camera Setup (OV5645 camera) ===
Copy below script in a file say v412-init.sh and run the script first before using camera.
<pre>
#!/bin/sh
cru=$(cat /sys/class/video4linux/video*/name | grep "CRU")
csi2=$(cat /sys/class/video4linux/v4l-subdev*/name | grep "csi2")
# Please choose one of a following resolution then comment out the rest.
ov5645_res=1280x960
#ov5645_res=1920x1080
if [ -z "$cru" ]
then
        echo "No CRU video device founds"
else
        media-ctl -d /dev/media0 -r
        if [ -z "$csi2" ]
        then
                echo "No MIPI CSI2 sub video device founds"
        else
                media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
                media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                echo "Link CRU/CSI2 to ov5645 0-003c with format UYVY8_2X8 and resolution $ov5645_res"
        fi
fi
</pre>
=== USB Camera Setup ===
<pre>
$ rmmod uvcvideo
$ insmod /lib/modules/5.10.83-cip1-yocto-standard/kernel/drivers/media/usb/uvc/uvcvideo.ko allocators=1
</pre>
=== MIPI Camera Examples ===
==== Capture from camera and display ====
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1280,height=960 ! waylandsink
</pre>
==== Capture and Scaling and Conversion using vspmfilter ====
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=ARGB,width=1280,height=960" ! vspmfilter outbuf-alloc=true ! "video/xraw,format=RGB16,width=640,height=480" !  waylandsink
</pre>
====H.264 encode and save to a MP4 container file (using omxh264enc)====
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output.mp4
</pre>
====H.264 decode stream (using omxh264dec)====
<pre>
gst-launch-1.0 filesrc location=output.mp4 ! h264parse ! omxh264dec ! waylandsink
</pre>
=== USB Camera Examples ===
====Encode an .264 stream from camera (using omxh264enc)====
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264
</pre>
====Decode (using omxh264dec)====
<pre>
gst-launch-1.0 filesrc location=output_xsga.264 ! h264parse ! omxh264dec ! waylandsink
</pre>
</pre>


Revision as of 21:10, 18 November 2022

← RZ-G

<img height=100 style="float:right" src=1200px-Gstreamer-logo.svg.png>

Examples for RZ/G2L

Down Scaling using vspmfilter

  • Below table show range of support:
Supported solution range
Input resolution Output scale down ratio
Min 128x72 1
Max 1920x1080 15

Following are sample commands for Up/Down Scaling with h264 Elementary Stream file.

Case of Down Scaling

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=1280, height=720 ! waylandsink

Case of Down Scaling (without color format conversion)

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, width=1280, height=720 ! waylandsink

Case of Down Scaling (without color format conversion – use dmabuf)

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter dmabuf-use=true ! video/x-raw, width=1280, height=720 ! waylandsink

Color format conversion using vspmfilter

  • Below table show color format conversion using vspmfilter:
Supported color format
No. Supported color format
1 BGRA
2 BGRx
3 RGB16
4 YUY2
5 NV12

Following are sample commands for converting color format. Please set the output format which begin supported following “format=”.

Case of BGRA output

gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA ! waylandsink

Camera Setup

MIPI Camera Setup (OV5645 camera)

Copy below script in a file say v412-init.sh and run the script first before using camera. 

#!/bin/sh

cru=$(cat /sys/class/video4linux/video*/name | grep "CRU")
csi2=$(cat /sys/class/video4linux/v4l-subdev*/name | grep "csi2")

# Please choose one of a following resolution then comment out the rest.
ov5645_res=1280x960
#ov5645_res=1920x1080

if [ -z "$cru" ]
then
        echo "No CRU video device founds"
else
        media-ctl -d /dev/media0 -r
        if [ -z "$csi2" ]
        then
                echo "No MIPI CSI2 sub video device founds"
        else
                media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
                media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                echo "Link CRU/CSI2 to ov5645 0-003c with format UYVY8_2X8 and resolution $ov5645_res"
        fi
fi

USB Camera Setup

$ rmmod uvcvideo
$ insmod /lib/modules/5.10.83-cip1-yocto-standard/kernel/drivers/media/usb/uvc/uvcvideo.ko allocators=1

MIPI Camera Examples

Capture from camera and display

 gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1280,height=960 ! waylandsink

Capture and Scaling and Conversion using vspmfilter

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=ARGB,width=1280,height=960" ! vspmfilter outbuf-alloc=true ! "video/xraw,format=RGB16,width=640,height=480" !  waylandsink

H.264 encode and save to a MP4 container file (using omxh264enc)

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output.mp4

H.264 decode stream (using omxh264dec)

gst-launch-1.0 filesrc location=output.mp4 ! h264parse ! omxh264dec ! waylandsink

USB Camera Examples

Encode an .264 stream from camera (using omxh264enc)

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264

Decode (using omxh264dec)

gst-launch-1.0 filesrc location=output_xsga.264 ! h264parse ! omxh264dec ! waylandsink

Examples for RZV2L

Camera Setup

This section describes how to setup the RZV2L Camera module OV5645.

Camera Setup VGA

media-ctl -d /dev/media0 -r ;

media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;

media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/640x480 field:none]" ;

media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/640x480 field:none]" ;

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink

Camera Setup 720p

media-ctl -d /dev/media0 -r ;

media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;

media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/1280x720 field:none]" ;

media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/1280x720 field:none]" ;

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink

Down Scaling using vspmfilter

Down Scale Camera 720p video

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! waylandsink

Down Scale H264 Video File

gst-launch-1.0 filesrc location=./in720p.mp4 ! qtdemux ! queue ! h264parse ! queue ! omxh264dec ! queue ! ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 !waylandsink

Create H264 Video File

This script can be run on the RZV2L

gst-launch-1.0 videotestsrc ! omxh264enc ! filesink location=./testvideo.h264

Stream H264 Video Over IP

RZV2L(Source) Device send h264 stream using RTP:

Set the IP address to the PC or device that receives the video

gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host=<PC IP Address> port=5000

PC that receives h264 rtp stream:

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

PC receives H264 RTP Stream with VLC Player:

1) Create a SDP script file ( i.e. test.sdp ).

2) Add the following to the file. The variable IP_ADDR_HOST_VID needs to be changed to the device (RZV2L) that is hosting the video.

c=IN IP4 IP_ADDR_HOST_VID

m=video 5000 RTP/AVP 96

a=rtpmap:96 H264/90000

3) Run the following Command

vlc --avcodec-hw=vaapi test.sdp

RZ GStreamer C Level API Demos

Here is a an example of how to use the GStreamer C Code API for the RZV and RZG. The demo utilizes the RZ H.264 and VSPM library.

RZ GStreamer Demo

Retrieved from "https://renesas.info/w/index.php?title=RZ-G/RZG_GStreamer&oldid=1829"