RZ-G/RZG GStreamer: Difference between revisions

From Renesas.info
No edit summary
 
(9 intermediate revisions by 3 users not shown)
Line 2: Line 2:
← [[RZ-G]]
← [[RZ-G]]


<img height=100 style="float:right" src=https://upload.wikimedia.org/wikipedia/commons/thumb/d/db/Gstreamer-logo.svg/1200px-Gstreamer-logo.svg.png>
<br>
<center>
<html><img height=100 src=https://upload.wikimedia.org/wikipedia/commons/thumb/d/db/Gstreamer-logo.svg/1200px-Gstreamer-logo.svg.png></html>
</center>
<br>


= Examples for RZ/G2L =
= Application Note by Renesas =
This User's Manual (Application Note) explains GStreamer and the custom plug-ins created by Renesas.


== Down Scaling using vspmfilter ==
Usage examples are also included.


* Below table show range of support:
* '''RZ/G2L Group and RZ/V2L Group Linux Interface Specification GStreamer User's Manual'''
{| class="wikitable"
* https://www.renesas.com/us/en/document/mas/rzg2l-group-and-rzv2l-group-linux-interface-specification-gstreamer-users-manual-software?utm_nooverride=1&language=en
|+ style="caption-side:bottom;"|''Supported solution range''
|
| Input resolution
| Output scale down ratio
|-
| Min
| 128x72
| 1
|-
| Max
| 1920x1080
| 15
|}


Following are sample commands for Up/Down Scaling with h264 Elementary Stream file.
= Sample Code by Renesas =
===Case of Down Scaling===
Sample code for using GStreamer on Renesas RZ devices are stored in '''github'''.
 
* https://github.com/renesas-rz/rz_gstreamer_sample_code
 
= Examples for RZ/G2L-V2L =
For most common use cases, refer to the gstreamer user's manual linked above. Below are reported other specific cases not covered in that document.
 
== USB Camera Examples ==
====Encode an .264 stream from camera (using omxh264enc)====
<pre>
<pre>
gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=1280, height=720 ! waylandsink
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264
</pre>
</pre>
===Case of Down Scaling (without color format conversion)===
====Stream H.264 video via Ethernet====
====Create .sdp file for VLC to receive stream using rtsp====
Let create a file named test.sdp and copy the below content in the file<br>
NOTE: The IP address is the IP4 is the IP address of the stream sender<br>
m=video 9001 RTP/AVP 96, 9001 is the port number which needs to be matched with the sender port
<pre>
<pre>
gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, width=1280, height=720 ! waylandsink
v=0
i=RZ Board Demo
c=IN IP4 192.168.86.57
s=ESP H264 STREAM
m=video 9001 RTP/AVP 96
a=rtpmap:96 H264/90000
</pre>
Save the file and launch the VLC application by clicking test.sdp file to receive the stream.
====Stream H.264 compressed test-pattern video via Ethernet to VLC media player====
<pre>
NOTE: host ip address is the ip address of PC running VLC application
gst-launch-1.0 -v videotestsrc ! queue ! videoconvert ! queue ! video/x-raw, width=320, height=240 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host= 192.168.86.36 port=9001
</pre>
</pre>
===Case of Down Scaling (without color format conversion – use dmabuf)===
====Send h.264 compressed Camera video over Ethernet to the VLC media player====
<pre>
<pre>
gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter dmabuf-use=true ! video/x-raw, width=1280, height=720 ! waylandsink
gst-launch-1.0 --gst-debug=3 v4l2src device=/dev/video1 ! video/x-raw, width=1280, height=720 ! videoconvert \
! video/x-raw, format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 use-dmabuf=false \
! video/x-h264, profile=\(string\)high,level=\(string\)4.2 ! h264parse ! rtph264pay ! queue ! udpsink host=192.168.86.36 port=9001
</pre>
</pre>


== Color format conversion using vspmfilter ==
====Send h.264 compressed MP4 video over Ethernet to the VLC media player====
* Below table show color format conversion using vspmfilter:
 
{| class="wikitable"
|+ style="caption-side:bottom;"|''Supported color format''
| No.
| Supported color format
|-
| 1
| BGRA
|-
| 2
| BGRx
|-
| 3
| RGB16
|-
| 4
| YUY2
|-
| 5
| NV12
|}
 
Following are sample commands for converting color format. Please set the output format which begin supported following “format=”.
===Case of BGRA output===
<pre>
<pre>
gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA ! waylandsink
gst-launch-1.0 -v filesrc location=sintel_trailer-1080p.mp4 ! decodebin ! omxh264enc ! rtph264pay ! udpsink host=192.168.86.36 port=9001
</pre>
</pre>
== Camera Setup ==
=== MIPI Camera Setup (OV5645 camera) ===
Copy below script in a file say v412-init.sh and run the script first before using camera.
<pre>
#!/bin/sh


cru=$(cat /sys/class/video4linux/video*/name | grep "CRU")
= RZ GStreamer C Level API Demos =
csi2=$(cat /sys/class/video4linux/v4l-subdev*/name | grep "csi2")
Here is a an example of how to use the GStreamer C Code API for the RZV and RZG. The demo utilizes the RZ H.264 and VSPM library.


# Please choose one of a following resolution then comment out the rest.
[https://github.com/renesas-rz/rz_gstreamer_demos RZ GStreamer Demo]
ov5645_res=1280x960
#ov5645_res=1920x1080


if [ -z "$cru" ]
= Examples for RZ/G2UL =
then
Compared to the big brothers the RZ/G2UL does not feature a GPU nor a HW video encoder / decoder. Nevertheless gstreamer can still be used, mainly using SW plugins and the CPU and, of course, the performance is limited due to this.
        echo "No CRU video device founds"
else
        media-ctl -d /dev/media0 -r
        if [ -z "$csi2" ]
        then
                echo "No MIPI CSI2 sub video device founds"
        else
                media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
                media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                echo "Link CRU/CSI2 to ov5645 0-003c with format UYVY8_2X8 and resolution $ov5645_res"
        fi
fi
</pre>


=== USB Camera Setup ===
== Camera ==
<pre>
=== MIPI Camera Setup (OV5645 camera) ===
$ rmmod uvcvideo
Since many VLP releases now, <code>v412-init.sh</code> is installed by default on the /home/root folder. It has to be run before the following commands can be given.
$ insmod /lib/modules/5.10.83-cip1-yocto-standard/kernel/drivers/media/usb/uvc/uvcvideo.ko allocators=1
</pre>


=== MIPI Camera Examples ===
=== MIPI Camera Examples ===
==== Capture from camera and display ====
==== Capture from camera and display ====
<pre>
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1280,height=960 ! waylandsink
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! waylandsink
</pre>
</pre>
==== Capture and Scaling and Conversion using vspmfilter ====
==== Capture and Scaling and Conversion using vspmfilter ====
<pre>
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=ARGB,width=1280,height=960" ! vspmfilter outbuf-alloc=true ! "video/xraw,format=RGB16,width=640,height=480" ! waylandsink
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=640, height=480 ! waylandsink
</pre>
</pre>
====H.264 encode and save to a MP4 container file (using omxh264enc)====
 
== Video decoding ==
As mentioned there's no HW video decoder, so SW decoding gstreamer plugins have to be used. Low resolution videos are normally played ok, medium / high resolution may be challenging.
 
=== VP8 decoding ===
<pre>
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output.mp4
gst-launch-1.0 filesrc location=./[VP8 encoded file].webm ! matroskademux ! avdec_vp8 ! videoconvert ! waylandsink
</pre>
</pre>
====H.264 decode stream (using omxh264dec)====
 
=== VP9 decoding ===
<pre>
<pre>
gst-launch-1.0 filesrc location=output.mp4 ! h264parse ! omxh264dec ! waylandsink
gst-launch-1.0 filesrc location=./[VP9 encoded file].webm ! matroskademux ! avdec_vp9 ! videoconvert ! waylandsink
</pre>
</pre>


=== USB Camera Examples ===
=== H.264 decoding ===
====Encode an .264 stream from camera (using omxh264enc)====
<pre>
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264
</pre>
====Decode (using omxh264dec)====
<pre>
<pre>
gst-launch-1.0 filesrc location=output_xsga.264 ! h264parse ! omxh264dec ! waylandsink
gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! waylandsink
</pre>
</pre>


= Examples for RZV2L =
=== H.264 decoding and down scaling ===
 
gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoscale ! videoconvert ! video/x-raw, width=640, height=480 ! waylandsink
== Camera Setup ==
This section describes how to setup the RZV2L Camera module OV5645.  
 
=== Camera Setup VGA ===
<code>media-ctl -d /dev/media0 -r ;</code>
 
<code>media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;</code>
 
<code>media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/640x480 field:none]" ;</code>
 
<code>media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/640x480 field:none]" ;</code>
 
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink</code>
 
=== Camera Setup 720p ===
<code>media-ctl -d /dev/media0 -r ;</code>
 
<code>media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;</code>
 
<code>media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/1280x720 field:none]" ;</code>
 
<code>media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/1280x720 field:none]" ;</code>
 
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink</code>
 
== Down Scaling using vspmfilter ==
 
=== Down Scale Camera 720p video ===
<code>gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! waylandsink</code>
 
=== Down Scale H264 Video File ===
gst-launch-1.0 filesrc location=./in720p.mp4 ! qtdemux ! queue ! h264parse ! queue ! omxh264dec ! queue ! <code>! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 !</code>waylandsink
 
=== Create H264 Video File ===
This script can be run on the RZV2L
 
<code>gst-launch-1.0 videotestsrc ! omxh264enc ! filesink location=./testvideo.h264</code>
 
== Stream H264 Video Over IP ==
 
=== RZV2L(Source)  Device send h264 stream using RTP: ===
'''''Set the IP address to the PC or device that receives the video'''''
 
<code>gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host=<PC IP Address> port=5000</code>
 
=== PC that receives h264 rtp stream: ===
<code>gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink</code>
 
=== PC receives H264 RTP Stream with VLC Player: ===
1) Create a SDP script file ( i.e. test.sdp ).
 
2) Add the following to the file. The variable <code>IP_ADDR_HOST_VID needs to be changed to the device (RZV2L) that is hosting the video.</code>
 
<code>c=IN IP4 IP_ADDR_HOST_VID</code>
 
<code>m=video 5000 RTP/AVP 96</code>
 
<code>a=rtpmap:96 H264/90000</code>
 
3) Run the following Command
 
<code>vlc --avcodec-hw=vaapi test.sdp</code>
==RZ GStreamer C Level API Demos==
Here is a an example of how to use the GStreamer C Code API for the RZV and RZG. The demo utilizes the RZ H.264 and VSPM library.
 
[https://github.com/renesas-rz/rz_gstreamer_demos RZ GStreamer Demo]

Latest revision as of 13:20, 6 December 2023

← RZ-G



Application Note by Renesas

This User's Manual (Application Note) explains GStreamer and the custom plug-ins created by Renesas.

Usage examples are also included.

Sample Code by Renesas

Sample code for using GStreamer on Renesas RZ devices are stored in github.

Examples for RZ/G2L-V2L

For most common use cases, refer to the gstreamer user's manual linked above. Below are reported other specific cases not covered in that document.

USB Camera Examples

Encode an .264 stream from camera (using omxh264enc)

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264

Stream H.264 video via Ethernet

Create .sdp file for VLC to receive stream using rtsp

Let create a file named test.sdp and copy the below content in the file
NOTE: The IP address is the IP4 is the IP address of the stream sender
m=video 9001 RTP/AVP 96, 9001 is the port number which needs to be matched with the sender port 

v=0
i=RZ Board Demo
c=IN IP4 192.168.86.57
s=ESP H264 STREAM
m=video 9001 RTP/AVP 96
a=rtpmap:96 H264/90000

Save the file and launch the VLC application by clicking test.sdp file to receive the stream.

Stream H.264 compressed test-pattern video via Ethernet to VLC media player

NOTE: host ip address is the ip address of PC running VLC application
gst-launch-1.0 -v videotestsrc ! queue ! videoconvert ! queue ! video/x-raw, width=320, height=240 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host= 192.168.86.36 port=9001

Send h.264 compressed Camera video over Ethernet to the VLC media player

gst-launch-1.0 --gst-debug=3 v4l2src device=/dev/video1 ! video/x-raw, width=1280, height=720 ! videoconvert \
! video/x-raw, format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 use-dmabuf=false \
! video/x-h264, profile=\(string\)high,level=\(string\)4.2 ! h264parse ! rtph264pay ! queue ! udpsink host=192.168.86.36 port=9001

Send h.264 compressed MP4 video over Ethernet to the VLC media player

 gst-launch-1.0 -v filesrc location=sintel_trailer-1080p.mp4 ! decodebin ! omxh264enc ! rtph264pay ! udpsink host=192.168.86.36 port=9001

RZ GStreamer C Level API Demos

Here is a an example of how to use the GStreamer C Code API for the RZV and RZG. The demo utilizes the RZ H.264 and VSPM library.

RZ GStreamer Demo

Examples for RZ/G2UL

Compared to the big brothers the RZ/G2UL does not feature a GPU nor a HW video encoder / decoder. Nevertheless gstreamer can still be used, mainly using SW plugins and the CPU and, of course, the performance is limited due to this.

Camera

MIPI Camera Setup (OV5645 camera)

Since many VLP releases now, v412-init.sh is installed by default on the /home/root folder. It has to be run before the following commands can be given.

MIPI Camera Examples

Capture from camera and display

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! waylandsink

Capture and Scaling and Conversion using vspmfilter

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=640, height=480 ! waylandsink

Video decoding

As mentioned there's no HW video decoder, so SW decoding gstreamer plugins have to be used. Low resolution videos are normally played ok, medium / high resolution may be challenging.

VP8 decoding

gst-launch-1.0 filesrc location=./[VP8 encoded file].webm ! matroskademux ! avdec_vp8 ! videoconvert ! waylandsink

VP9 decoding

gst-launch-1.0 filesrc location=./[VP9 encoded file].webm ! matroskademux ! avdec_vp9 ! videoconvert ! waylandsink

H.264 decoding

gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! waylandsink

H.264 decoding and down scaling

gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoscale ! videoconvert ! video/x-raw, width=640, height=480 ! waylandsink
Retrieved from "https://renesas.info/w/index.php?title=RZ-G/RZG_GStreamer&oldid=2592"