GStreamer

From Renesas.info
Revision as of 14:48, 4 October 2023 by MicBis (talk | contribs) (→‎Examples for RZ/G2UL: Added some examples for RZ/G2UL that does not have GPU nor HW codec)

← RZ-G



Application Note by Renesas

This User's Manual (Application Note) explains GStreamer and the custom plug-ins created by Renesas.

Usage examples are also included.

Examples for RZ/G2L

Down Scaling using vspmfilter

  • Below table show range of support:
Supported solution range
Input resolution Output scale down ratio
Min 128x72 1
Max 1920x1080 15

Following are sample commands for Up/Down Scaling with h264 Elementary Stream file.

Case of Down Scaling

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=1280, height=720 ! waylandsink

Case of Down Scaling (without color format conversion)

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, width=1280, height=720 ! waylandsink

Case of Down Scaling (without color format conversion – use dmabuf)

gst-launch-1.0 filesrc location=./<FullHDsize_h264_file> ! h264parse ! queue ! omxh264dec ! queue ! vspmfilter dmabuf-use=true ! video/x-raw, width=1280, height=720 ! waylandsink

Color format conversion using vspmfilter

  • Below table show color format conversion using vspmfilter:
Supported color format
No. Supported color format
1 BGRA
2 BGRx
3 RGB16
4 YUY2
5 NV12

Following are sample commands for converting color format. Please set the output format which begin supported following “format=”.

Case of BGRA output

gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! queue ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA ! waylandsink

Camera Setup

MIPI Camera Setup (OV5645 camera)

Copy below script in a file say v412-init.sh and run the script first before using camera. 

#!/bin/sh

cru=$(cat /sys/class/video4linux/video*/name | grep "CRU")
csi2=$(cat /sys/class/video4linux/v4l-subdev*/name | grep "csi2")

# Please choose one of a following resolution then comment out the rest.
ov5645_res=1280x960
#ov5645_res=1920x1080

if [ -z "$cru" ]
then
        echo "No CRU video device founds"
else
        media-ctl -d /dev/media0 -r
        if [ -z "$csi2" ]
        then
                echo "No MIPI CSI2 sub video device founds"
        else
                media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
                media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/$ov5645_res field:none]"
                echo "Link CRU/CSI2 to ov5645 0-003c with format UYVY8_2X8 and resolution $ov5645_res"
        fi
fi

USB Camera Setup

$ rmmod uvcvideo
$ insmod /lib/modules/5.10.83-cip1-yocto-standard/kernel/drivers/media/usb/uvc/uvcvideo.ko allocators=1

MIPI Camera Examples

Capture from camera and display

 gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1280,height=960 ! waylandsink

Capture and Scaling and Conversion using vspmfilter

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=640, height=480 !  waylandsink

v4l2src resolution must be specified to get the scaling working.

H.264 encode and save to a MP4 container file (using omxh264enc)

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output.mp4

H.264 decode stream (using omxh264dec)

gst-launch-1.0 filesrc location=output.mp4 ! h264parse ! omxh264dec ! waylandsink

USB Camera Examples

Encode an .264 stream from camera (using omxh264enc)

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! video/x-raw,width=1280,height=960 ! vspmfilter dmabuf-use=true ! video/x-raw,format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 ! video/x-h264,profile=\(string\)high,level=\(string\)4.2 ! filesink location=output_xsga.264

Decode (using omxh264dec)

gst-launch-1.0 filesrc location=output_xsga.264 ! h264parse ! omxh264dec ! waylandsink

Stream H.264 video via Ethernet

Create .sdp file for VLC to receive stream using rtsp

Let create a file named test.sdp and copy the below content in the file
NOTE: The IP address is the IP4 is the IP address of the stream sender
m=video 9001 RTP/AVP 96, 9001 is the port number which needs to be matched with the sender port 

v=0
i=RZ Board Demo
c=IN IP4 192.168.86.57
s=ESP H264 STREAM
m=video 9001 RTP/AVP 96
a=rtpmap:96 H264/90000

Save the file and launch the VLC application by clicking test.sdp file to receive the stream.

Stream H.264 compressed test-pattern video via Ethernet to VLC media player

NOTE: host ip address is the ip address of PC running VLC application
gst-launch-1.0 -v videotestsrc ! queue ! videoconvert ! queue ! video/x-raw, width=320, height=240 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host= 192.168.86.36 port=9001

Send h.264 compressed Camera video over Ethernet to the VLC media player

gst-launch-1.0 --gst-debug=3 v4l2src device=/dev/video1 ! video/x-raw, width=1280, height=720 ! videoconvert \
! video/x-raw, format=NV12 ! omxh264enc control-rate=2 target-bitrate=10485760 interval_intraframes=14 periodicty-idr=2 use-dmabuf=false \
! video/x-h264, profile=\(string\)high,level=\(string\)4.2 ! h264parse ! rtph264pay ! queue ! udpsink host=192.168.86.36 port=9001

Send h.264 compressed MP4 video over Ethernet to the VLC media player

 gst-launch-1.0 -v filesrc location=sintel_trailer-1080p.mp4 ! decodebin ! omxh264enc ! rtph264pay ! udpsink host=192.168.86.36 port=9001

Send and receive h264 video between the devices over Ethernet using gstreamer

Sender:
gst-launch-1.0 -v filesrc location=sintel_trailer-1080p.mp4 ! decodebin ! omxh264enc ! rtph264pay ! udpsink host=192.168.86.36 port=9001

Receiver:
gst-launch-1.0 -v udpsrc port=9001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

Examples for RZV2L

Camera Setup

This section describes how to setup the RZV2L Camera module OV5645.

Camera Setup VGA

media-ctl -d /dev/media0 -r ;

media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;

media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/640x480 field:none]" ;

media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/640x480 field:none]" ;

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink

Camera Setup 720p

media-ctl -d /dev/media0 -r ;

media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]" ;

media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/1280x720 field:none]" ;

media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/1280x720 field:none]" ;

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! waylandsink

Down Scaling using vspmfilter

Down Scale Camera 720p video

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! waylandsink

Down Scale H264 Video File

gst-launch-1.0 filesrc location=./in720p.mp4 ! qtdemux ! queue ! h264parse ! queue ! omxh264dec ! queue ! ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 !waylandsink

Create H264 Video File

This script can be run on the RZV2L

gst-launch-1.0 videotestsrc ! omxh264enc ! filesink location=./testvideo.h264

Stream H264 Video Over IP

RZV2L(Source) Device send h264 stream using RTP:

Set the IP address to the PC or device that receives the video

gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! vspmfilter dmabuf-use=true ! video/x-raw, width=640, height=480 ! queue ! omxh264enc ! queue ! rtph264pay ! queue ! udpsink host=<PC IP Address> port=5000

PC that receives h264 rtp stream:

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

PC receives H264 RTP Stream with VLC Player:

1) Create a SDP script file ( i.e. test.sdp ).

2) Add the following to the file. The variable IP_ADDR_HOST_VID needs to be changed to the device (RZV2L) that is hosting the video.

c=IN IP4 IP_ADDR_HOST_VID

m=video 5000 RTP/AVP 96

a=rtpmap:96 H264/90000

3) Run the following Command

vlc --avcodec-hw=vaapi test.sdp

RZ GStreamer C Level API Demos

Here is a an example of how to use the GStreamer C Code API for the RZV and RZG. The demo utilizes the RZ H.264 and VSPM library.

RZ GStreamer Demo

Examples for RZ/G2UL

Compared to the big brothers the RZ/G2UL does not feature a GPU nor a HW video encoder / decoder. Nevertheless gstreamer can still be used, mainly using SW plugins and the CPU and, of course, the performance are limited due to this.

Camera

MIPI Camera Setup (OV5645 camera)

Since many VLP releases now, v412-init.sh is installed by default on the /home/root folder. It has to be run before the following commands.

MIPI Camera Examples

Capture from camera and display

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! waylandsink

Capture and Scaling and Conversion using vspmfilter

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=ARGB,width=1280,height=960 ! vspmfilter outbuf-alloc=true ! video/x-raw, format=BGRA, width=640, height=480 ! waylandsink

Video decoding

As mentioned there's no HW video decoder, so SW decoding gstreamer plugins have to be used. Low resolution videos are normally played ok, medium / high resolution may be challenging.

VP8 decoding

gst-launch-1.0 filesrc location=./[VP8 encoded file].webm ! matroskademux ! avdec_vp8 ! videoconvert ! waylandsink

VP9 decoding

gst-launch-1.0 filesrc location=./[VP9 encoded file].webm ! matroskademux ! avdec_vp9 ! videoconvert ! waylandsink

H.264 decoding

gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! waylandsink

H.264 decoding and down scaling

gst-launch-1.0 filesrc location=./[H.264 encoded file].mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoscale ! videoconvert ! video/x-raw, width=640, height=480 ! waylandsink
Retrieved from "https://renesas.info/w/index.php?title=RZ-G/RZG_GStreamer&oldid=2533"