Chromium HTML Browser

Please refer to this page for how to add Chromium to your system: RZ-G/Chromium

Qt

Build Qt outside of Yocto

First of all the RZ/G2 SDK is needed: core-image-qt-sdk is what you want to build. By default it is installed in /opt/poky/2.4.3 (RZ/G2E-M-N-H).

Install gperf and nss-dev to target rootfs (needed by qtwebengine, TODO: still not working because of nss includes not found).

Actually if gperf is not there, qtwebengine is not selected by the configure, so no problem in building, in other words no need to explicitly set -skip qtwebengine.

Steps to build (refer to this script for more details):

clone (potentially checkout specific version)
init repository
mkdir/cd qt5-build
../configure ….. (many flags)
make
make install
rsync to sysroot location (e.g. SD)

About flags, a notable one is: -qpa, should be set to wayland or wayland-egl. This page has more info. -no-feature-xcomposite-egl, important otherwise the install fails at link stage.

Version 5.6.3 and the newly built version can coexist. If not changed Qt is installed into /usr/local/Qt5.x.y.

Qt apps development environment set-up

In order to develop a Qt application it would be easier to install Qt on the host PC, that includes also the IDE (Qt creator).

Once installed in Ubuntu, Qt creator icon should be visible in the application list, or it can be launched from the command prompt:

 ~/Qt5.12.10/Tools/QtCreator/bin/qtcreator.sh

If not already done, before launching Qt creator it is required to source the SDK environment setup script, for example:

source /opt/poky/2.4.3/environment-setup-aarch64-poky-linux

Qt creator is normally configured to develop, debug and run x86 applications. In order to cross develop, debug and run applications on the RZ/G2 some configurations are needed.

First, a new device has to be added, Tools -> Options -> Devices, then add a new device as per screenshot below:

Obviously adapting the IP address to match the RZ/G2 board IP address. Note that SFTP (openssh-sftp-server) and/or rsync must be installed on the target to be able to deploy the binaries.

Then go to the Kits tab and add a new Kit in the manual section:

Now Qt version tab:

Compilers:

And finally Debuggers:

Once the kit is configured, you can pick one of the many Qt examples available in the standard installation and make sure to use the Kit(s) just created:

At this point you should be able to build, debug and run Qt applications on a RZ/G2 board.

OpenCV with OpenCL

Build OpenCV using Yocto (enabling OpenCL)

OpenCV can be built also using Yocto but it won't include the support for OpenCL by default. There is a way to build OpenCV (version 3.3.0 in Rocko, version 4.1.0 in Dunfell) using Yocto by adding:

PACKAGECONFIG_append_pn-opencv = " gtk opencl"

to local.conf. The opencv recipe has to be modified as well, in opencv_3.3.bb or opencv_4.1.bb, replace:

PACKAGECONFIG[opencl] = "-DWITH_OPENCL=ON,-DWITH_OPENCL=OFF,opencl-headers virtual/opencl-icd,"

with:

PACKAGECONFIG[opencl] = "-DWITH_OPENCL=ON,-DWITH_OPENCL=OFF,"

Actually normally you don't want to modify the original recipe but rather write you own opencv_3.3.bbappend to do that. Anyway the OpenCV rocko version is rather old, newer versions have much better performances.

Build OpenCV from sources

First of all we need to build an SDK and rootfs WITHOUT OpenCV support, the libraries have to be added manually to the rootfs. The rootfs system is important to make sure that opencv gets compiled correctly, actually cmake searches for libraries and dependencies there and then generates the appropriate makefile.

To do this, please follow the instruction available in the latest release note for RZ/G2E-N-M-H, or this document for RZ/G2L . Do not forget to source the environment setup file, e.g:

source /opt/poky/2.4.3/environment-setup-aarch64-poky-linux

or

source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux

Then we're ready to start building OpenCV:

git clone https://github.com/opencv/opencv

We also need the extra packages:

git clone git://github.com/opencv/opencv_contrib.git contrib

Then:

cd opencv
git checkout tags/4.5.1
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE -DOPENCV_EXTRA_MODULES_PATH=../../contrib/modules \
-DCMAKE_INSTALL_PREFIX=/PATH_TO_OPENCV_INSTALL/usr -D INSTALL_C_EXAMPLES=ON \
-DCMAKE_TOOLCHAIN_FILE=/PATH_TO_AARCH64_CMAKE/rzg2_aarch64-gnu.toolchain.cmake ..

You would want to adjust "PATH_TO_OPENCV_INSTALL" and "PATH_TO_AARCH64_CMAKE".

The rzg2_arch64-gnu.toolchain.cmake file is needed to configure cmake for a cross-build. This is where we specify where cmake has to search for tools and libraries, the paths in this file should be potentially adjusted.

It is also important to specify a directory where the opencv package should be installed, we have to make sure we select a path where it is easy to copy to the final rootfs, a directory inside home should be fine. Actually we can also select the final rootfs directly and then copy from there to the target. This file can be obtained here:

wget https://raw.githubusercontent.com/seebe/rzg_stuff/master/opencv_cl_sample_code/rzg2_aarch64-gnu.toolchain.cmake

It is important to check the output, we want the GTK libraries to be recognized, otherwise the GUI examples won't work. If something is missing, then the rootfs shall be rebuilt to include the missing packages. If something goes wrong you could also try to source the SDK environment script (environment-setup-aarch64-poky-linux).

Instead, if everything's fine, then we are ready to build:

make -j$(nproc)

If it is successful (it'll take a while) we can install it:

make install

The openCV library is going to be installed in the path of the host machine specified in the cmake command (PATH_TO_OPENCV_INSTALL), e.g.:

/home/USER/opencv_install/usr

The home folder is used to avoid that the cross compiled library are mixed with potentially installed native libraries (x86_64). In some cases the CMAKE_INSTALL_PREFIX may be ignored, if this happens, manually modify the default installation path in the cmake_install.cmake file.

Another point to highlight is that OpenCV libraries are built by default with OpenCL support enabled (WITH_OPENCL ON).

Now, to use this library we should copy it to the rootfs, both on the host (for the cross compilation of the examples) and the target.

Assuming the target rootfs is mounted in the path /media/user/RZ_ext:

sudo cp -rpf usr/* /media/USER/RZ_ext/usr/
sync

For the host:

sudo cp -rpf usr/* /opt/poky/2.4.3/sysroots/aarch64-poky-linux/usr/

or

sudo cp -rpf usr/* /opt/poky/3.1.5/sysroots/aarch64-poky-linux/usr/

Now let's build some examples.

git clone https://github.com/seebe/rzg_stuff/tree/master/
cd opencv_cl_sample_code/simple

The first example is simply opening an image and displaying it.

cmake -DCMAKE_TOOLCHAIN_FILE=../rzg2_aarch64-gnu.toolchain.cmake .

Then:

make

Now we can download the binary onto the target:

scp DisplayImage root@TARGET_IP_ADDRESS:/home/root

On the RZ/G2, it can be run by giving the following command:

./DisplayImage Cherries.jpg

The previous example uses OpenCV but it does not use any OpenCL feature. In order to try this, we need to use an example that uses the CPU or the GPU taking advantage of the OpenCL libraries. First of all, install the libraries (contact sales rep to get them) as described in the section 5 "How to Apply" of the Release Note and verify that the libraries are working by testing the example (section 6.2).

Now we build and deploy another OpenCV example that uses the Transparent APIs.

cd ..
cd MatUMat

Similarly to what was done for the simple example:

cmake -DCMAKE_TOOLCHAIN_FILE=../rzg2_aarch64-gnu.toolchain.cmake .

Then:

make

Now we can download the binary onto the target:

scp Mat root@TARGET_IP_ADDRESS:/home/root
scp UMat root@TARGET_IP_ADDRESS:/home/root
scp image.jpg root@TARGET_IP_ADDRESS:/home/root

On the RZ/G2 by running Mat we load the image.jpg, perform some image processing and display the result image. It also prints on the terminal the time to process that image:

root@hihope-rzg2m:~# ./Mat
0.396389

UMat, instead, uses the GPU, when we run it the first time:

root@hihope-rzg2m:~# ./UMat
[ WARN:0] global /home/micbis/OpenCL/OpenCV/opencv/modules/core/src/utils/filesystem.cpp (489) getCacheDirectory Using world accessible cache directory. This may be not secure: /var/tmp/
1.86048

So slower than the CPU. But the next times:

root@hihope-rzg2m:~# ./UMat
[ WARN:0] global /home/micbis/OpenCL/OpenCV/opencv/modules/core/src/utils/filesystem.cpp (489) getCacheDirectory Using world accessible cache directory. This may be not secure: /var/tmp/
0.019532

That is roughly 20x faster than the CPU on this kind of processing: color conversion, Gaussian Blur and Canny edge. A good improvement, uh? However this kind of result is not guaranteed, there might be situations where the GPU is slower, others where the improvement is even higher. Of course it also depends on the GPU vs CPU performance. In the case above the GPU of the RZ/G2M is quite powerful, different derivatives may have different result.

Note that at the moment on the RZ/G2L when executing the UMat example you may get the following error:

OpenCL program build log: imgproc/color_rgb
Status -11: CL_BUILD_PROGRAM_FAILURE
-D depth=0 -D scn=3 -D PIX_PER_WI_Y=1 -D dcn=1 -D bidx=0 -D STRIPE_SIZE=1
<built-in>:161:9: error: expected member name or ';' after declaration specifiers
int32_t depth;             /**< The image depth. */
~~~~~~~ ^
<built-in>:1:15: note: expanded from here
#define depth 0
              ^
<built-in>:161:8: error: expected ';' at end of declaration list
int32_t depth;             /**< The image depth. */
       ^
error: Compiler frontend failed (error code 62)
OpenCL error CL_INVALID_WORK_GROUP_SIZE (-54) during call: clEnqueueNDRangeKernel('row_filter_C1_D0', dims=2, globalsize=640x1712x1, localsize=16x16x1) sync=false

Open Source Panfrost Mali Driver for RZ/G2L

How to use the Open Source Panfrost GPU Driver
Page: RZ-G/Panfrost_for_RZG2L

DRM Interface to Display Driver

If you are not using the Weston desktop, you can directly access the display driver to write images to the LCD.
This means you can use the tradition legacy Frame Buffer interface (/dev/fb0) or the Direct Rendering Manager (DRM) interface (/dev/dri/card0)
This page will discuss in more details 🡆 RZ-G/DRM

Image Overlay using OpenGL

https://github.com/renesas-rz/demo-camera-image-overlay
Capture camera images to memory, then use hardware acceleration (OpenGL ES) to overlay images/text and then do h.264 encoding