First of all the RZ/G2 SDK is needed: core-image-qt-sdk is what you want to build. By default it is installed in /opt/poky/2.4.3 (RZ/G2E-M-N-H).
Install gperf and nss-dev to target rootfs (needed by qtwebengine, TODO: still not working because of nss includes not found).
Actually if gperf is not there, qtwebengine is not selected by the configure, so no problem in building, in other words no need to explicitly set -skip qtwebengine.
Steps to build (refer to this script for more details):
About flags, a notable one is: -qpa, should be set to wayland or wayland-egl. This page has more info. -no-feature-xcomposite-egl, important otherwise the install fails at link stage.
Version 5.6.3 and the newly built version can coexist. If not changed Qt is installed into /usr/local/Qt5.x.y.
A complete Qt 5.15.2 build and integration sample can be found here.
In order to develop a Qt application it would be easier to install Qt on the host PC, that includes also the IDE (Qt creator).
Once installed in Ubuntu, Qt creator icon should be visible in the application list, or it can be launched from the command prompt:
~/Qt5.12.10/Tools/QtCreator/bin/qtcreator.sh
If not already done, before launching Qt creator it is required to source the SDK environment setup script, for example:
source /opt/poky/2.4.3/environment-setup-aarch64-poky-linux
Qt creator is normally configured to develop, debug and run x86 applications. In order to cross develop, debug and run applications on the RZ/G2 some configurations are needed.
First, a new device has to be added, Tools -> Options -> Devices, then add a new device as per screenshot below:
Obviously adapting the IP address to match the RZ/G2 board IP address. Note that SFTP (openssh-sftp-server) and/or rsync must be installed on the target to be able to deploy the binaries.
Then go to the Kits tab and add a new Kit in the manual section:
Now Qt version tab:
Compilers:
And finally Debuggers:
Once the kit is configured, you can pick one of the many Qt examples available in the standard installation and make sure to use the Kit(s) just created:
At this point you should be able to build, debug and run Qt applications on a RZ/G2 board.
OpenCV can be built also using Yocto but it won't include the support for OpenCL by default. There is a way to build OpenCV (version 3.3.0 in Rocko, version 4.1.0 in Dunfell) using Yocto by adding:
PACKAGECONFIG_append_pn-opencv = " gtk opencl"
to local.conf. The opencv recipe has to be modified as well, in opencv_3.3.bb or opencv_4.1.bb, replace:
PACKAGECONFIG[opencl] = "-DWITH_OPENCL=ON,-DWITH_OPENCL=OFF,opencl-headers virtual/opencl-icd,"
with:
PACKAGECONFIG[opencl] = "-DWITH_OPENCL=ON,-DWITH_OPENCL=OFF,"
Actually normally you don't want to modify the original recipe but rather write you own opencv_3.3.bbappend to do that. Anyway the OpenCV rocko version is rather old, newer versions have much better performances.
First of all we need to build an SDK and rootfs WITHOUT OpenCV support, the libraries have to be added manually to the rootfs. The rootfs system is important to make sure that opencv gets compiled correctly, actually cmake searches for libraries and dependencies there and then generates the appropriate makefile.
To do this, please follow the instruction available in the latest release note for RZ/G2E-N-M-H, or this document for RZ/G2L . Do not forget to source the environment setup file, e.g:
source /opt/poky/2.4.3/environment-setup-aarch64-poky-linux
or
source /opt/poky/3.1.5/environment-setup-aarch64-poky-linux
Then we're ready to start building OpenCV:
git clone https://github.com/opencv/opencv
We also need the extra packages:
git clone git://github.com/opencv/opencv_contrib.git contrib
Then:
cd opencv git checkout tags/4.5.1 mkdir build cd build cmake -DCMAKE_BUILD_TYPE=RELEASE -DOPENCV_EXTRA_MODULES_PATH=../../contrib/modules \ -DCMAKE_INSTALL_PREFIX=/PATH_TO_OPENCV_INSTALL/usr -D INSTALL_C_EXAMPLES=ON \ -DCMAKE_TOOLCHAIN_FILE=/PATH_TO_AARCH64_CMAKE/rzg2_aarch64-gnu.toolchain.cmake ..
You would want to adjust "PATH_TO_OPENCV_INSTALL" and "PATH_TO_AARCH64_CMAKE".
The rzg2_arch64-gnu.toolchain.cmake
file is needed to configure cmake for a cross-build. This is where we specify where cmake has to search for tools and libraries, the paths in this file should be potentially adjusted.
It is also important to specify a directory where the opencv package should be installed, we have to make sure we select a path where it is easy to copy to the final rootfs, a directory inside home should be fine. Actually we can also select the final rootfs directly and then copy from there to the target. This file can be obtained here:
wget https://raw.githubusercontent.com/seebe/rzg_stuff/master/opencv_cl_sample_code/rzg2_aarch64-gnu.toolchain.cmake
It is important to check the output, we want the GTK libraries to be recognized, otherwise the GUI examples won't work. If something is missing, then the rootfs shall be rebuilt to include the missing packages. If something goes wrong you could also try to source the SDK environment script (environment-setup-aarch64-poky-linux).
Instead, if everything's fine, then we are ready to build:
make -j$(nproc)
If it is successful (it'll take a while) we can install it:
make install
The openCV library is going to be installed in the path of the host machine specified in the cmake command (PATH_TO_OPENCV_INSTALL), e.g.:
/home/USER/opencv_install/usr
The home folder is used to avoid that the cross compiled library are mixed with potentially installed native libraries (x86_64). In some cases the CMAKE_INSTALL_PREFIX may be ignored, if this happens, manually modify the default installation path in the cmake_install.cmake
file.
Another point to highlight is that OpenCV libraries are built by default with OpenCL support enabled (WITH_OPENCL ON).
Now, to use this library we should copy it to the rootfs, both on the host (for the cross compilation of the examples) and the target.
Assuming the target rootfs is mounted in the path /media/user/RZ_ext:
sudo cp -rpf usr/* /media/USER/RZ_ext/usr/ sync
For the host:
sudo cp -rpf usr/* /opt/poky/2.4.3/sysroots/aarch64-poky-linux/usr/
or
sudo cp -rpf usr/* /opt/poky/3.1.5/sysroots/aarch64-poky-linux/usr/
Now let's build some examples.
git clone https://github.com/seebe/rzg_stuff/tree/master/ cd opencv_cl_sample_code/simple
The first example is simply opening an image and displaying it.
cmake -DCMAKE_TOOLCHAIN_FILE=../rzg2_aarch64-gnu.toolchain.cmake .
Then:
make
Now we can download the binary onto the target:
scp DisplayImage root@TARGET_IP_ADDRESS:/home/root
On the RZ/G2, it can be run by giving the following command:
./DisplayImage Cherries.jpg
The previous example uses OpenCV but it does not use any OpenCL feature. In order to try this, we need to use an example that uses the CPU or the GPU taking advantage of the OpenCL libraries. First of all, install the libraries (contact sales rep to get them) as described in the section 5 "How to Apply" of the Release Note and verify that the libraries are working by testing the example (section 6.2).
Now we build and deploy another OpenCV example that uses the Transparent APIs.
cd .. cd MatUMat
Similarly to what was done for the simple example:
cmake -DCMAKE_TOOLCHAIN_FILE=../rzg2_aarch64-gnu.toolchain.cmake .
Then:
make
Now we can download the binary onto the target:
scp Mat root@TARGET_IP_ADDRESS:/home/root scp UMat root@TARGET_IP_ADDRESS:/home/root scp image.jpg root@TARGET_IP_ADDRESS:/home/root
On the RZ/G2 by running Mat we load the image.jpg, perform some image processing and display the result image. It also prints on the terminal the time to process that image:
root@hihope-rzg2m:~# ./Mat 0.396389
UMat, instead, uses the GPU, when we run it the first time:
root@hihope-rzg2m:~# ./UMat [ WARN:0] global /home/micbis/OpenCL/OpenCV/opencv/modules/core/src/utils/filesystem.cpp (489) getCacheDirectory Using world accessible cache directory. This may be not secure: /var/tmp/ 1.86048
So slower than the CPU. But the next times:
root@hihope-rzg2m:~# ./UMat [ WARN:0] global /home/micbis/OpenCL/OpenCV/opencv/modules/core/src/utils/filesystem.cpp (489) getCacheDirectory Using world accessible cache directory. This may be not secure: /var/tmp/ 0.019532
That is roughly 20x faster than the CPU on this kind of processing: color conversion, Gaussian Blur and Canny edge. A good improvement, uh? However this kind of result is not guaranteed, there might be situations where the GPU is slower, others where the improvement is even higher. Of course it also depends on the GPU vs CPU performance. In the case above the GPU of the RZ/G2M is quite powerful, different derivatives may have different result.
Note that at the moment on the RZ/G2L when executing the UMat example you may get the following error:
OpenCL program build log: imgproc/color_rgb Status -11: CL_BUILD_PROGRAM_FAILURE -D depth=0 -D scn=3 -D PIX_PER_WI_Y=1 -D dcn=1 -D bidx=0 -D STRIPE_SIZE=1 <built-in>:161:9: error: expected member name or ';' after declaration specifiers int32_t depth; /**< The image depth. */ ~~~~~~~ ^ <built-in>:1:15: note: expanded from here #define depth 0 ^ <built-in>:161:8: error: expected ';' at end of declaration list int32_t depth; /**< The image depth. */ ^ error: Compiler frontend failed (error code 62) OpenCL error CL_INVALID_WORK_GROUP_SIZE (-54) during call: clEnqueueNDRangeKernel('row_filter_C1_D0', dims=2, globalsize=640x1712x1, localsize=16x16x1) sync=false