Building optimized Raspberry Pi 4 libraries

Product support
1

I’m trying to use OpenVINO with the Raspberry Pi 4 (to use the Intel NCS 2 accelerator with the new USB 3 ports) and I’m attempting to build optimized versions of OpenCV and the OpenVINO platform (https://github.com/opencv/dldt) since the release version of OpenVINO is bound to the Raspberry Pi 3 and an older version of Python (3.5) than I’m needing to use for the next stage of this project.

I’m running into issues, though, as the OpenCV seems to be building armv7 code and the OpenVINO is building armv8 and then the linker gets angry at me.

With the Dockerfile at https://github.com/shelbydavis/BalenaOpenVINO/blob/master/Dockerfile it gives me an error:
[main] [ 20%] Linking CXX shared library /dldt/inference-engine/bin/armv7l/Release/lib/libformat_reader.so
[main] /usr/bin/ld: /dldt/inference-engine/temp/opencv_4.1.2_debian9arm/lib/libopencv_videoio.so.4.1.2: error adding symbols: file in wrong format
[main] collect2: error: ld returned 1 exit status
[main]
[main] make[2]: *** [samples/common/format_reader/CMakeFiles/format_reader.dir/build.make:133: /dldt/inference-engine/bin/armv7l/Release/lib/libformat_reader.so] Error 1
[main]
[main] make[1]: *** [CMakeFiles/Makefile2:2652: samples/common/format_reader/CMakeFiles/format_reader.dir/all] Error 2
[main] make[1]: *** Waiting for unfinished jobs…

if I change the configuration of the OpenVINO cmake to -DCMAKE_CXX_FLAGS=’-march=armv7-a’ it fails with
[main] CMake Error at /usr/share/cmake-3.13/Modules/CMakeTestCXXCompiler.cmake:45 (message):
[main] The C++ compiler
[main] “/usr/bin/c++”
[main] is not able to compile a simple test program.
[main] It fails with the following output:
[main] Change Dir: /inference-engine-build/CMakeFiles/CMakeTmp
[main] Run Build Command:"/usr/bin/make" “cmTC_9857a/fast”
[main] /usr/bin/make -f CMakeFiles/cmTC_9857a.dir/build.make CMakeFiles/cmTC_9857a.dir/build
[main] make[1]: Entering directory ‘/inference-engine-build/CMakeFiles/CMakeTmp’
[main] Building CXX object CMakeFiles/cmTC_9857a.dir/testCXXCompiler.cxx.o
[main] /usr/bin/c++ -march=armv7-a -o CMakeFiles/cmTC_9857a.dir/testCXXCompiler.cxx.o -c /inference-engine-build/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
[main] cc1plus: error: unknown value ‘armv7-a’ for -march
[main] cc1plus: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a native; did you mean ‘armv8-a’?
[main] make[1]: *** [CMakeFiles/cmTC_9857a.dir/build.make:66: CMakeFiles/cmTC_9857a.dir/testCXXCompiler.cxx.o] Error 1
[main] make[1]: Leaving directory ‘/inference-engine-build/CMakeFiles/CMakeTmp’
[main] make: *** [Makefile:121: cmTC_9857a/fast] Error 2
[main] CMake will not be able to correctly generate this project.
[main] Call Stack (most recent call first):
[main] CMakeLists.txt:12 (project)

As added clues, when it builds the numpy package it gets installed as armv7. What are the correct compiler flags for generating RPI4 code?

Thanks

2

Also, not passing an architecture into the OpenVINO build produces the same error as passing armv8-a

3

Getting closer to success following the advice in the OpenVINO forum: https://software.intel.com/en-us/forums/computer-vision/topic/813265

I’ll post the Dockerfile when I get successful results or get stuck again.

6

Hi and sorry for the late answer. Thank you for sharing your progress, we look forward to it! Tell us if you have any other doubts and we’ll do our best to help

1 Like
8

OK, apparently you need to build the OpenVINO Inference engine before OpenCV, and then tell OpenCV where the inference engine is, and not the other way around.
Also, the Inference Engine python tools include a ‘statistics_collector’ which has not been completely checked into the GIT repository… so I just used sed to delete that from the CMake.

Anyone who wants to play around with this is welcome to use the Dockerfile in my Git Repo at https://github.com/shelbydavis/BalenaOpenVINO

10

Just as an update, I got the ‘optimized’ libraries built, but there’s no communication with the Intel NCS 2 stick. Apparently this is a known issue with running in a Docker image, and I tried the solution of building a custom libUSB without udev support, but that still seems to not work.
(from https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_docker_linux.html)

  • UDEV events are not forwarded to the container by default it does not know about device reconnection

I’ve got it working on Rasbian running as the OS (not virtualized), but I really don’t know if this is a thing that’s going to be working on Balena. Because I like tilting at windmills, now I’m going to go test out the Google Coral USB accelerator…

11

From your repo it looks like you’re using a single container application… In that case the container should be running as privileged and have access to /dev not sure what could be the problem. If you want to share more about what things you tried to make your application see the intel stick maybe we can suggest some things…

13

I don’t have it running right now (it’s not in a place I can easily get to today, the Balena interface does make it so much more convenient to test these things out) so I can’t test out stuff today, but what I noticed is that before I add the UDEV rules

  • cp /dldt/inference-engine/thirdparty/movidius/mvnc/src/97-myriad-usbboot.rules /etc/udev/rules.d
  • udevadm control --reload-rules
  • udevadm trigger

when I’m in the container on the console) I can see the NCS stick using lsusb and the sample script hello_query_device (https://github.com/opencv/dldt/tree/2019/inference-engine/samples/hello_query_device) but afterwards the hello_query_device no longer sees the USB stick.

14

Hi @shelbydavis, sounds strange that this stops working for no apparent reason. Once you have it running again, could you take a look at the udev logs via udevadm control --log-priority=debug and the journalctl logs?

Perhaps we could see what’s going wrong in the logs. Let us know if we could help out with debugging or the setup.

Next, your project sounds interesting. I wonder what cool project you are building :slightly_smiling_face: Please share how it goes in general, your test with Coral and what you come up with at the end.

16

Getting similar issues with Coral, by the way. I can make Docker containers that run on a Raspbian host running full Docker that can access the USB, but deploying with Balena make it unable to connect to the USB devices. I’ll try to collect logs sometime soon, but I’m trying it on a RPi3 right now ‘just in case’

17

And Both the Google Coral and Intel NCS 2 using OpenVINO work just fine with the Raspberry Pi 3.

For the curious. Both compile the most recent released versions of their respective libraries and execute the first demo code snippet of their tutorials.

18

Hi,
Can you please clarify the balenaOS & the Raspbian versions that you are trying this on?
Could you also try defining the udev rules directly in the config.json, so that the device picks the up right away after the boot?
Let me point you to the respective balenaOS docs:

Looking forward to get those logs.
Kind regards,
Thodoris

1 Like
20

Hi @shelbydavis, might be a bit late to the party, but I updated the Google Coral USB tpu example here https://github.com/balena-io-playground/edge-tpu-usb to work with balenaOS. I commented the Dockerfile, but it was necessary to enable UDEV and added the udev rule in containers. Hope it helps

21

The Balena OS image I’m using is balenalib/raspberrypi4-64-debian-python:3.6-buster-build
Host OS is balenaOS 2.44.0+rev3
When I ran it just with the Rasbian, it’s the 2019-09-26 image of Rasbian Buster Lite

Here’s the journalctl output, hopefully capturing what you need for the udev output.
journalctl.log (315.1 KB)

23

Hi @shelbydavis can you try the project i posted above on your RPI4 with coral USB. The key part in that project is to have the udev rule file in the container here: https://github.com/balena-io-playground/edge-tpu-usb/blob/master/99-tpu.rules . Something in the coral lib seems to invoke that can prompt the TPU firmware to show up correctly. I tested this on a pi4 on OS 2.44 last week and it was working great. I also think for these types of projects its probably best to target the 32bit userspace for the rpi since all the tools are built for that, so i would recommend using FROM balenalib/raspberrypi3-debian.

25

It definitely looks like 32 bit userspace and adding to /mnt/boot/config.json worked. I’m able to run on the Intel NCS 2 and Coral devices on the RPi 4!

26 
"os" : {                                                                                                                                                                                                                                     
  "udevRules": {                                                                                                                                                                                                                             
    "97" : "SUBSYSTEM==\"usb\", ATTRS{idProduct}==\"2150\", ATTRS{idVendor}==\"03e7\", GROUP=\"users\", MODE=\"0660\", ENV{ID_MM_DEVICE_IGNORE}=\"1\"",                                                                                      
    "98" : "SUBSYSTEM==\"usb\", ATTRS{idProduct}==\"2485\", ATTRS{idVendor}==\"03e7\", GROUP=\"users\", MODE=\"0660\", ENV{ID_MM_DEVICE_IGNORE}=\"1\"",                                                                                      
    "99" : "SUBSYSTEM==\"usb\", ATTRS{idProduct}==\"f63b\", ATTRS{idVendor}==\"03e7\", GROUP=\"users\", MODE=\"0660\", ENV{ID_MM_DEVICE_IGNORE}=\"1\""                                                                                       
  }                                                                                                                                                                                                                                          
}

This is the relevant addition to /mnt/boot/config.json for the Intel NCS 2, since I probably won’t get around to a writeup of this part of my project at least until late December. (Publishing deadline is March 13 for the project I’m working on)

27

@shelbydavis thats awesome and thanks for sharing that udevRules snip with us, that will be very helpful for others. Looking forward to the write up and good luck with the deadline!

30

Really cool @shelbydavis. I’m new to balena but balena-engine CLI just looks like docker cli, so once I’ve built the container image I was expecting to just run it but it’s failing … How is the container produced meant to be start ?

31

@cledorze your message lacks some context. Which container image ? How did you build it ? How do you run it ? How does it fail ?