32-bit container (based on balenalib) on Pi-4 64 doesn't work

Hi,

I’m migrating to RPI 4 64 OS and have problems with my 32-bit app. My container relies on balenalib/armv7hf-debian. All is built ok including the correct architecture used (see below).
However, uname -a within the container returns aarch64 and my service won’t start with the following error:

filebeat Failed to initialize: trying to run 32Bit binary on 64Bit system

Why the container built runs on wrong architecture? Everything works well with RPI 3.

Thanks in advance,

Jiri

[Info] Building on arm02
[Info] Pulling previous images for caching purposes…
[Success] Successfully pulled cache images
[filebeat] Step 1/6 : FROM balenalib/armv7hf-debian
[filebeat] —> d24e54e09892
[filebeat] Step 2/6 : COPY ./filebeat /filebeat/filebeat
[filebeat] —> 4dc2e0d4dff1
[filebeat] Step 3/6 : COPY ./certs /filebeat/certs
[filebeat] —> eb178873382e
[filebeat] Step 4/6 : COPY ./filebeat.yml /filebeat/filebeat.yml
[filebeat] —> 480d55474a40
[filebeat] Step 5/6 : WORKDIR /filebeat/
[filebeat] —> Running in f7b88cc9341a
[filebeat] Removing intermediate container f7b88cc9341a
[filebeat] —> 75862a868b6c
[filebeat] Step 6/6 : ENTRYPOINT ["./filebeat"]
[filebeat] —> Running in 34d2c0410a43
[filebeat] Removing intermediate container 34d2c0410a43
[filebeat] —> 94e2275ebc4e
[filebeat] Successfully built 94e2275ebc4e
[Info] Generating image deltas from release feacc09dade4fb187851e7bdc0178b95 (id: 2201531)

Hi @filemon

container orchestration uses the kernel of the host operating system.
As you’re using an RPi 4 with 64 bit OS, your container will use the 64 bit kernel and work as that, at least that is my understanding. I think the only way to get a 32 bit container running is to use an 32 bit kernel if I am not mistaken.

Cheers,

Nico

1 Like

Actually not @nmaas87 if you run a 64bit balenaOS you can run 32bit containers. You can’t run 64bit containers on 32bit balenaOS :slight_smile:

Let me explore this @filemon could you please share more information of the service that you are running? I can see that the problem might be that binary… so not sure if this is an open-source project we can try to replicate!

1 Like

I was aware that 32 bit software does run on 64 bit hardware/kernel without a hitch, but I was not aware that you could “pressure” the environment to really switch it to 32 bit, including memory region locks, etc. interesting :). Do you need to put a special tag to the container so that it completely switches to 32 bit mode and not “mere emulation”?

Cheers,

Nico

1 Like

@nmaas87 let me explore the technical details of how this works. I can imagine that balenaOS can run balenaEngine for 32bit and 64bit images.

Since 2019 balenaOS for Raspberry Pi 4 was designed with this intention → Run a mixed-architecture app using 64-bit balenaOS on the Raspberry Pi 4

Having said that, I found some cases where the 32 bit image needs something specific of 32bit OS (e.g. TensorFlow)

1 Like

Yes. I had an impression if I pick the 32-bit balenalib image (like armv7hf-debian in my case) I would see armv7hl when running ‘uname -a’. This is what happens when I run this container on raspberrypi3 base image.

I see aarch64 instead. This filebeat service is one case, which I might to solve by recompilation. However I have another service, which I can’t change easily to 64-bit, so I would like to better understand this mixed-architecture feature beforehands. I will pack the filebeat service so you can check yourself @mpous.

that would be helpful @filemon Thanks :smiley:

So I packed my filebeat service → Dropbox - filebeat.zip - Simplify your life

Behaviour: Container runs ok on raspberrypi3 base image. System architecture is

Linux 769ef6ce34cc 5.10.95-v7 #1 SMP Thu Feb 17 11:43:01 UTC 2022 armv7l GNU/Linux

Container doesn’t run ok on raspberrypi4 base image with the message

Failed to initialize: trying to run 32Bit binary on 64Bit system.

I can give you access to our fleet/devices if you wish. Your help is greatly appreciated @mpous.

Just to clarify @filemon @nmaas87 from the ARM CPU side, the ARMv8-A is backwards compatible with AArch32. That means that from the Linux kernel, the 64bits kernel can run 32-bit applications if configured properly. Then running them in containers might just work fine.

I would see armv7hl when running ‘uname -a’.

The kernel is your host’s kernel. This is why you see the 64 bit kernel instead of the 32bit.

@filemon is the application compiled with deprecated compiled flags and things like that? Some instructions have been deprecated were supported in aarch32 but will not work in armv8. Read more here Documentation – Arm Developer

Hi @mpous, initial problem solved by recompilation of the filebeat for aarch64.

I have a question though. My fleet contains now a mix of RPI3 and RPI4. When I issue a ‘balena push’ command, raspberry3 is taken as a device type and used instead of %%BALENA_MACHINE_NAME%%.

Shouldn’t Balena build actually a separate release for each type of device I have in my fleet? Or should I have a separate fleet per device type? This wouldn’t be very handy as you can’t easily copy one fleet settings (variables, configurations) to another fleet as far as I’m aware.

Your thoughts?