build and test armv7 on x86

Product support balenaOS
,
1

Hi,
I am wanting to dockerise this project https://www.mysterium.network/ for the raspberrypi 3, which has guides for both docker and raspberrypi3

I created this Dockerfile:

FROM balenalib/armv7hf-debian
RUN [ "cross-build-start" ]
RUN apt-get update \
    && apt-get -y install debconf-utils \
    && echo resolvconf resolvconf/linkify-resolvconf boolean false | debconf-set-selections \
    && apt-get -y install resolvconf
COPY ./install.sh .
RUN ./install.sh
RUN [ "cross-build-end" ]

The content of the install.sh is a lightly modified version of the official Mysterium install script, found here https://raw.githubusercontent.com/mysteriumnetwork/node/master/install.sh

After building the image I get the following output:

> docker build --platform=linux/arm/v7 --tag mysterium .
Sending build context to Docker daemon  13.31kB
Step 1/6 : FROM balenalib/armv7hf-debian
 ---> 843ea840213c
Step 2/6 : RUN [ "cross-build-start" ]
 ---> Using cache
 ---> b5001b2c60b9
Step 3/6 : RUN apt-get update     && apt-get -y install debconf-utils     && echo resolvconf resolvconf/linkify-resolvconf boolean false | debconf-set-selections     && apt-get -y install resolvconf
 ---> Using cache
 ---> e9624c78ca01
Step 4/6 : COPY ./install.sh .
 ---> Using cache
 ---> e91c42cd1178
Step 5/6 : RUN ./install.sh
 ---> Using cache
 ---> fde60165648c
Step 6/6 : RUN [ "cross-build-end" ]
 ---> Using cache
 ---> 412fd2f4ed0c
Successfully built 412fd2f4ed0c
Successfully tagged mysterium:latest

At first I immediately uploaded the new image to BalenaCloud to test, but burning the image to an SD card and running produced a broken container, by which the ‘mysterium’ service would start and immediately stop.

I have sinced installed some qemu parts to test the build on my x86 computer, and get the following output:

> docker run -t mysterium:latest uname -a
WARNING: The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
Linux 073f461317c8 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) armv7l GNU/Linux

So the emulation part must work, so then I tried to debug why the container and service does not run, omitting the uname -a part doesn’t shed any light on the problem, just runs and returns to prompt, but no running docker container.

Next I have tried running docker events but that just proves what I already (obviously) knew:

 2021-11-29T16:20:21.297793544Z image tag sha256:412fd2f4ed0c425e96e8a39485da925c31188ad7577b165465a49e6065c214d7 (io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=mysterium:latest)
2021-11-29T16:23:04.630017077Z container create 073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562 (image=mysterium:latest, io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=sharp_goldwasser)
2021-11-29T16:23:04.631568141Z container attach 073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562 (image=mysterium:latest, io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=sharp_goldwasser)
2021-11-29T16:23:04.658808228Z network connect d45221cb1227b1f438c3ce4fa73041f747b86367daf9a9085f9a76792929d9e9 (container=073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562, name=bridge, type=bridge)
2021-11-29T16:23:04.976894871Z container start 073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562 (image=mysterium:latest, io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=sharp_goldwasser)
2021-11-29T16:23:04.978761132Z container resize 073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562 (height=25, image=mysterium:latest, io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=sharp_goldwasser, width=236)
2021-11-29T16:23:05.382823643Z container die 073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562 (exitCode=0, image=mysterium:latest, io.balena.architecture=armv7hf, io.balena.qemu.version=6.0.0+balena1-arm, name=sharp_goldwasser)
2021-11-29T16:23:05.477351484Z network disconnect d45221cb1227b1f438c3ce4fa73041f747b86367daf9a9085f9a76792929d9e9 (container=073f461317c8f1f1bfe124e310858a51ba4c890a5e2d9dfd302180e042563562, name=bridge, type=bridge)

Any ideas? Thanks in advance if you read this far! :slight_smile:

1 Like
2

Responding to my own ticket, but the amd64 part of this uname output is wrong for a start, no?

1 Like
3

I changed things up a little bit, and think I made some progress, if only because the base image size became a lot larger! :laughing:

I built the image again using the Dockerfile below.

FROM balenalib/raspberrypi3:bullseye
RUN [ "cross-build-start" ]
RUN install_packages debconf-utils \
    && echo resolvconf resolvconf/linkify-resolvconf boolean false | debconf-set-selections \
    && install_packages resolvconf
COPY ./install.sh .
RUN ./install.sh
RUN [ "cross-build-end" ]

I then pushed the image to Docker Hub and added to my docker-compose.yaml:

version: '2'
services:
  mysterium:
    image: n1md4/raspterium:latest
    container_name: mysterium
    cap_add:
      - NET_ADMIN
    restart: unless-stopped
    ports:
      - "4449:4449"
    volumes:
      - mysterium-data:/var/lib/mysterium-node
volumes:
  mysterium-data:

This is pushed to balena, before finally moving a device to run the application. I don’t get any output though, just a constant loop of the service restarting and exiting:

Restarting service 'mysterium sha256:d20f51d130f700e42a0576351aedc255716be698531d9c58a63402bccebc20ea'
Service exited 'mysterium sha256:d20f51d130f700e42a0576351aedc255716be698531d9c58a63402bccebc20ea'

Still after ideas as to how to debug this.

4

Hello @n1md4 thanks for introducing the mysterium network project into balena. It’s great!

Sadly the arm architecture is not compatible with x86. So my recommendation would be to build it using base images based on x86 instead of armv7hf. Probably you are getting issues from libraries prepared for armv7 and not x86.

If you still need to buld armv7 containers on x86, you can try with QEMU. I’m not a QEMU expert but QEMU emulates other CPU architectures on a device. Read more here or here Get started with QEMU X86 64bit and Go - Balena Documentation

Let me know if that works for you :slight_smile:

6

Hello! did you find a solution with your problem?

8

Hello. Thank you for your response and the follow-up. Sorry for the delay in reply.

I did make progress, I had a booting image and it would start a service, but I hit a couple of different problems, a potential issue with storage volumes and a definite issue with the service starting; related but not identical to the original issue I posted.

I may post again for assistance, but only when I exhaust my own further testing.

9

Thank you for the update @n1md4

Let us know if we can help you more :slight_smile: