Unfortunately after loading up a fresh NUC image and loading that Dockerfile into the image, the container just restarts over and over. Produces the familiar error message: ‘standard_init_linux.go:207: exec user process caused “exec format error”’.
Any ideas?
Host OS Version: BalenaOS 2.39.0+rev3
Supervisor Version: 10.0.3
Dockerfile:
FROM balenalib/armv7hf-debian
RUN [ "cross-build-start" ]
RUN apt-get update
RUN apt-get install python-pip
RUN pip install virtualenv
RUN [ "cross-build-end" ]
Forget my comment above about binfmt_misc, it shouldn’t be necessary.
The idea of running ARM code on x86 is to be able to build an arm image on x86.
If you have a nuc, you should use an x86 base image.
You might be able to run arm code on x86 with the binfmt_misc trick above.
You’d need to run it in your Dockerfile.
You still need an x86 base image.
Something like:
FROM balenalib/intel-nuc-debian
COPY qemu-arm-static /usr/bin/qemu-arm-static
RUN mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
RUN echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:' > /proc/sys/fs/binfmt_misc/register
RUN /some/arm/executable/file
The container probably needs to be privileged.
I’m not sure this will work as I don’t know if /proc/sys/fs/binfmt_misc is accessible from containers.
I did end up playing with it a bit more and ended up simplifying some. I found out the following packages take care of all that work! So this dockerfile allows for transparently running ARM binaries on x86:
FROM balenalib/amd64-debian
# Installs packages needed to transparently run ARM binaries on x64
RUN install_packages qemu-user-static binfmt-support crossbuild-essential-armhf
# Keep container alive so we can SSH in to experiment
CMD sleep infinity
I’m using the balenalib/raspberrypi3:latest image from dockerhub as a base image.
Building works fine without installing qemu-user-static and binfmt-support on my x86_64 host because of the marvelous RUN [ "cross-build-start" ] prologue and RUN [ "cross-build-end" ] epilogue, which is brilliantly explained in this balena.io blog article:
… however it seems that this mechanism can’t be leveraged when running arm binaries via docker runas well (not just docker build). I have to resort to installing qemu-user-static and binfmt-support again on the x86_64 host.
Is there a simple modification to the command I give to docker run that will get it to execute in the same way as arm binaries do during docker build?
You can install binfmt on your host computer, so that all arm binaries get ran with qemu. Depending on your host OS the process could be quite different.
What is the reason that you want to run arm containers on your host? Often you have just change the base image of the docker build and most things will work out of the box.
I’m running balena images on a CI server to create sustainable build environments that compile executables for a range of different systems and architectures, irrespective of the host doing the build.
The CI server builds the docker images and then instantiates each of them using docker run --rm -t -v <image_name> make all to compile architecture/platform-specific executables, which are written out to a mounted volume.
Unfortunately, I don’t have permission to install qemu-user-static and binfmt-support on the CI server itself. This isn’t a problem for docker build which you’ve provided a nice solution for, but do you have any advice for using ``docker run``` on the same CI server?
For example, is the solution to prefix each build command with /usr/bin/qemu-user-static e.g.:
docker run --rm -it <image_name> /usr/bin/qemu-user-static make all
Or perhaps I should omit the RUN [ "cross-build-end" ] statement in the Dockerfile to leave the /bin/sh shim in place?
Ideally, I’d like to strive for the same clean syntax you’ve made possible for docker build.
