Trying to automate arm64 build on Docker Hub

From what I understood, it is possible to build an arm64v8 image on the Docker Hub infrastructure (that use amd64). According to this thread it can be done using Qemu.

So I added a pre_build hook:

#!/bin/bash
docker run --rm --privileged multiarch/qemu-user-static:register --reset

The Qemu binaries are also downloaded inside the container:

FROM alpine AS builder
RUN apk update
RUN apk add curl
WORKDIR /qemu
# downloaded here...
RUN curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .

FROM area51/gdal:arm64v8-2.2.3
# ...then added here
COPY --from=builder /qemu/qemu-arm-static /usr/bin

RUN apt-get update
RUN apt-get install -y libgdal-dev python3-pip libspatialindex-dev unar bc

ENV CPLUS_INCLUDE_PATH=/usr/include/gdal
ENV C_INCLUDE_PATH=/usr/include/gdal

ADD ./requirements.txt .
RUN pip3 install -r requirements.txt

RUN mkdir /code
ADD . /code/
WORKDIR /code

CMD python3.5 server.py

EXPOSE 8080

Unfortunatly, it doesn’t works:

Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.114.4' to the list of known hosts.
Switched to a new branch 'auto-build'
Executing pre_build hook...
Unable to find image 'multiarch/qemu-user-static:register' locally
register: Pulling from multiarch/qemu-user-static
bdbbaa22dec6: Pulling fs layer
42399a41a764: Pulling fs layer
ed8a5179ae11: Pulling fs layer
1ec39da9c97d: Pulling fs layer
1ec39da9c97d: Waiting
42399a41a764: Verifying Checksum
42399a41a764: Download complete
bdbbaa22dec6: Verifying Checksum
bdbbaa22dec6: Download complete
ed8a5179ae11: Verifying Checksum
ed8a5179ae11: Download complete
1ec39da9c97d: Verifying Checksum
1ec39da9c97d: Download complete
bdbbaa22dec6: Pull complete
42399a41a764: Pull complete
ed8a5179ae11: Pull complete
1ec39da9c97d: Pull complete
Digest: sha256:7502ce31890ab5da0ab6e5e5edc1e2563caa45da1c5d76aaf7dc4252aea926dc
Status: Downloaded newer image for multiarch/qemu-user-static:register
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'18.03.1-ee-3', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2018-08-30T18:42:30.000000000+00:00', u'ApiVersion': u'1.37', u'MinAPIVersion': u'1.12', u'GitCommit': u'b9a5c95', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.2'}}]
Arch: amd64
BuildTime: 2018-08-30T18:42:30.000000000+00:00
ApiVersion: 1.37
Platform: {u'Name': u''}
Version: 18.03.1-ee-3
MinAPIVersion: 1.12
GitCommit: b9a5c95
Os: linux
GoVersion: go1.10.2
Starting build of index.docker.io/cl00e9ment/open-elevation:latest...
Step 1/18 : FROM alpine AS builder
---> e7d92cdc71fe
Step 2/18 : RUN apk update
---> Running in a62df65e92ac
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
v3.11.3-6-gb1cd1b7acf [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.3-5-gb26b362c4a [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
OK: 11259 distinct packages available
Removing intermediate container a62df65e92ac
---> 9decee1216df
Step 3/18 : RUN apk add curl
---> Running in 440f41edd63d
(1/4) Installing ca-certificates (20191127-r0)
(2/4) Installing nghttp2-libs (1.40.0-r0)
(3/4) Installing libcurl (7.67.0-r0)
(4/4) Installing curl (7.67.0-r0)
Executing busybox-1.31.1-r9.trigger
Executing ca-certificates-20191127-r0.trigger
OK: 7 MiB in 18 packages
Removing intermediate container 440f41edd63d
---> 54c70441e6d3
Step 4/18 : WORKDIR /qemu
Removing intermediate container 58b03a58671b
---> 89c6e32b5854
Step 5/18 : RUN curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .
---> Running in 11696855e374
[91m % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0[0m
[91m 100 619 0 619 0 0 3327 0 --:--:-- --:--:-- --:--:-- 3917
[0m
qemu-3.0.0+resin-arm/
[91m 19 1678k 19 321k 0 0 846k 0 0:00:01 --:--:-- 0:00:01 846k[0m
[91m 100 1678k 100 1678k 0 0 2535k 0 --:--:-- --:--:-- --:--:-- 4828k
[0m
qemu-3.0.0+resin-arm/qemu-arm-static
Removing intermediate container 11696855e374
---> 80668e34eb37
Step 6/18 : FROM area51/gdal:arm64v8-2.2.3
---> 4edbfeef8f1a
Step 7/18 : COPY --from=builder /qemu/qemu-arm-static /usr/bin
---> 91c196da9280
Step 8/18 : RUN apt-get update
---> Running in 37c97a8903f3
[91mstandard_init_linux.go:190: exec user process caused "no such file or directory"
[0m
Removing intermediate container 37c97a8903f3
The command '/bin/sh -c apt-get update' returned a non-zero code: 1

The error:

standard_init_linux.go:190: exec user process caused "no such file or directory"

Looks like that one:

standard_init_linux.go:190: exec user process caused "exec format error"

That I’m starting to be used to see and means that there is an architecture problem. Does the first one mean the same thing?

If there is again an architecture problem, what I’m missing?

I would ask in the Docker Forums for help, I think the complete setup is outdated by now. The qemu 3.0.0 release you are referencing is from 2018, so outdated now. I think back at that time, Docker Hub did not have the same concepts for multi-arch builds/images available that are now in play - and a lot of this stuff were bandaids to make it work. However, I haven’t read through all of it, just heared it has been reworked a lot in the last years, hence my recommendation to ask directly at Docker Forums, maybe what you want to achieve is far easier now :slight_smile: