I have a Dockerfile starting like this:
FROM resin/rpi-raspbian
RUN apt-get update && apt-get install -y git ...
This works fine on the Pi but it fails on Docker Hub with
“exec format error”
I get it, resin/rpi-raspbian requires ARM hardware but Docker Hub is x86.
docker build . NOT fail on macOS then?I have no idea why your build worked on macOS, because as far as I know it shouldn’t. In terms of creating cross-platform Dockerfiles, the best reference is this blog post by Petros: https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/
I didn’t expect it to run on macOS either. That’s why I didn’t initially believe “exec format error” in my case be an architecture issue even though all internet resources I found pointed into that direction.
I had found Petros’ post as well and I now understand (I believe) what RUN [ "cross-build-start" ] does. I’m reasonable certain it would make the build run on Docker Hub but what happens if it runs on the Pi?
I’m not sure. I think we’d need @petrosagg to chime in on that.
I still need to get back to the drawing board (i.e. read the article again) because I was expecting cross-build-start to be available in resin/rpi-raspbian but that was a wrong assumption.
Hey, do you have qemu-system-arm set up on your Mac? That would explain why it would have worked, the system would emulate ARM for those base images. For example I have that set up on my local Linux machine, and can cross compile locally these containers (that that will run on the original architecture).
Looking into the blogpost, it seems like the key is using the armv7hf-debian-qemu base image, which includes that qemu executable and the cross-compile setup.
I’d think you’d need to adapt that repo to your need, guessing you can just replace FROM resin/armv7hf-debian:jessie with the base you’d like FROM resin/rpi-raspbian or FROM resin/raspberrypi3-debian for example, and you could create thus a relevant ...-qemu version that you can use for your own project.
It’s a guess, but I’m more confident in it than not that it would work 
The usual resin base images are not setup like this, imho, because 1) they would ship with more binary that they really need and most of the time that’s not great; 2) if there’s external qemu-system-arm set up then the container can be built on other architectures as well (as the resin.io builders, and as opposed to the DockerHub builders).
I wouldn’t know - certainly didn’t do that on purpose. However, when I asked Spotlight for “qemu” it turned up a MacPorts port that some other piece of software must have installed several months ago.
Great, one mystery unraveled, thanks.
Actually, @paulmuller, after some more input from @shaunmulligan, it seems that the reason why your Docker setup can cross-build on Mac is because Docker for Mac has multi-CPU support, including ARM:
So that’s really the mystery unraveled 
Regarding cross-building in general, I’ve took the original repo linked above, and tried setting up with a different base image (raspberrypi3-debian), the result indeed just works:
The automated build is then at https://hub.docker.com/r/imrehg/raspberrypi3-debian-qemu/ That you could then use as the base of another image, that you can create your own application base image, for example here I installed python and numpy:
The automated build of that is then at https://hub.docker.com/r/imrehg/raspberrypi3-debian-qemu-numpy/ and could be used by a Dockerfile (this one deployed on resin.io onto a device) as like this, installing the nose python test suite and running it on the device:
FROM imrehg/raspberrypi3-debian-qemu-numpy
RUN pip3 install nose
CMD python3 -c 'import numpy; numpy.test("full");' && while : ; do echo "Idling..."; sleep 600; done
All these are just some examples, that the original method works, and some extra hints on how to use them.
On the other hand, if you can build your containers locally as well with docker build (without needing to do automated builds), the resulting image you can docker push to DockerHub, and can work from there similarly, without messing with all these cross-compilation steps at all!
What kind of use case you have in mind? Why would you prefer to do builds on DockerHub?
Thanks a lot, really helpful.
It’s all about the convenience that I get when done in combination with GitHub. From wherever I am I can simply push the Dockerfile to GitHub and then I’m done, or accept a PR and I’m done.
When you mean you are done, what is the outcome that you mean? Are you trying to integrate (custom) base images with DockerHub like this, or are you trying to integrate deploying to resin.io devices some way easier?
By “I’m done” I meant that I didn’t have to worry about building the image. I provide images for the general public on DockerHub based on Dockerfiles in public GitHub repositories. Some of those are for the Raspberry Pi. So, basically I’ve got two options if I don’t want to build images manually:
I see, thanks. In our case I think we are cross-compiling locally and just pushing to DockerHub (we are doing a few thousand base images pretty much every day). DockerHub automatic build has the issue that long running builds are killed (e.g. I couldn’t get an image that has both numpy and scipy built, as the process was killed by DockerHub for taking too long). That can be a limitation for this setup.
Travis sounds interesting, because there you can modify your building host, and can add emulation in a way, that you don’t need to do any of these cross-building tricks (and ship those files with all of your containers). Was looking at for example this blogpost:
It’s definitely interesting, and would love to hear what you make of it in the future, whichever path you choose!
I was looking at https://github.com/hypriot/rpi-node/blob/master/.travis.yml for inspiration (they use your images).