TL;DR: you can install some Python packages on ARMv7 platforms quicker now by using
pip install --extra-index-url=https://gergely.imreh.net/wheels/ <packagename>
for a number of packages, including numpy, scipy, pillow, RPi.GPIO, simplejson,… For the whole list and the hosted versions browse https://gergely.imreh.net/wheels/.
This is work in progress! You’ll need pip 8.1 version or later!
Inspiration
I’m using Python for a lot of my projects, and on ARM machines (like the Raspberry Pi), and when installing libraries which have components to be compiled (such as numpy), ARM is doubly at a disadvantage: it takes much longer compilation, and when there are precompiled versions available for x86 (so those machines don’t even need to compile anything just install) that’s not the case for ARM. So I thought it would be a good Friday Hack to change this and let ARM devices install native compiled packages faster. 
The standard way to distribute precompiled Python libraries is using wheels. If you check the PyPi page for numpy for example, there are a bunch of wheels available to download for x86, and when you issue a pip install numpy, if a compatible wheel is found, pip will use that instead of working things out from scratch.
I got some inspiration from the manylinux project, and set things up as Docker containers that spit out .whl files.
Details
Inspired by the manylinux project, I’ve set up a similar organization: a base Docker container with Python, and separate containers for the different libraries that install dependencies and compile them. Unfortunately manylinux’s containers are based on CentOS, and no ARM versions of those are available, otherwise it’s very cool, how they organize things. To get things done in less than a day, I went instead with our base images, and modified the Python images to work from armv7 directly, install Python, add wheel, and create tags for pythons 2.7-3.6.
Then added Dockerfiles for “generic” Python packages which don’t need anything extra besides what’s normally in build-essential (more precisely the -buildpack-deps resin.io base images). This can be used to compile packages like RPi.GPIO, simplejson, Twisted, etc.
Then set up a couple of other packages separately, such as numpy (my main motivation for this project), scipy, and pillow at this time.
The Dockerfile for each setup and the scripts are included in the arm-wheels repo on Github, though will have to fill out some more details.
- The
archdirectory has theDockerfilesfor the Python bases (so far onlyarmv7hf, the results are on Dockerhub) - The
packagesdirectory have the Dockerfiles for specific Python packages and a_generic, as mentioned above. Can be found on Dockerhub as well: generic, numpy, scipy, pillow.
You can use them by running the cross.sh script as ./cross.sh <pythonversion> <packagename> where <pythonversion> is one of 2.7, 3.3, 3.4, 3.5, 3.6 (or fewer, check the available tags for on Docker hub!), and <packagename> is in the standard pip format something like numpy or numpi==1.12.0. The ./cross.sh script within spins up the container, does the work, and then spits out the resulting binaries to the ./target directory, so for example numpy's script:
docker run \
--rm \
--detach \
-v $PWD/target:/usr/src/target \
imrehg/armv7hf-python-numpy:$1 $2
I’ve uploaded the resulting files to my server at the moment, so the available packages and versions are at https://gergely.imreh.net/wheels/, and you can use them in pip 8.1 or later for example as:
pip install --extra-index-url=https://gergely.imreh.net/wheels/ numpy
This one line above should save ~40 minutes of compilation time easily (scipy package saves a few hours).
Future
It is a very rough, and work in progress project, and I learned a big bunch! 
Here are a few things that I’d like to change or improve on 
:
- reorganize the base images, similar to
manylinux: I think it would have a bunch of benefits to create a similar setup to manylinux, including - a single base image Docker container with all available Linux versions installed, instead of using tags
- compile Python from scratch instead of using resin.io’s precompiled distributions, so could add more platforms, such as
aarch64 - ideally upstream and add these abilities to
manylinux - have an organization that can run through this compilation automatically (a CI setup), and build all required packages (while checking which ones were already built), and ideally automatically distribute them too
- have a better way to distribute these packages, such as a
devpiserver hosted somewhere - add more Python libraries, probably all that are listed on Python Wheels as having wheels, but maybe just those ones that really need compilation (non-fully-native packages). Suggestions for what other packages to build are welcome!
What do you think, do you find this helpful? Have any feedback? 
