# Precompiled Python wheels for ARM

**URL:** <https://forums.balena.io/t/precompiled-python-wheels-for-arm/591>\
**Category:** Project help\
**Created:** [February 17, 2017, 11:57am UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591 "2017-02-17T11:57:01Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [February 17, 2017, 11:57am UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/1 "2017-02-17T11:57:01Z")

</div>

**TL;DR:** you can install some Python packages on ARMv7 platforms quicker now by using

```bash
pip install --extra-index-url=https://gergely.imreh.net/wheels/ <packagename>

```

for a number of packages, including `numpy`, `scipy`, `pillow`, `RPi.GPIO`, `simplejson`,… For the whole list and the hosted versions browse [`https://gergely.imreh.net/wheels/`](https://gergely.imreh.net/wheels/).  
🚧 This is work in progress! You’ll need `pip` 8.1 version or later! 🚧

* * *

### Inspiration

I’m using Python for a lot of my projects, and on ARM machines (like the Raspberry Pi), and when installing libraries which have components to be compiled (such as numpy), ARM is doubly at a disadvantage: it takes much longer compilation, and when there are precompiled versions available for x86 (so those machines don’t even need to compile anything just install) that’s not the case for ARM. So I thought it would be a good Friday Hack to change this and let ARM devices install native compiled packages faster. 🚨

The standard way to distribute precompiled Python libraries is using [wheels](http://wheel.readthedocs.io/en/latest/). If you check the [PyPi page for numpy](https://pypi.python.org/pypi/numpy/) for example, there are a bunch of wheels available to download for x86, and when you issue a `pip install numpy`, if a compatible wheel is found, `pip` will use that instead of working things out from scratch.

I got some inspiration from the [manylinux](https://github.com/pypa/manylinux) project, and set things up as [Docker](http://docker.io/) containers that spit out `.whl` files.

### Details

Inspired by the [manylinux](https://github.com/pypa/manylinux) project, I’ve set up a similar organization: a base Docker container with Python, and separate containers for the different libraries that install dependencies and compile them. Unfortunately manylinux’s containers are based on CentOS, and no ARM versions of those are available, otherwise it’s very cool, how they organize things. To get things done in less than a day, I went instead with our [base images](https://github.com/resin-io-library/base-images), and modified the Python images to work from `armv7` directly, install Python, add [`wheel`](https://pypi.python.org/pypi/wheel), and create tags for pythons 2.7-3.6.

Then added Dockerfiles for “generic” Python packages which don’t need anything extra besides what’s normally in `build-essential` (more precisely the `-buildpack-deps` [resin.io](http://resin.io) base images). This can be used to compile packages like [RPi.GPIO](https://pypi.python.org/pypi/RPi.GPIO), [simplejson](https://pypi.python.org/pypi/simplejson), [Twisted](https://pypi.python.org/pypi/Twisted), etc.

Then set up a couple of other packages separately, such as [numpy](https://pypi.python.org/pypi/numpy) (my main motivation for this project), [scipy](https://pypi.python.org/pypi/scipy), and [pillow](https://pypi.python.org/pypi/pillow) at this time.

The `Dockerfile` for each setup and the scripts are included in the [arm-wheels repo on Github](https://github.com/imrehg/arm-wheels/), though will have to fill out some more details.

- The `arch` directory has the `Dockerfiles` for the Python bases (so far only `armv7hf`, the [results are on Dockerhub](https://hub.docker.com/r/imrehg/armv7hf-python/))
- The `packages` directory have the Dockerfiles for specific Python packages and a `_generic`, as mentioned above. Can be found on Dockerhub as well: [generic](https://hub.docker.com/r/imrehg/armv7hf-python-generic/), [numpy](https://hub.docker.com/r/imrehg/armv7hf-python-numpy/), [scipy](https://hub.docker.com/r/imrehg/armv7hf-python-scipy/), [pillow](https://hub.docker.com/r/imrehg/armv7hf-python-pillow/).

You can use them by running the `cross.sh` script as `./cross.sh <pythonversion> <packagename>` where `<pythonversion>` is one of `2.7`, `3.3`, `3.4`, `3.5`, `3.6` (or fewer, check the available tags for on Docker hub!), and `<packagename>` is in the standard `pip` format something like `numpy` or `numpi==1.12.0`. The `./cross.sh` script within spins up the container, does the work, and then spits out the resulting binaries to the `./target` directory, so for example `numpy`'s script:

```auto
docker run \
  --rm \
  --detach \
  -v $PWD/target:/usr/src/target \
  imrehg/armv7hf-python-numpy:$1 $2

```

I’ve uploaded the resulting files to my server at the moment, so the available packages and versions are at [`https://gergely.imreh.net/wheels/`](https://gergely.imreh.net/wheels/), and you can use them in `pip` 8.1 or later for example as:

```auto
pip install --extra-index-url=https://gergely.imreh.net/wheels/ numpy

```

This one line above should save ~40 minutes of compilation time easily (`scipy` package saves a few hours).

 ![](https://global.discourse-cdn.com/balena/original/1X/6ec5b42e668ee09304d0215527f8dac0d3448838.png)

### Future

It is a very rough, and work in progress project, and I learned a big bunch! 🎓 Here are a few things that I’d like to change or improve on ⚒ :

- reorganize the base images, similar to `manylinux`: I think it would have a bunch of benefits to create a similar setup to manylinux, including
- a single base image Docker container with all available Linux versions installed, instead of using tags
- compile Python from scratch instead of using [resin.io](http://resin.io)’s precompiled distributions, so could add more platforms, such as `aarch64`
- ideally upstream and add these abilities to `manylinux`
- have an organization that can run through this compilation automatically (a CI setup), and build all required packages (while checking which ones were already built), and ideally automatically distribute them too
- have a better way to distribute these packages, such as a [`devpi`](http://doc.devpi.net/latest/) server hosted somewhere
- add more Python libraries, probably all that are listed on [Python Wheels](http://pythonwheels.com/) as having wheels, but maybe just those ones that really need compilation (non-fully-native packages). **Suggestions for what other packages to build are welcome!**

* * *

What do you think, do you find this helpful? Have any feedback? 👷

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [February 18, 2017, 10:07am UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/2 "2017-02-18T10:07:03Z")

</div>

I was working on this a bit more last night, and a few changes since:

First, fixed up some naming issues with the directories as the modules are hosted at the moment (capitalization was tricky for `pip` 😛

Then, added a few more modules:

- [coverage](https://pypi.python.org/pypi/coverage/4.3.4)
- [cryptography](https://pypi.python.org/pypi/cryptography/1.7.2)
- [greenlet](https://pypi.python.org/pypi/greenlet/0.4.12)
- [lxml](https://pypi.python.org/pypi/lxml/3.7.2)
- [psutil](https://pypi.python.org/pypi/psutil/5.1.3)
- [pymongo](https://pypi.python.org/pypi/pymongo/3.4.0)
- [PyYAML](https://pypi.python.org/pypi/PyYAML/3.12)
- [rpi\_ws281x](https://github.com/pimoroni/rpi_ws281x-python)

None of these were really tested so far, so if anyone gives them a spin, it will be appreciated ☑

Also, figured out that indeed some of the packaging is tricky for the aspect of making sure the right libraries are pulled in and compiled with. This especially important for things like `cryptography` (OpenSSL), `lxml` (libxml), and possibly `PyYAML` (libyaml, though here it should be distributed without compiling against it). Would need some checking whether they really do work across the board as distributed, or better not to do them instead of doing them badly…

Just a brain dump, as weekend is the time for thinking, usually 🙂

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [February 20, 2017, 3:26pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/3 "2017-02-20T15:26:10Z")

</div>

Trying this further, looks like there’s need for further adjustments to make these distributions actually handy (due to the included libraries). Started to use [`auditwheel`](https://github.com/pypa/auditwheel) for creating `manylinux1`-type packages, it should be a first step towards actually being cross-platform. On Debian/Raspbian it should work as it is now, on other distros it might need more work / not guaranteed (e.g. Alpine).

In practice this change means that after running `pip wheel <packagename>`, need to also run `auditwheel repair <wheelfile>` to create a combined wheel. It also needs [`patchelf`](https://github.com/NixOS/patchelf), which might or might not work on all platforms…

Let’s see if it improves things in practice 😛 Probably should have expected it but it’s turning out to be a heavier project than originally thought…

---

<div class="post-metadata">

**Author:** ![cdowns](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/cdowns/32/438_2.png) [@cdowns](https://forums.balena.io/u/cdowns)\
**Post date:** [June 1, 2017, 4:21pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/4 "2017-06-01T16:21:49Z")

</div>

This is great!

I’m looking at the /wheels/ repo you created and currently [scipy](https://gergely.imreh.net/wheels/scipy/) is empty. Is this intentional?

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [June 1, 2017, 4:35pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/5 "2017-06-01T16:35:49Z")

</div>

Hey, I kinda messed up the process by trying to make it better (happens, right?), and scipy was one of the ones I messed up the most. Will revisit it in the next couple of days and try to make them work again 🙂!

What OS are you running? (Debian/Raspbian, Alpine, other?)

---

<div class="post-metadata">

**Author:** ![cdowns](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/cdowns/32/438_2.png) [@cdowns](https://forums.balena.io/u/cdowns)\
**Post date:** [June 1, 2017, 4:47pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/6 "2017-06-01T16:47:57Z")

</div>

Great, thanks! I’m currently building into a docker image based on `resin/armv7hf-debian`.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [June 1, 2017, 5:00pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/7 "2017-06-01T17:00:05Z")

</div>

Cool! Debian is easier in general, compared to making “universal” wheels.

---

<div class="post-metadata">

**Author:** ![fenris](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/fenris/32/360_2.png) [@fenris](https://forums.balena.io/u/fenris)\
**Post date:** [July 25, 2017, 4:01pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/8 "2017-07-25T16:01:32Z")

</div>

A couple packages that I find would benefit from wheels would be [cffi](https://pypi.org/project/cffi/), [smbus-cffi](https://pypi.org/project/smbus-cffi/), and [dbus-python](https://pypi.org/project/dbus-python/).

Right now I have them broken out of my requirements.txt into a separate step so they don’t have to be rebuilt as often.

I created a pull request for dbus-python as that has dependencies, but the other two can be built with `_generic`.

I also found [Adafruit\_BBIO](https://pypi.org/project/Adafruit_BBIO/) that could be built from `_generic` that didn’t already have a wheel for any architecture.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [November 14, 2017, 9:53pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/9 "2017-11-14T21:53:52Z")

</div>

I guess this news from the Raspberry Pi folks is pretty relevant (especially as haven’t had a chance to take this approach above much further yet)

> **[piwheels: making "pip install" fast - Raspberry Pi](https://www.raspberrypi.org/blog/piwheels/)**
>
> Ben Nuttall has developed piwheels to help you install Python packages on your Raspberry Pi within seconds instead of minutes or hours.

What do you think?

---

<div class="post-metadata">

**Author:** ![fenris](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/fenris/32/360_2.png) [@fenris](https://forums.balena.io/u/fenris)\
**Post date:** [November 27, 2017, 6:26pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/10 "2017-11-27T18:26:12Z")

</div>

I haven’t tried them yet. I’m wondering how tightly they will are built to Rasberian.

Will they be enough to run in an Alpine container on a BeagleboneBlack?

Something to play with when I’m not in the process of moving.

---

<div class="post-metadata">

**Author:** ![pajogdand](https://avatars.discourse-cdn.com/v4/letter/p/ecd19e/32.png) [@pajogdand](https://forums.balena.io/u/pajogdand)\
**Post date:** [May 28, 2019, 12:17pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/11 "2019-05-28T12:17:02Z")

</div>

Hi,

I want to install the latest version of Scipy(e.g: 1.2.1) on my ubuntu Arm7 board.

when I used this command its installs 0.38.0 versio of scipy  
pip install --extra-index-url=https://gergely.imreh.net/wheels/ scipy

How to install 1.2.1 ?

---

<div class="post-metadata">

**Author:** ![lucianbuzzo](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/lucianbuzzo/32/615_2.png) [@lucianbuzzo](https://forums.balena.io/u/lucianbuzzo)\
**Post date:** [May 28, 2019, 12:22pm UTC](https://forums.balena.io/t/precompiled-python-wheels-for-arm/591/12 "2019-05-28T12:22:11Z")

</div>

@pajogdand welcome to the forums! I suggest that you open an issue (or even better a pull request!) on the wheels repository here [https://github.com/imrehg/arm-wheels/issues](https://github.com/imrehg/arm-wheels/issues)