I am installing packages using pip while building for a Raspberry Pi 3. I have found a source of wheels that should allow the installation to run much faster however the installation does not pickup up wheels with architecture linux_armv7l
pip install --requirement requirements.txt --extra-index-url https://www.piwheels.hostedpi.com/simple
Has anyone had any experience with the above? From looking at piwheels blog it may be that I am just getting the architecture wrong and it needs to be armv7l not linux_armv7l. It would be great to get this working to avoid building massive packages.
1 Like
Hello @hpgmiskin,
Here’s a simple Dockerfile.template that you can use, as an example. You want to use --index-url instead of --extra-index-url, because you want to override the PyPi default repository. As you can see in the logs I pasted below, from my test build, there is only a single index being searched for Python wheels, instead of two. Also, you can observe that the URIs where wheels are being pulled from, are using the piwheels.org host, instead of files.pythonhosted.org.
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3
RUN apt-get update \
&& apt-get upgrade --yes \
&& pip3 install flake8 --index-url=https://piwheels.org/simple
Logs
[main] Looking in indexes: https://piwheels.org/simple
[main] Collecting flake8
[main] Downloading https://piwheels.org/simple/flake8/flake8-3.7.7-py2.py3-none-any.whl (68kB)
[main] Collecting pyflakes<2.2.0,>=2.1.0 (from flake8)
[main] Downloading https://piwheels.org/simple/pyflakes/pyflakes-2.1.0-py2.py3-none-any.whl (62kB)
[main] Collecting entrypoints<0.4.0,>=0.3.0 (from flake8)
[main] Downloading https://piwheels.org/simple/entrypoints/entrypoints-0.3-py2.py3-none-any.whl
[main] Collecting mccabe<0.7.0,>=0.6.0 (from flake8)
[main] Downloading https://piwheels.org/simple/mccabe/mccabe-0.6.1-py2.py3-none-any.whl
[main] Collecting pycodestyle<2.6.0,>=2.5.0 (from flake8)
[main] Downloading https://piwheels.org/simple/pycodestyle/pycodestyle-2.5.0-py2.py3-none-any.whl (51kB)
[main] Installing collected packages: pyflakes, entrypoints, mccabe, pycodestyle, flake8
[main] Successfully installed entrypoints-0.3 flake8-3.7.7 mccabe-0.6.1 pycodestyle-2.5.0 pyflakes-2.1.0
I hope this helps.
Cheers,
Trevor Sullivan
1 Like
Hello @pcgeek86,
Thanks for such a rapid response.
I experimented with using --index-url and --extra-index-url. I think its better to use --extra-index-url as pip will search both indexes for .whl files and always prefer them over .tar.gz. I had installations fail when I did not give pip two options for where to find packages.
In writhing this response to you I have identified the issue with my installation. I was using Python 3.7 however the .whl I was attempting to use was only compiled for python version 3.6. I would recommend trying to find a version of package that can be installed from a .whl. In order to find which versions are available its important to understand the structure of the .whl files.
Here is a section from PEP 427 in the code snippet below.
{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
For example if trying to install numpy on Raspberry Pi 3 the platform is armv7l. Once you know your platform it’s just a case of searching https://pypi.org/simple/numpy/ and https://www.piwheels.hostedpi.com/simple/numpy/ to find a matching .whl for the library.
There are no numpy .whl files compiled for armv7l on the default index however looking at the piwheels index there are a number of versions available. At time of writing you could choose numpy==1.10.4 for python3.6 or use numpy==1.16.0 for python 3.5 (from the release candidate for numpy==1.16.1 it looks like there will be a .whl available for python3.6).
Perhaps this is a counterintuitive approach and I would welcome comments if that is the case. However I found that using the wheel binary package format improved installation times dramatically and avoided having to install the toolbelt required to build packages from source.
I hope this helps others in a similar predicament,
Henry