Trouble installing Balena Python SDK on Python container image

Hey guys, I am trying to install the Balena Python SDK in my device image (for controlling some Balena functions from within the device), but having trouble.

Even if I create the smallest docker file possible:

FROM balenalib/%%BALENA_MACHINE_NAME%%-python:2.7

WORKDIR /usr/src/app

ENV INITSYSTEM on

RUN apt-get clean && apt-get update && apt-get install -yq \
	gcc \
    && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

RUN pip install balena-sdk

It won’t build. I get the error:

[main]     Step 5/6 : RUN pip install balena-sdk
[main]      ---> Running in 8ae5d051946a
[main]     DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
[main]     
[main]     Collecting balena-sdk
[main]       Downloading https://files.pythonhosted.org/packages/42/61/7b8431dfe4156789529320595c2ff98de6bd45a5118834baeec34d91f53e/balena-sdk-7.7.0.zip (52kB)
[main]     Collecting PyJWT>=1.5.0 (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/87/8b/6a9f14b5f781697e51259d81657e6048fd31a113229cf346880bb7545565/PyJWT-1.7.1-py2.py3-none-any.whl
[main]     Collecting requests>=2.19.1 (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB)
[main]     Collecting pyotp>=2.2.5 (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/8d/3b/523e8fbe65fc71294b98d84832577cdb4610b2aaee9ddbd8ee571a562895/pyotp-2.2.7-py2.py3-none-any.whl
[main]     Collecting pyOpenSSL==18.0.0 (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/96/af/9d29e6bd40823061aea2e0574ccb2fcf72bfd6130ce53d32773ec375458c/pyOpenSSL-18.0.0-py2.py3-none-any.whl (53kB)
[main]     Collecting Twisted>=18.7.0 (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/f8/2b/a80a70f71eb2b86992ffa5aaae41457791ae67faa70927fd16b76127c2b7/Twisted-19.2.0.tar.bz2 (3.1MB)
[main]     Collecting service-identity (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/e9/7c/2195b890023e098f9618d43ebc337d83c8b38d414326685339eb024db2f6/service_identity-18.1.0-py2.py3-none-any.whl
[main]     Collecting semver (from balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/21/18/a0de8cda637ba3efee1b3617ded00601507ce15bd70a39399740e0fd415f/semver-2.8.1-py2.py3-none-any.whl
[main]     Collecting certifi>=2017.4.17 (from requests>=2.19.1->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl (158kB)
[main]     Collecting idna<2.9,>=2.5 (from requests>=2.19.1->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
[main]     Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.19.1->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
[main]     Collecting urllib3<1.25,>=1.21.1 (from requests>=2.19.1->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/df/1c/59cca3abf96f991f2ec3131a4ffe72ae3d9ea1f5894abe8a9c5e3c77cfee/urllib3-1.24.2-py2.py3-none-any.whl (131kB)
[main]     Collecting six>=1.5.2 (from pyOpenSSL==18.0.0->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
[main]     Collecting cryptography>=2.2.1 (from pyOpenSSL==18.0.0->balena-sdk)
[main]       Downloading https://files.pythonhosted.org/packages/07/ca/bc827c5e55918ad223d59d299fff92f3563476c3b00d0a9157d9c0217449/cryptography-2.6.1.tar.gz (491kB)
[main]       Installing build dependencies: started
[main]       Installing build dependencies: finished with status 'error'
[main]       Complete output from command /usr/local/bin/python /usr/local/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-HbIt0b/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=18.5 wheel "cffi>=1.8,!=1.11.3; python_implementation != 'PyPy'":
[main]       DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
[main]       Collecting setuptools>=18.5
[main]         Downloading https://files.pythonhosted.org/packages/ec/51/f45cea425fd5cb0b0380f5b0f048ebc1da5b417e48d304838c02d6288a1e/setuptools-41.0.1-py2.py3-none-any.whl (575kB)
[main]       Collecting wheel
[main]         Downloading https://files.pythonhosted.org/packages/96/ba/a4702cbb6a3a485239fbe9525443446203f00771af9ac000fa3ef2788201/wheel-0.33.1-py2.py3-none-any.whl
[main]       Collecting cffi!=1.11.3,>=1.8
[main]         Downloading https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz (456kB)
[main]       Collecting pycparser (from cffi!=1.11.3,>=1.8)
[main]         Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
[main]       Building wheels for collected packages: cffi, pycparser
[main]         Building wheel for cffi (setup.py): started
[main]         Building wheel for cffi (setup.py): finished with status 'error'
[main]         Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-YUSu9l/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-gdsHPy --python-tag cp27:
[main]         /usr/bin/ld: cannot find crt1.o: No such file or directory
[main]         /usr/bin/ld: cannot find crti.o: No such file or directory
[main]         collect2: error: ld returned 1 exit status
[main]         Note: will not use '__sync_synchronize()' in the C code
[main]         ***** The above error message can be safely ignored.
[main]         running bdist_wheel
[main]         running build
[main]         running build_py
[main]         creating build
[main]         creating build/lib.linux-armv7l-2.7
[main]         creating build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/cparser.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/pkgconfig.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/commontypes.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/vengine_gen.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/recompiler.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/api.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/error.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/model.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/ffiplatform.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/lock.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/__init__.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/verifier.py -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/_cffi_include.h -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/parse_c_type.h -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/_embedding.h -> build/lib.linux-armv7l-2.7/cffi
[main]         copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-2.7/cffi
[main]         running build_ext
[main]         building '_cffi_backend' extension
[main]         creating build/temp.linux-armv7l-2.7
[main]         creating build/temp.linux-armv7l-2.7/c
[main]         gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-armv7l-2.7/c/_cffi_backend.o
[main]         In file included from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/syslimits.h:7:0,
[main]                          from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:34,
[main]                          from /usr/local/include/python2.7/Python.h:19,
[main]                          from c/_cffi_backend.c:2:
[main]         /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
[main]          #include_next <limits.h>  /* recurse down to the real one */
[main]                                                                      ^
[main]         compilation terminated.
[main]         error: command 'gcc' failed with exit status 1
[main]         ----------------------------------------
[main]         Failed building wheel for cffi
[main]         Running setup.py clean for cffi
[main]         Building wheel for pycparser (setup.py): started
[main]         Building wheel for pycparser (setup.py): finished with status 'done'
[main]         Stored in directory: /root/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
[main]       Successfully built pycparser
[main]       Failed to build cffi
[main]       Installing collected packages: setuptools, wheel, pycparser, cffi
[main]         Running setup.py install for cffi: started
[main]           Running setup.py install for cffi: finished with status 'error'
[main]           Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-YUSu9l/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-W2JFDT/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-HbIt0b/overlay --compile:
[main]           /usr/bin/ld: cannot find crt1.o: No such file or directory
[main]           /usr/bin/ld: cannot find crti.o: No such file or directory
[main]           collect2: error: ld returned 1 exit status
[main]           Note: will not use '__sync_synchronize()' in the C code
[main]           ***** The above error message can be safely ignored.
[main]           running install
[main]           running build
[main]           running build_py
[main]           creating build
[main]           creating build/lib.linux-armv7l-2.7
[main]           creating build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/cparser.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/pkgconfig.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/commontypes.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/vengine_gen.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/recompiler.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/api.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/error.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/model.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/ffiplatform.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/lock.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/__init__.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/verifier.py -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/_cffi_include.h -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/parse_c_type.h -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/_embedding.h -> build/lib.linux-armv7l-2.7/cffi
[main]           copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-2.7/cffi
[main]           running build_ext
[main]           building '_cffi_backend' extension
[main]           creating build/temp.linux-armv7l-2.7
[main]           creating build/temp.linux-armv7l-2.7/c
[main]           gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-armv7l-2.7/c/_cffi_backend.o
[main]           In file included from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/syslimits.h:7:0,
[main]                            from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:34,
[main]                            from /usr/local/include/python2.7/Python.h:19,
[main]                            from c/_cffi_backend.c:2:
[main]           /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
[main]            #include_next <limits.h>  /* recurse down to the real one */
[main]                                                                        ^
[main]           compilation terminated.
[main]           error: command 'gcc' failed with exit status 1
[main]           ----------------------------------------
[main]       Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-YUSu9l/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-W2JFDT/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-HbIt0b/overlay --compile" failed with error code 1 in /tmp/pip-install-YUSu9l/cffi/
[main]       You are using pip version 19.0.1, however version 19.0.3 is available.
[main]       You should consider upgrading via the 'pip install --upgrade pip' command.
[main]       ----------------------------------------
[main]     Command "/usr/local/bin/python /usr/local/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-HbIt0b/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=18.5 wheel "cffi>=1.8,!=1.11.3; python_implementation != 'PyPy'"" failed with error code 1 in None
[main]     
[main]     You are using pip version 19.0.1, however version 19.0.3 is available.
[main]     You should consider upgrading via the 'pip install --upgrade pip' command.
[main]     
[main]     Removing intermediate container 8ae5d051946a
[Info]     Uploading images
[main]     The command '/bin/sh -c pip install balena-sdk' returned a non-zero code: 1
[Success]  Successfully uploaded images
[Error]    Some services failed to build:
[Error]      Service: main
[Error]        Error: The command '/bin/sh -c pip install balena-sdk' returned a non-zero code: 1
[Error]    Not deploying release.

Any idea’s?

What is your target device? Raspberry Pi 3?

Yes, raspberry 2 and 3 (but mostly 3)

Thanks

You have to replace following section of your Dockerfile.template file …

RUN apt-get clean && apt-get update && apt-get install -yq \
	gcc \
    && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

… with …

RUN apt-get clean && apt-get update && apt-get install -yq \
	build-essential libffi-dev libssl-dev \
    && \
    apt-get clean && rm -rf /var/lib/apt/lists/*
  • balena-sdk has dependencies and those dependencies require additional packages to be installed (like libssl-dev is required by the cryptography, …)
  • build-essential is just a wrapper which installs dpkg-dev, g++, gcc, libc-dev (contains crt1.o, …) & make, it’s not enough to install gcc only

Very useful info! Will try it asap!

Maybe useful to put these dependencies in the readme of the Python SDK? Or is it in there and did I miss it?

Thanks for the suggestion. It looks like that we already have an issue for this.