ImportError: No module named 'pkg_resources.py31compat' during "pip3 install awsiotsdk" and "pip3 install AWSIoTPythonSDK"

Product support balenaOS
1

I’m using a BalenaFin, running kernel 4.19.71.
Linux 95a5b00 4.19.71 #1 SMP Wed Jan 15 20:59:27 UTC 2020 armv7l GNU/Linux

I’m starting with the python3 container:
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run

Running “pip3 install awsiotsdk” (v2 of API) or “pip3 install AWSIoTPythonSDK” (v1 of API) results in a lengthy error message which centers around “ImportError: No module named ‘pkg_resources.py31compat’”.

This breaks when pushing the container, and also breaks when entered by hand in the terminal. Strangely, running the command a 2nd time works correctly. However, this can not be used in an automated deployment.

I have found a few posts out there for similar error messages, but they haven’t helped.
e.g. https://stackoverflow.com/questions/46998858/importerror-no-module-named-py31compat

Has anyone got any ideas how to install the sdk without creating this error? Thank you!

Here are transcripts of running the commands in the CLI:

API v1:

root@95a5b00:/usr/src/app/src/acoustic# pip3 install AWSIoTPythonSDK
Collecting AWSIoTPythonSDK
  Downloading AWSIoTPythonSDK-1.4.8.tar.gz (80 kB)
     |████████████████████████████████| 80 kB 1.0 MB/s 
Using legacy setup.py install for AWSIoTPythonSDK, since package 'wheel' is not installed.
Installing collected packages: AWSIoTPythonSDK
    Running setup.py install for AWSIoTPythonSDK ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python3.5 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-v7n09v8i/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.5m/AWSIoTPythonSDK
         cwd: /tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/
    Complete output (165 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/AWSIoTPythonSDK
    copying AWSIoTPythonSDK/__init__.py -> build/lib/AWSIoTPythonSDK
    copying AWSIoTPythonSDK/MQTTLib.py -> build/lib/AWSIoTPythonSDK
    creating build/lib/AWSIoTPythonSDK/core
    copying AWSIoTPythonSDK/core/__init__.py -> build/lib/AWSIoTPythonSDK/core
    creating build/lib/AWSIoTPythonSDK/core/util
    copying AWSIoTPythonSDK/core/util/__init__.py -> build/lib/AWSIoTPythonSDK/core/util
    copying AWSIoTPythonSDK/core/util/enums.py -> build/lib/AWSIoTPythonSDK/core/util
    copying AWSIoTPythonSDK/core/util/providers.py -> build/lib/AWSIoTPythonSDK/core/util
    creating build/lib/AWSIoTPythonSDK/core/shadow
    copying AWSIoTPythonSDK/core/shadow/deviceShadow.py -> build/lib/AWSIoTPythonSDK/core/shadow
    copying AWSIoTPythonSDK/core/shadow/__init__.py -> build/lib/AWSIoTPythonSDK/core/shadow
    copying AWSIoTPythonSDK/core/shadow/shadowManager.py -> build/lib/AWSIoTPythonSDK/core/shadow
    creating build/lib/AWSIoTPythonSDK/core/protocol
    copying AWSIoTPythonSDK/core/protocol/mqtt_core.py -> build/lib/AWSIoTPythonSDK/core/protocol
    copying AWSIoTPythonSDK/core/protocol/__init__.py -> build/lib/AWSIoTPythonSDK/core/protocol
    creating build/lib/AWSIoTPythonSDK/core/jobs
    copying AWSIoTPythonSDK/core/jobs/__init__.py -> build/lib/AWSIoTPythonSDK/core/jobs
    copying AWSIoTPythonSDK/core/jobs/thingJobManager.py -> build/lib/AWSIoTPythonSDK/core/jobs
    creating build/lib/AWSIoTPythonSDK/core/protocol/paho
    copying AWSIoTPythonSDK/core/protocol/paho/__init__.py -> build/lib/AWSIoTPythonSDK/core/protocol/paho
    copying AWSIoTPythonSDK/core/protocol/paho/client.py -> build/lib/AWSIoTPythonSDK/core/protocol/paho
    creating build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/queues.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/workers.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/defaults.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/clients.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/__init__.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/events.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    copying AWSIoTPythonSDK/core/protocol/internal/requests.py -> build/lib/AWSIoTPythonSDK/core/protocol/internal
    creating build/lib/AWSIoTPythonSDK/core/protocol/connection
    copying AWSIoTPythonSDK/core/protocol/connection/__init__.py -> build/lib/AWSIoTPythonSDK/core/protocol/connection
    copying AWSIoTPythonSDK/core/protocol/connection/cores.py -> build/lib/AWSIoTPythonSDK/core/protocol/connection
    copying AWSIoTPythonSDK/core/protocol/connection/alpn.py -> build/lib/AWSIoTPythonSDK/core/protocol/connection
    creating build/lib/AWSIoTPythonSDK/core/greengrass
    copying AWSIoTPythonSDK/core/greengrass/__init__.py -> build/lib/AWSIoTPythonSDK/core/greengrass
    creating build/lib/AWSIoTPythonSDK/core/greengrass/discovery
    copying AWSIoTPythonSDK/core/greengrass/discovery/__init__.py -> build/lib/AWSIoTPythonSDK/core/greengrass/discovery
    copying AWSIoTPythonSDK/core/greengrass/discovery/providers.py -> build/lib/AWSIoTPythonSDK/core/greengrass/discovery
    copying AWSIoTPythonSDK/core/greengrass/discovery/models.py -> build/lib/AWSIoTPythonSDK/core/greengrass/discovery
    creating build/lib/AWSIoTPythonSDK/exception
    copying AWSIoTPythonSDK/exception/operationTimeoutException.py -> build/lib/AWSIoTPythonSDK/exception
    copying AWSIoTPythonSDK/exception/operationError.py -> build/lib/AWSIoTPythonSDK/exception
    copying AWSIoTPythonSDK/exception/__init__.py -> build/lib/AWSIoTPythonSDK/exception
    copying AWSIoTPythonSDK/exception/AWSIoTExceptions.py -> build/lib/AWSIoTPythonSDK/exception
    running install_lib
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception
    copying build/lib/AWSIoTPythonSDK/exception/operationTimeoutException.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception
    copying build/lib/AWSIoTPythonSDK/exception/operationError.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception
    copying build/lib/AWSIoTPythonSDK/exception/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception
    copying build/lib/AWSIoTPythonSDK/exception/AWSIoTExceptions.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util
    copying build/lib/AWSIoTPythonSDK/core/util/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util
    copying build/lib/AWSIoTPythonSDK/core/util/enums.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util
    copying build/lib/AWSIoTPythonSDK/core/util/providers.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow
    copying build/lib/AWSIoTPythonSDK/core/shadow/deviceShadow.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow
    copying build/lib/AWSIoTPythonSDK/core/shadow/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow
    copying build/lib/AWSIoTPythonSDK/core/shadow/shadowManager.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection
    copying build/lib/AWSIoTPythonSDK/core/protocol/connection/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection
    copying build/lib/AWSIoTPythonSDK/core/protocol/connection/cores.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection
    copying build/lib/AWSIoTPythonSDK/core/protocol/connection/alpn.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection
    copying build/lib/AWSIoTPythonSDK/core/protocol/mqtt_core.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol
    copying build/lib/AWSIoTPythonSDK/core/protocol/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/queues.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/workers.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/defaults.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/clients.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/events.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    copying build/lib/AWSIoTPythonSDK/core/protocol/internal/requests.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/paho
    copying build/lib/AWSIoTPythonSDK/core/protocol/paho/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/paho
    copying build/lib/AWSIoTPythonSDK/core/protocol/paho/client.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/paho
    copying build/lib/AWSIoTPythonSDK/core/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass
    copying build/lib/AWSIoTPythonSDK/core/greengrass/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery
    copying build/lib/AWSIoTPythonSDK/core/greengrass/discovery/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery
    copying build/lib/AWSIoTPythonSDK/core/greengrass/discovery/providers.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery
    copying build/lib/AWSIoTPythonSDK/core/greengrass/discovery/models.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery
    creating /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/jobs
    copying build/lib/AWSIoTPythonSDK/core/jobs/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/jobs
    copying build/lib/AWSIoTPythonSDK/core/jobs/thingJobManager.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/jobs
    copying build/lib/AWSIoTPythonSDK/__init__.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK
    copying build/lib/AWSIoTPythonSDK/MQTTLib.py -> /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception/operationTimeoutException.py to operationTimeoutException.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception/operationError.py to operationError.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/exception/AWSIoTExceptions.py to AWSIoTExceptions.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util/enums.py to enums.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/util/providers.py to providers.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow/deviceShadow.py to deviceShadow.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/shadow/shadowManager.py to shadowManager.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection/cores.py to cores.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/connection/alpn.py to alpn.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/mqtt_core.py to mqtt_core.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/queues.py to queues.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/workers.py to workers.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/defaults.py to defaults.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/clients.py to clients.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/events.py to events.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/internal/requests.py to requests.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/paho/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/protocol/paho/client.py to client.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery/providers.py to providers.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/greengrass/discovery/models.py to models.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/jobs/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/core/jobs/thingJobManager.py to thingJobManager.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/__init__.py to __init__.cpython-35.pyc
    byte-compiling /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK/MQTTLib.py to MQTTLib.cpython-35.pyc
    running install_egg_info
    running egg_info
    creating AWSIoTPythonSDK.egg-info
    writing dependency_links to AWSIoTPythonSDK.egg-info/dependency_links.txt
    writing AWSIoTPythonSDK.egg-info/PKG-INFO
    writing top-level names to AWSIoTPythonSDK.egg-info/top_level.txt
    writing manifest file 'AWSIoTPythonSDK.egg-info/SOURCES.txt'
    reading manifest file 'AWSIoTPythonSDK.egg-info/SOURCES.txt'
    writing manifest file 'AWSIoTPythonSDK.egg-info/SOURCES.txt'
    Copying AWSIoTPythonSDK.egg-info to /usr/local/lib/python3.5/site-packages/AWSIoTPythonSDK-1.4.8-py3.5.egg-info
    running install_scripts
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/setup.py", line 32, in <module>
        "Programming Language :: Python :: 3.5"
      File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.5/distutils/command/install.py", line 551, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/install_scripts.py", line 17, in run
        import setuptools.command.easy_install as ei
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 52, in <module>
        from setuptools.sandbox import run_setup
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 15, in <module>
        import pkg_resources.py31compat
    ImportError: No module named 'pkg_resources.py31compat'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python3.5 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w5y6qx9f/AWSIoTPythonSDK/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-v7n09v8i/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.5m/AWSIoTPythonSDK Check the logs for full command output.

And Here is with V2:
(note, i have snipped a large amount of the compiling in order to fit this post here!)

root@95a5b00:/usr/src/app/src/acoustic# pip3 install awsiotsdk
Collecting awsiotsdk
  Using cached awsiotsdk-1.1.0-py2.py3-none-any.whl (18 kB)
Collecting awscrt==0.5.15
  Using cached awscrt-0.5.15.tar.gz (2.0 MB)
Using legacy setup.py install for awscrt, since package 'wheel' is not installed.
Installing collected packages: awscrt, awsiotsdk
    Running setup.py install for awscrt ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python3.5 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u13pr5k5/awscrt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u13pr5k5/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-acn51f1t/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.5m/awscrt
         cwd: /tmp/pip-install-u13pr5k5/awscrt/
    Complete output (898 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-armv7l-3.5
    creating build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/http.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/mqtt.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/auth.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/__init__.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/crypto.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/awsiot_mqtt_connection_builder.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/io.py -> build/lib.linux-armv7l-3.5/awscrt
    copying awscrt/exceptions.py -> build/lib.linux-armv7l-3.5/awscrt
    running build_ext
    cmake version 3.7.2

(snip to fit post on forum)

gcc -pthread -shared build/temp.linux-armv7l-3.5/source/http_stream.o build/temp.linux-armv7l-3.5/source/mqtt_client_connection.o build/temp.linux-armv7l-3.5/source/io.o build/temp.linux-armv7l-3.5/source/http_proxy.o build/temp.linux-armv7l-3.5/source/http_message.o build/temp.linux-armv7l-3.5/source/auth_signing_config.o build/temp.linux-armv7l-3.5/source/auth_signer.o build/temp.linux-armv7l-3.5/source/module.o build/temp.linux-armv7l-3.5/source/http_connection.o build/temp.linux-armv7l-3.5/source/auth_credentials.o build/temp.linux-armv7l-3.5/source/crypto.o build/temp.linux-armv7l-3.5/source/mqtt_client.o build/temp.linux-armv7l-3.5/source/http_headers.o -L/usr/local/lib -L/tmp/pip-install-u13pr5k5/awscrt/build/deps/install/lib -laws-c-mqtt -laws-c-auth -laws-c-http -laws-c-compression -laws-c-cal -laws-c-io -laws-c-common -ls2n -l:libcrypto.a -lrt -lpython3.5m -o build/lib.linux-armv7l-3.5/_awscrt.cpython-35m-arm-linux-gnueabihf.so
running install_lib
creating /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/http.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/mqtt.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/auth.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/__init__.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/crypto.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/awsiot_mqtt_connection_builder.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/io.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/awscrt/exceptions.py -> /usr/local/lib/python3.5/site-packages/awscrt
copying build/lib.linux-armv7l-3.5/_awscrt.cpython-35m-arm-linux-gnueabihf.so -> /usr/local/lib/python3.5/site-packages
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/http.py to http.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/mqtt.py to mqtt.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/auth.py to auth.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/__init__.py to __init__.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/crypto.py to crypto.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/awsiot_mqtt_connection_builder.py to awsiot_mqtt_connection_builder.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/io.py to io.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/site-packages/awscrt/exceptions.py to exceptions.cpython-35.pyc
running install_egg_info
running egg_info
writing awscrt.egg-info/PKG-INFO
writing requirements to awscrt.egg-info/requires.txt
writing dependency_links to awscrt.egg-info/dependency_links.txt
writing top-level names to awscrt.egg-info/top_level.txt
reading manifest file 'awscrt.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.git*/**' found anywhere in distribution
warning: no previously-included files matching '.travis*' found anywhere in distribution
warning: no previously-included files matching '.travis/**' found anywhere in distribution
warning: no previously-included files matching 'codebuild/**' found anywhere in distribution
warning: no previously-included files matching '.cbmc-batch/**' found anywhere in distribution
warning: no previously-included files matching 'aws-common-runtime/*/docs/**' found anywhere in distribution
warning: no previously-included files matching 'docker-images/**' found anywhere in distribution
no previously-included directories found matching 'aws-common-runtime/aws-c-auth/tests/aws-sig-v4-test-suite'
no previously-included directories found matching 'aws-common-runtime/aws-c-auth/tests/fuzz/corpus'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/fuzz/corpus'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/ctverif'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/pems'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/saw'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/sidetrail'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/integration/trust-store'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/integration/data'
no previously-included directories found matching 'aws-common-runtime/s2n/tests/unit/kats'
writing manifest file 'awscrt.egg-info/SOURCES.txt'
Copying awscrt.egg-info to /usr/local/lib/python3.5/site-packages/awscrt-0.5.15-py3.5.egg-info
running install_scripts
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-u13pr5k5/awscrt/setup.py", line 267, in <module>
    'boto3'
  File "/usr/local/lib/python3.5/site-packages/setuptools/__init__.py", line 144, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.5/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/usr/local/lib/python3.5/distutils/command/install.py", line 551, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.5/site-packages/setuptools/command/install_scripts.py", line 17, in run
    import setuptools.command.easy_install as ei
  File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 52, in <module>
    from setuptools.sandbox import run_setup
  File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 15, in <module>
    import pkg_resources.py31compat
ImportError: No module named 'pkg_resources.py31compat'
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python3.5 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u13pr5k5/awscrt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u13pr5k5/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-acn51f1t/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.5m/awscrt Check the logs for full command output.
root@95a5b00:/usr/src/app/src/acoustic#
3

Hey there,

I tested this on my own fin and was able to install AWSIoTPythonSDK without any problems. The reference to the image balenalib/fincm3-python:3-stretch-run changes in time so it’s possible that your python setup is different from mine.

The error ImportError: No module named 'pkg_resources.py31compat' suggests to me that there might be a problem with your version of setuptools. You should be able to get the version via: easy_install --version. The container I ran on my fin has setuptools 46.1.3 (above command prints out setuptools 46.1.3 from /usr/local/lib/python3.5/site-packages (Python 3.5)). Can you confirm that your container is running the same version?

In order to install awsiotsdk, I had to finally install a few other dependencies:

apt-get update
apt-get install libssl-dev build-essential cmake

Try running this before pip3 install awsiotsdk. It might be that updating setuptools would fix this too.

Lucy-Jane

6

Hi Lucy-Jane,

Thank you for getting back to me! I tried your suggestions.

Intriguingly, I get conflicting messages about setuptools. e.g.

root@95a5b00:/usr/src/app/src/acoustic# easy_install --version
Traceback (most recent call last):
  File "/usr/local/bin/easy_install", line 7, in <module>
    from setuptools.command.easy_install import main
  File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 52, in <module>
    from setuptools.sandbox import run_setup
  File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 15, in <module>
    import pkg_resources.py31compat
ImportError: No module named 'pkg_resources.py31compat'

Yet trying to install setuptools, it reports it’s already installed???

root@95a5b00:/usr/src/app/src/acoustic# apt install setuptools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package setuptools
root@95a5b00:/usr/src/app/src/acoustic_sensors_acquire_sw# pip3 install setuptools
Requirement already satisfied: setuptools in /usr/local/lib/python3.5/site-packages (46.1.3)

I also did the line:

apt-get update
apt-get install libssl-dev build-essential cmake

which of course ran ok on their own, but still I have the same error for pip3 install AWSIoTPythonSDK.

I also tried a suggestion elsewhere which was to uninstall then re-install. i.e.:
pip3 uninstall setuptools
pip3 install setuptools
but that gives the same results…

Also, I enabled support access for 24 hours if you’d like to have a look at it.

7

Hi there, can you provide a minimal Dockerfile that we can use to reproduce this issue?

10

Hiya! Thank you for the good idea.

I started new BalenaFin CM3 with an “empty” DOckerfile. THis is what it had in it:

# base-image for python on any machine using a template variable,
# see more about dockerfile templates here: https://www.balena.io/docs/learn/develop/dockerfi\
le/
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run

# Set our working directory
WORKDIR /usr/src/app

# copy all local files to container
COPY . ./

RUN apt-get update

# development
#RUN apt install git
#RUN apt-get -y install gcc
#RUN apt install -y vim
#RUN apt install -y dos2unix
#RUN apt install joe
#RUN pip3 install --upgrade pip

I found that pip3 install AWSIoTPythonSDK installed fine!

Then i uncommented those RUN statements and pushed again.
That time I got the familiar error:
ModuleNotFoundError: No module named pkg_resources.py31compat'

So that must mean it’s one of those statements… I guess the most suspicious is “pip3 install --upgrade pip”?? I’ll try it again with that statement commented out…

11

OK, good news! With “pip3 install --upgrade pip” removed, I can now perform “pip3 install AWSIoTPythonSDK” That’s great news! Thank you!

Now, instead I upgrade it using the command:
RUN /usr/local/bin/python3.8 -m pip install --upgrade pip

Also, when I used @Lucy-Jane’s install commands above, I was able to build awsiotsdk (version 2.0 of the SDK) also. So thank you again for that.

:rocket:

(now the only thing left to do is uncomment the rest of Dockerfile)

12

I’m glad you solved the problem. Please let us know if any other problems occur.

14

ok! after many more iterations of enabling different lines of my Dockerfile, I found another “collision.” I was able to include about 20 different packages successfully. However, for whatever reason, this line:

RUN apt install awscli

makes it so that AWSIoTPythonSDK can not install.

?!

I’m going to try another minimal container with just awscli and AWSIoTPythonSDK and see how that goes.

Has anyone got any idea why awscli might intefere with AWSIoTPythonSDK?

Thank you,
-Noah

15

OK, I can confirm that this minimal container breaks on AWSIoTPythonSDK:

# base-image for python on any machine using a template variable,
# see more about dockerfile templates here: https://www.balena.io/docs/learn/develop/dockerfile/
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run

# Set our working directory
WORKDIR /usr/src/app

# copy all local files to container
COPY . ./

RUN apt-get update


# development
#RUN apt install git
#RUN apt-get -y install gcc
#RUN apt install -y vim
#RUN apt install -y dos2unix
#RUN apt install joe
RUN /usr/local/bin/python3.8 -m pip install --upgrade pip

# test for compatibility
RUN apt install awscli wget libssl-dev
RUN apt install cmake
RUN pip3 install AWSIoTPythonSDK