I am trying to migrate our build system to a different PC, and as part of that upgrading several components. I am trying to use balena 18.2.0 on Ubuntu 23.10
I am stuck with this error, the Balena build fails with a network problem:
[Build] main Step 19/21 : RUN pip3 install 'pyserial==3.4'
[Build] main ---> Running in 110510e6fdd2
[Build] main WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ecc9ad0c130>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/pyserial/
… which occurs when running this command: balena build --deviceType intel-nuc --arch intel-nuc --docker '/var/run/balena-engine.sock'
The same command works fine on the old system where balena 12.47.0 is installed, but I am uncertain whether this is due to the older version or some configuration I have overlooked?
The command also works if I use the /var/run/docker.sock (which is also running on the same system): balena build --deviceType intel-nuc --arch intel-nuc --docker '/var/run/docker.sock'
… so somehow dockerd daemon is different than balena-engine daemon?
The problem is clearly a general network access issue. It is not specific to pip3 install. For example, it will also fail if the first uncached RUN command is apt-get update.
The problem is there regardless of using docker build ... or balena build ...
but for docker I can fix the problem using docker build --network=host ...
However, balena build ... does not support the --network option
… so this is not a solution…
ARG PARENT_VERSION=noetic-desktop
ARG PARENT_LOCATION=osrf/
ARG IMAGE_NAME=ros
FROM ${PARENT_LOCATION}${IMAGE_NAME}:${PARENT_VERSION}
RUN apt-get update
Here is a failing scenario:
$ balena build --deviceType intel-nuc --arch intel-nuc --docker '/var/run/balena-engine.sock'
[Info] No "docker-compose.yml" file found at "/var/lib/jenkins/workspace/Docker_Build_Application/containers/balena_build_network_issue"
[Info] Creating default composition with source: "/var/lib/jenkins/workspace/Docker_Build_Application/containers/balena_build_network_issue"
[Build] Building services...
[Build] main Preparing...
[Info] Building for intel-nuc/intel-nuc
[Build] main Step 1/5 : ARG PARENT_VERSION=noetic-desktop
[Build] main Step 2/5 : ARG PARENT_LOCATION=osrf/
[Build] main Step 3/5 : ARG IMAGE_NAME=ros
[Build] main Step 4/5 : FROM ${PARENT_LOCATION}${IMAGE_NAME}:${PARENT_VERSION}
[Build] main ---> 0189cf4607be
[Build] main Step 5/5 : RUN apt-get update
[Build] main ---> Running in 33821b0c31c6
[Build] main Err:1 http://archive.ubuntu.com/ubuntu focal InRelease
[Build] main Temporary failure resolving 'archive.ubuntu.com'
[Build] main Err:2 http://packages.ros.org/ros/ubuntu focal InRelease
[Build] main Temporary failure resolving 'packages.ros.org'
^C[Build] Built 1 service in 2:14
[Error] Build failed.
Here is a successful alternative (using a different socket):
$ balena build --deviceType intel-nuc --arch intel-nuc --docker '/var/run/docker.sock'
[Info] No "docker-compose.yml" file found at "/var/lib/jenkins/workspace/Docker_Build_Application/containers/balena_build_network_issue"
[Info] Creating default composition with source: "/var/lib/jenkins/workspace/Docker_Build_Application/containers/balena_build_network_issue"
[Build] Building services...
[Build] main Preparing...
[Info] Building for intel-nuc/intel-nuc
[Build] main Step 1/5 : ARG PARENT_VERSION=noetic-desktop
[Build] main Step 2/5 : ARG PARENT_LOCATION=osrf/
[Build] main Step 3/5 : ARG IMAGE_NAME=ros
[Build] main Step 4/5 : FROM ${PARENT_LOCATION}${IMAGE_NAME}:${PARENT_VERSION}
[Build] main ---> 0189cf4607be
[Build] main Step 5/5 : RUN apt-get update
[Build] main ---> Running in 00e878e29a41
[Build] main Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
[Build] main Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB]
[Build] main Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB]
[Build] main Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB]
... [snip]...
Why would you like to build using balena-engine instead of docker? Building with docker should bring you at least the same or better docker capabilities.
I gave a minimal docker file example to demonstrate the error I get. But as you see in the original example, the problem occurs when commanding: balena build --deviceType intel-nuc --arch intel-nuc --docker '/var/run/balena-engine.sock'
… which to the best of my knowledge can not be replaced with native docker.
Hey @krix , how did you setup the balenaEngine on your host? Was it just with the script provided here?
I have a theory that if we provide the balenaEngine daemon some startup parameters for default networking we may be able to get it working.
On balenaOS we use the flags --dns, --bip, and --fixed-cidr as seen here but --default-address-pool might also be helpful as documented in the Docker docs here.
Would you like to try some of those options to see if this resolves the networking issue when running balenaEngine on your workstation? If any of those options provide network access to the RUN instructions it may be worth updating our install steps.
Another option could be to use a balena device in local mode as your engine socket. This would likely be slower than your workstation but would at least support the secrets functionality and have working networking.
e.g. balena build --dockerHost <short-uuid>.local --dockerPort 2375