Custom BalenaOS NUC build 'find' error

Product support balenaOS
1

G’day

Trying to get a quick balenaos build going - ran into the find error below, find works fine out of the Python process call.

My goal is to enable a few existing modules and flags in the base BalenaOS to get the NUC spin working for my device.

My main dev machine is Arch, not a directly supported Yocto host platform but that really should be fine. Before I re-do the whole thing in a Ubuntu container - is there a quick fix for the below ? Barring that, seems likely a balena build container already exists - are there any current docs on that ?

❯ ./balena-yocto-scripts/build/barys -m genericx86-64
Building JSON manifest...
audited 3 packages in 1.224s
found 0 vulnerabilities
test_append_and_override_1 (bb.tests.data.TestOverrides) ... ok
...Done
test_append_and_override_3 (bb.tests.data.TestOverrides) ... ok
  _           _                   ___  ____
 | |__   __ _| | ___ _ __   __ _ / _ \/ ___|
 | '_ \ / _` | |/ _ \ '_ \ / _` | | | \___ \
 | |_) | (_| | |  __/ | | | (_| | |_| |___) |
 |_.__/ \__,_|_|\___|_| |_|\__,_|\___/|____/
test_one_override (bb.tests.data.TestOverrides) ... ok
 -------------------------------------------- 
test_remove_with_override (bb.tests.data.TestOverrides) ... ok
Resin specific images available:
	resin-image-flasher
test_append (bb.tests.data.TestRemoteConcat) ... ok
Intel NUC                                : $ MACHINE=genericx86-64 bitbake resin-image-flasher
Microsoft Surface Go (NEW)               : $ MACHINE=surface-go bitbake resin-image-flasher
Microsoft Surface 6 (NEW)                : $ MACHINE=surface-pro-6 bitbake resin-image-flasher

[000000002][LOG]BalenaOS build initialized in directory: build.
[000000002][LOG]Run build for genericx86-64: MACHINE=genericx86-64 bitbake resin-image-flasher 
[000000002][LOG]This might take a while ...
WARNING: Host distribution "arch" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |###################################################################################################################################################################################################################################################################| Time: 0:00:00
Loaded 3341 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.42.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "arch"
TARGET_SYS           = "x86_64-poky-linux"
MACHINE              = "genericx86-64"
DISTRO               = "balena-os"
DISTRO_VERSION       = "2.48.0"
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "HEAD:b021992106c6b5ba02b825afa7dcc422b135b59b"
meta-oe              
meta-filesystems     
meta-networking      
meta-python          = "HEAD:6fa72d587aab8e9d56d67a2552eb9cfbe25c86d2"
meta-balena-common   
meta-balena-warrior  = "HEAD:19c838a3322f6b419a6c2a08fa33677016a2bac9"
meta-balena-genericx86 = "master:af1de5a1b7acf31cbf66476bda754a2ee951d4b1"
meta-rust            = "HEAD:c5ff05d1e99c1e80844c1673456e2368499918c7"

WARNING: Your host glibc verson (2.31) is newer than that in uninative (2.30). Disabling uninative so that sstate is not corrupted.
Initialising tasks: 100% |##############################################################################################################################################################################################################################################################| Time: 0:00:02
Sstate summary: Wanted 1407 Found 0 Missed 1407 Current 72 (0% match, 4% complete)
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: base-files-3.0.14-r89 do_package: Fatal errors occurred in subprocesses:
Command '['file', '-b', '/home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.profile']' died with <Signals.SIGSYS: 31>.: Traceback (most recent call last):
  File "/home/thys/balena/balena-intel/build/../layers/poky/meta/lib/oe/utils.py", line 276, in run
    ret = self._target(*self._args, **self._kwargs)
  File "/home/thys/balena/balena-intel/build/../layers/poky/meta/lib/oe/package.py", line 74, in is_elf
    result = subprocess.check_output(["file", "-b", path], stderr=subprocess.STDOUT).decode("utf-8")
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['file', '-b', '/home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.profile']' died with <Signals.SIGSYS: 31>.

Command '['file', '-b', '/home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.bashrc']' died with <Signals.SIGSYS: 31>.: Traceback (most recent call last):
  File "/home/thys/balena/balena-intel/build/../layers/poky/meta/lib/oe/utils.py", line 276, in run
    ret = self._target(*self._args, **self._kwargs)
  File "/home/thys/balena/balena-intel/build/../layers/poky/meta/lib/oe/package.py", line 74, in is_elf
    result = subprocess.check_output(["file", "-b", path], stderr=subprocess.STDOUT).decode("utf-8")
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['file', '-b', '/home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.bashrc']' died with <Signals.SIGSYS: 31>.
Test LogExecTTY event class ... ok
test_LogHandler (bb.tests.event.EventClassesTest)
ERROR: base-files-3.0.14-r89 do_package: 
ERROR: base-files-3.0.14-r89 do_package: Function failed: split_and_strip_files
ERROR: Logfile of failure stored in: /home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/temp/log.do_package.2601353
ERROR: Task (/home/thys/balena/balena-intel/build/../layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_package) failed with exit code '1'
NOTE: Tasks Summary: Attempted 761 tasks of which 751 didn't need to be rerun and 1 failed.
test_MultipleProviders (bb.tests.event.EventClassesTest)
Summary: 1 task failed:
  /home/thys/balena/balena-intel/build/../layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_package
Summary: There were 2 WARNING messages shown.
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
[000000020][LOG]Build for genericx86-64 failed. Check failed log in build/tmp/log/cooker/genericx86-64 .
[000000020][LOG]If build for genericx86-64 succeeded, final image should have been generated here:
[000000020][LOG]   build/tmp/deploy/images/genericx86-64/resin-image-flasher-genericx86-64.resinos-img
[000000020][LOG]Done.
❯ file -b /home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.profile
ASCII text
❯ file -b /home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.profile
ASCII text
❯ less build/tmp/log/cooker/genericx86-64/console-latest.log
❯ file -b /home/thys/balena/balena-intel/build/tmp/work/genericx86_64-poky-linux/base-files/3.0.14-r89/package/etc/skel/.bashrc
ASCII text
3

Hi, given that Arch is not a Yocto Project supported distribution there might be other issues waiting along the road. My suggestion would be to use a container as you suggest. There is no official BalenaOS builder, but you can try to use https://hub.docker.com/r/aggurio/docker-balena.
Let me know how it goes,

6

I came much further with the container builder than before thanks, however when I try and build the NUC image like so:

docker run --rm -it --privileged -v $(pwd)/build/balena-intel:/home/build balena-builder:$RELEASE ./balena-yocto-scripts/build/barys --rm-work -d --machine genericx86-64

I get the following failure, and no *img file

| Docker started.
| Pulling balena/amd64-supervisor:v10.8.0...
| Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
| Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
| WARNING: /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/temp/run.do_compile.1212:1 exit 1 from '$DOCKER run --privileged --rm -e BALENA_STORAGE=aufs -e USER_ID=$(id -u) -e USER_GID=$(id -u) -e TARGET_REPOSITORY="balena/amd64-supervisor" -e TARGET_TAG="v10.8.0" -e HELLO_REPOSITORY="amd64/hello-world" -e HELLO_PLATFORM="linux/amd64" -e PRIVATE_REGISTRY="${PRIVATE_REGISTRY}" -e PRIVATE_REGISTRY_USER="${PRIVATE_REGISTRY_USER}" -e PRIVATE_REGISTRY_PASSWORD="${PRIVATE_REGISTRY_PASSWORD}" -e PARTITION_SIZE="192" -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/build:/build --name ${_container_name} ${_image_name}'
| ERROR: Function failed: do_compile (log file is located at /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/temp/log.do_compile.1212)
ERROR: Task (/home/build/build/../layers/meta-balena/meta-balena-common/recipes-containers/docker-disk/docker-disk.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3543 tasks of which 3492 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/build/build/../layers/meta-balena/meta-balena-common/recipes-containers/docker-disk/docker-disk.bb:do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
[000000081][LOG]Build for genericx86-64 failed. Check failed log in build/tmp/log/cooker/genericx86-64 .
[000000081][LOG]If build for genericx86-64 succeeded, final image should have been generated here:
[000000081][LOG]   build/tmp/deploy/images/genericx86-64/resin-image-flasher-genericx86-64.resinos-img
[000000081][LOG]Done.

Interestingly the surface-go built seems to be fine, and it produces a *img file.

docker run --rm -it --privileged -v $(pwd)/build/balena-intel:/home/build balena-builder:$RELEASE ./balena-yocto-scripts/build/barys --rm-work -d --machine surface-go

7

Hi
Are you building on Arch or the builder that Alex linked above?

9

So I’m using the builder, like above, on my Arch dev machine.

11

Are you consistently running into this issue with the x86 builds?

12

*genericx86-64

13

Yes, just tried to rebuild again:

RELEASE=warrior

docker run --rm -it --privileged -v $(pwd)/build/balena-intel:/home/build balena-builder:$RELEASE ./balena-yocto-scripts/build/barys --rm-work -d --machine genericx86-64

| time=“2020-04-14T13:41:59.195531558Z” level=info msg=“pickfirstBalancer: HandleSubConnStateChange: 0xc420460770, CONNECTING” module=grpc
| time=“2020-04-14T13:41:59.195765535Z” level=info msg=“pickfirstBalancer: HandleSubConnStateChange: 0xc420460770, READY” module=grpc
| time=“2020-04-14T13:41:59.256113936Z” level=error msg=“AUFS was not found in /proc/filesystems” storage-driver=aufs
| Error starting daemon: error initializing graphdriver: driver not supported
| Docker started.
| Pulling balena/amd64-supervisor:v10.8.0…
| Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
| Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
| WARNING: /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/temp/run.do_compile.1803:1 exit 1 from ‘$DOCKER run --privileged --rm -e BALENA_STORAGE=aufs -e USER_ID=$(id -u) -e USER_GID=$(id -u) -e TARGET_REPOSITORY=“balena/amd64-supervisor” -e TARGET_TAG=“v10.8.0” -e HELLO_REPOSITORY=“amd64/hello-world” -e HELLO_PLATFORM=“linux/amd64” -e PRIVATE_REGISTRY=“${PRIVATE_REGISTRY}” -e PRIVATE_REGISTRY_USER=“${PRIVATE_REGISTRY_USER}” -e PRIVATE_REGISTRY_PASSWORD=“${PRIVATE_REGISTRY_PASSWORD}” -e PARTITION_SIZE=“192” -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/build:/build --name ${_container_name} ${_image_name}’
| ERROR: Function failed: do_compile (log file is located at /home/build/build/tmp/work/genericx86_64-poky-linux/docker-disk/balena_amd64-supervisor-v10.8.0-r0/temp/log.do_compile.1803)
ERROR: Task (/home/build/build/…/layers/meta-balena/meta-balena-common/recipes-containers/docker-disk/docker-disk.bb:do_compile) failed with exit code ‘1’
NOTE: Tasks Summary: Attempted 4402 tasks of which 4266 didn’t need to be rerun and 1 failed.

Summary: 1 task failed:
/home/build/build/…/layers/meta-balena/meta-balena-common/recipes-containers/docker-disk/docker-disk.bb:do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
[000000092][LOG]Build for genericx86-64 failed. Check failed log in build/tmp/log/cooker/genericx86-64 .
[000000092][LOG]If build for genericx86-64 succeeded, final image should have been generated here:
[000000092][LOG] build/tmp/deploy/images/genericx86-64/resin-image-flasher-genericx86-64.resinos-img

16

Hi, looks like your host machine doens’t support AUFS

| time=“2020-04-14T13:41:59.256113936Z” level=error msg=“AUFS was not found in /proc/filesystems” storage-driver=aufs
| Error starting daemon: error initializing graphdriver: driver not supported

You can overwrite it here: https://github.com/alexgg/docker-balena/blob/master/Dockerfile#L26

17

You can check what your docker is using by checking docker info

19

So I changed my containers extra opts to

ENV DOCKER_EXTRA_OPTS ‘–storage-driver=overlay2’

as this is what my docker uses:

❯ docker info
Client:
Debug Mode: false

Server:
 Containers: 110
  Running: 1
  Paused: 0
  Stopped: 109
 Images: 622
 Server Version: 19.03.8-ce
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d76c121f76a5fc8a462dc64594aea72fe18e1178.m
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.6.3-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.28GiB
 Name: rog
 ID: QZD3:7Z3E:3KAP:JO47:B2HS:QNQG:DDVE:YHVS:LJYY:Q67J:OGRT:5YKU
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

But that made no difference to the genericx86-64 build - keep in mind it works fine for the surface-go build.

26

But that made no difference to the genericx86-64 build - keep in mind it works fine for the surface-go build.

@sthysel, I’ve gathered “some evidence” (as opposed to certainty) that the NUC images require the AUFS driver, whereas surface-go require the overlay2 driver:

These settings:

ENV DOCKER_EXTRA_OPTS ‘–storage-driver=aufs’

ENV DOCKER_EXTRA_OPTS ‘–storage-driver=overlay2’

… would tell Docker which storage drivers to load, and the Linux kernel needs the module to be loaded as well, but it is not a choice: the NUC image needs AUFS, the surface-go image needs overlay2.

The following Docker documentation indicates how to check that the host OS kernel supports AUFS: Use the AUFS storage driver | Docker Docs

$ grep aufs /proc/filesystems
nodev   aufs

Try ensuring that the host OS / kernel and Docker have support for AUFS when building with the NUC image.