ResinOS custom build with Docker

brice · October 5, 2017, 2:09am

Was reading through the documentation to do a custom image build for ResinOS: https://resinos.io/docs/custombuild/#bake-your-own-image

Do you have a Docker image that you could share to setup the Yocto environment and run the builds?

Thanks,
Brice

willnewton · October 5, 2017, 10:00am

We do have a docker image which can be found on dockerhub:

https://hub.docker.com/r/resin/yocto-build-env/

It’s built from a Dockerfile in this repo:

However it’s not really well documented or user friendly as it’s considered internal infrastructure. If you’d like to repurpose it for your own builds then we will do what we can to help though.

brice · October 5, 2017, 9:40pm

Awesome, thanks @willn!

brice · October 6, 2017, 2:44am

@willn running into an error trying to run Docker directly, initially couldn’t run because “groupadd: GID ‘20’ already exists” so I’ve manually changed the group id but then I run into an issue later through the build.

Command

docker run --rm -v $(pwd):/yocto/resin-board -v $(pwd)/shared-downloads:/yocto/shared-downloads -v $(pwd)/shared-sstate:/yocto/shared-sstate -e BUILDER_UID=$(id -u) -e BUILDER_GID=2000 --name yocto-build-raspberrypi3 --privileged resin/yocto-build-env /prepare-and-start.sh --log --machine "raspberrypi3" --shared-downloads /yocto/shared-downloads --shared-sstate /yocto/shared-sstate --rm-work

Error

ERROR: quilt-native-0.64-r0 do_populate_sysroot: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_task_postfunc(d)
     0003:
File: '/yocto/resin-board/build/../layers/poky/meta/classes/sstate.bbclass', lineno: 672, function: sstate_task_postfunc
     0668:
     0669:python sstate_task_postfunc () {
     0670:    shared_state = sstate_state_fromvars(d)
     0671:
 *** 0672:    sstate_install(shared_state, d)
     0673:    for intercept in shared_state['interceptfuncs']:
     0674:        bb.build.exec_func(intercept, d, (d.getVar("WORKDIR", True),))
     0675:    omask = os.umask(0o002)
     0676:    if omask != 0o002:
File: '/yocto/resin-board/build/../layers/poky/meta/classes/sstate.bbclass', lineno: 276, function: sstate_install
     0272:
     0273:    # Run the actual file install
     0274:    for state in ss['dirs']:
     0275:        if os.path.exists(state[1]):
 *** 0276:            oe.path.copyhardlinktree(state[1], state[2])
     0277:
     0278:    for postinst in (d.getVar('SSTATEPOSTINSTFUNCS', True) or '').split():
     0279:        # All hooks should run in the SSTATE_INSTDIR
     0280:        bb.build.exec_func(postinst, d, (sstateinst,))
File: '/yocto/resin-board/build/../layers/poky/meta/lib/oe/path.py', lineno: 91, function: copyhardlinktree
     0087:            source = source + '%s/*' % src
     0088:        else:
     0089:            source = src
     0090:        cmd = 'cp -afl --preserve=xattr %s %s' % (source, dst)
 *** 0091:        subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
     0092:    else:
     0093:        copytree(src, dst)
     0094:
     0095:def remove(path, recurse=True):
File: '/usr/lib/python3.5/subprocess.py', lineno: 626, function: check_output
     0622:        # empty string. That is maintained here for backwards compatibility.
     0623:        kwargs['input'] = '' if kwargs.get('universal_newlines', False) else b''
     0624:
     0625:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
 *** 0626:               **kwargs).stdout
     0627:
     0628:
     0629:class CompletedProcess(object):
     0630:    """A process that has finished running.
File: '/usr/lib/python3.5/subprocess.py', lineno: 708, function: run
     0704:            raise
     0705:        retcode = process.poll()
     0706:        if check and retcode:
     0707:            raise CalledProcessError(retcode, process.args,
 *** 0708:                                     output=stdout, stderr=stderr)
     0709:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0710:
     0711:
     0712:def list2cmdline(seq):
Exception: subprocess.CalledProcessError: Command 'cp -afl --preserve=xattr /yocto/resin-board/build/tmp/work/x86_64-linux/quilt-native/0.64-r0/sysroot-destdir/yocto/resin-board/build/tmp/sysroots/x86_64-linux/* /yocto/resin-board/build/tmp/sysroots/x86_64-linux' returned non-zero exit status 1

Subprocess output:
cp: cannot create hard link '/yocto/resin-board/build/tmp/sysroots/x86_64-linux/usr/share/quilt/compat/awk' to '/yocto/resin-board/build/tmp/work/x86_64-linux/quilt-native/0.64-r0/sysroot-destdir/yocto/resin-board/build/tmp/sysroots/x86_64-linux/usr/share/quilt/compat/awk': No such file or directory

ERROR: quilt-native-0.64-r0 do_populate_sysroot: Function failed: sstate_task_postfunc
ERROR: Logfile of failure stored in: /yocto/resin-board/build/tmp/work/x86_64-linux/quilt-native/0.64-r0/temp/log.do_populate_sysroot.1511
NOTE: recipe quilt-native-0.64-r0: task do_populate_sysroot: Failed
ERROR: Task (/yocto/resin-board/build/../layers/poky/meta/recipes-devtools/quilt/quilt-native_0.64.bb:do_populate_sysroot) failed with exit code '1'

floion · October 10, 2017, 6:57am

Hi Brice,
you could use the resin-yocto-scripts repository and build using the above mentioned Docker image. Have a look at the available options for running it here:

github.com

resin-os/resin-yocto-scripts/blob/master/automation/jenkins_build.sh

#!/bin/bash

set -ex

BUILD_CONTAINER_NAME=yocto-build-$$

print_help() {
	echo -e "Script options:\n\
	\t\t -h | --help\n
	\t\t -m | --machine\n\
	\t\t\t (mandatory) Machine to build for. This is a mandatory argument\n
	\t\t --shared-dir\n\
	\t\t\t (mandatory) Directory where to store shared downloads and shared sstate.\n
	\t\t -b | --build-flavor\n\
	\t\t\t (mandatory) The build flavor. Can be one of the following: managed-dev, managed-prod, unmanaged-dev, unmanaged-prod\n
	\t\t --meta-resin-branch\n\
	\t\t\t (optional) The meta-resin branch to checkout before building.\n\
\t\t\t\t Default value is __ignore__ which means it builds the meta-resin revision as configured in the git submodule.\n
	\t\t --supervisor-tag\n\
	\t\t\t (optional) The resin supervisor tag specifying which supervisor version is to be included in the build.\n\

This file has been truncated. show original

brice · October 11, 2017, 4:21am

@floion found out the issue, using mac and docker doesn’t play nicely with hardlinks and symlinks.

floion · October 11, 2017, 4:42am

Glad to hear that. The above method I pointed could also let you build by using the docker container as pointed above, as the host environment and not depend on the OS installed on your development machine

brice · October 11, 2017, 5:53am

@floion are you building on macos? I’ve tried the jenkins script initially and had the same issue with mounted volumes not playing well with hardlinks.

floion · October 11, 2017, 1:07pm

No, used it on Linux only

brice · October 23, 2017, 10:24pm

@floion I’m trying to build the Jetson TX2 version of resinos but running into issues. Any idea what’s going wrong please?

Cloned:
https://github.com/resin-os/resin-jetson-tx2

I’m running:
./resin-yocto-scripts/build/barys -m jetson-tx2 -d --resinio -r

And this is the error I get after few hours:

| Adding dependency on xproto-native
| NOTE: gcc-runtime exists in sysroot, skipping
| NOTE: glibc exists in sysroot, skipping
| NOTE: quilt-native exists in sysroot, skipping
| NOTE: linux-tegra exists in sysroot, skipping
| NOTE: gcc-cross-aarch64 exists in sysroot, skipping
| NOTE: linux-libc-headers exists in sysroot, skipping
| NOTE: libgcc exists in sysroot, skipping
| NOTE: automake-native exists in sysroot, skipping
| NOTE: binutils-cross-aarch64 exists in sysroot, skipping
| NOTE: zlib-native exists in sysroot, skipping
| NOTE: gnu-config-native exists in sysroot, skipping
| NOTE: libtool-native exists in sysroot, skipping
| NOTE: libmpc-native exists in sysroot, skipping
| NOTE: autoconf-native exists in sysroot, skipping
| NOTE: texinfo-dummy-native exists in sysroot, skipping
| NOTE: mpfr-native exists in sysroot, skipping
| NOTE: gmp-native exists in sysroot, skipping
| NOTE: bison-native exists in sysroot, skipping
| NOTE: flex-native exists in sysroot, skipping
| NOTE: xz-native exists in sysroot, skipping
| NOTE: m4-native exists in sysroot, skipping
| NOTE: gettext-minimal-native exists in sysroot, skipping
| DEBUG: sed -e 's:^[^/]*/:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot/:g' /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/jetson_tx2/depmodwrapper-cross/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot-native:g' -e 's:FIXME_COMPONENTS_DIR:/home/Documents/resin-jetson-tx2/build/tmp/sysroots-components:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/Documents/resin-jetson-tx2/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/Documents/resin-jetson-tx2/build/tmp/pkgdata/jetson-tx2:g'
| DEBUG: sed -e 's:^[^/]*/:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot-native/:g' /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/gtk-doc-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/pkgconfig-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/elfutils-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/python-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/glib-2.0-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/e2fsprogs-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/rpm-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/openssl-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/libpcre-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/python3-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/gettext-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/libpng-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/dbus-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/curl-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/ncurses-native/fixmepath /home/Documents/resin-jetson-tx2/build/tmp/sysroots-components/x86_64/nspr-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/recipe-sysroot-native:g' -e 's:FIXME_COMPONENTS_DIR:/home/Documents/resin-jetson-tx2/build/tmp/sysroots-components:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/Documents/resin-jetson-tx2/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/Documents/resin-jetson-tx2/build/tmp/pkgdata/jetson-tx2:g'
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_install
| cp: cannot stat '/home/Documents/resin-jetson-tx2/build/tmp/deploy/images/jetson-tx2/u-boot-dtb.bin': No such file or directory
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_install (log file is located at /home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/temp/log.do_install.27860)
ERROR: Task (/home/Documents/resin-jetson-tx2/build/../layers/meta-resin-jetson/recipes-support/tegra-binaries-prepare/tegra-binaries-prepare_28.1.0.bb:do_install) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2879 tasks of which 6 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/Documents/resin-jetson-tx2/build/../layers/meta-resin-jetson/recipes-support/tegra-binaries-prepare/tegra-binaries-prepare_28.1.0.bb:do_install
Summary: There were 3 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
[000005328][LOG]Build for jetson-tx2 failed. Check failed log in build/tmp/log/cooker/jetson-tx2 .
[000005328][LOG]If build for jetson-tx2 succeeded, final image should have been generated here:
[000005328][LOG]   build/tmp/deploy/images/jetson-tx2/resin-image-flasher-jetson-tx2.resinos-img
[000005328][LOG]Done.

floion · October 24, 2017, 7:45am

I missed this thread, sorry.
Are you still seeing these issues with the latest codebase?

brice · October 24, 2017, 8:03am

@floion all good still having that issue, I cloned the resinos/jetson-tx2 repo this morning and tried to build on Ubuntu 16.04 directly with the Yocto dependencies, npm and jq. (Had similar issues with the docker container).

floion · October 24, 2017, 8:16am

What is the git clone command you are using?

brice · October 24, 2017, 8:53am

I use:
git clone --recursive https://github.com/resin-os/resin-jetson-tx2

floion · October 24, 2017, 10:12am

@telphan can you look into this please?

telphan · October 24, 2017, 12:06pm

I have tested this and I run into no issues. Would you mind telling me the SHA of the repo that you are trying to build?

brice · October 24, 2017, 10:53pm

The SHA is 11ae421f8d3ccade9e2011d4780f0d103bd0967e, cloned from master 2 days ago.

telphan · October 25, 2017, 9:42am

I think I see the issue here. The tegra-binaries-prepare package needs to depend on virtual/bootloader, to ensure the existents of u-boot-dtb.bin.
A PR addressing this can be found: https://github.com/resin-os/resin-jetson-tx2/pull/25

brice · October 26, 2017, 5:04am

Tried again today from SHA 55127a72c5bf826a9a8fe8a9db4055adbb5b8d2c (release_v2.7.4+rev1 branch) and still complains about the same issue:

| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_install
| cp: cannot stat '/home/Documents/resin-jetson-tx2/build/tmp/deploy/images/jetson-tx2/u-boot-dtb.bin': No such file or directory
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_install (log file is located at /home/Documents/resin-jetson-tx2/build/tmp/work/aarch64-poky-linux/tegra-binaries-prepare/28.1.0-r0/temp/log.do_install.4636)

Full log from log.do_install.4636
https://pastebin.com/VJmq48XD

brice · October 27, 2017, 10:34am

So I tried one more time with the jenkins_build.sh script and it works now, must be have been an issue with a dependency somewhere as I couldn’t get it to work on ubuntu directly last time.

Thanks again for your help @floion @willnewton and @telphan