Building BalenaOS for Rockchip 3328 / NanoPi Neo3

mmelnick · February 12, 2021, 6:57pm

Hi all,

I’m in the process of attempting to build BalenaOS support for the NanoPi Neo3. So far I’ve been reasonably successful in overcoming a variety of issues, including some documentation that doesn’t line up with what actually needs to be in the build, but I’ve run into a roadblock with little further documentation that I’m hoping someone can shed some light on.

The error occurs after running barys:

Preformatted text`ERROR: Nothing PROVIDES 'resin-image
[2]balena-nanopineo3/blob/a1ff517f70bd70434161cb89060288cd56f0c2c1/build/tmp/log/cooker/nanopi-neo3/20210211211422.log#L2) resin-image was skipped: missing required distro feature ‘systemd’ (not in DISTRO_FEATURES)

I’ve done some work to try and track this down that so far hasn’t budged the issue or made clear to me where it’s originating. I’m using the BSP found here:

which does include references to systemd, and I note that BalenaOS makes reference to a DISTRO ? = 'resin-systemd', however removing references to systemd in the BSP layer, as well as changing them to ‘resin-systemd’ haven’t made any change to the error, leading me to believe the problem must exist somewhere else. Can anyone provide some usual suspects to look at next? I’m concerned that there’s a specific Balena/Resin implementation here that I’m missing.

Thanks very much,

~Michael

alexgg · February 15, 2021, 5:29pm

Hi, it may not be that you are not using the resin-systemd distribution or you maybe haven’t included the meta-balena layer as part of your bblayers.conf? See how the balena-raspberrypi device type repository configures the distro for all device types as a local.conf template:

github.com

balena-os/balena-raspberrypi/blob/56448fe991e84fa38e7b7a38d79fef1690256376/layers/meta-balena-raspberrypi/conf/samples/local.conf.sample#L61


#SUPERVISOR_TAG ?= "master"
# Compress final raw image
#RESIN_RAW_IMG_COMPRESSION ?= "xz"
# Parallelism Options
BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
# Resin specific distros
DISTRO ?= "resin-systemd"
# Custom downloads directory
#DL_DIR ?= "${TOPDIR}/downloads"
# Custom sstate directory
#SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
# Inheriting this class has shown to speed up builds due to significantly lower
# amounts of data stored in the data cache as well as on disk.
# http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-classes-rm-work

mmelnick · February 15, 2021, 5:56pm

Hi @alexgg,

Thanks for the response - so I actually modeled my local.conf template off of what I saw in that repository. I’m reproducing it below for you in case I missed something, but it does include the reference to resin-systemd as it should there. This is also why I tried replacing other references to simple systemd to resin-systemd but to no obvious avail. Any idea if there are other dependencies that could be hiding elsewhere? I could make the repository available publicly if that’s likely to help.

You’ll notice that much of this is commented out / redundant, potentially irrelevant to my current build - I expect that’s true at this stage, I was mostly focused on retaining key pieces (like perhaps resin-systemd) and handling errors downstream for unnecessary specifications here. That being said - anything you see that’s obviously wrong by all means!

# Supported machines
MACHINE ?= "nanopi-neo3"
MACHINE ?= "rockchip"
DISTRO ?= "poky"
#MACHINE ?= "nanopirs2"

#BALENA_STORAGE_rt-rpi-300 = "overlay2"
#BALENA_STORAGE_raspberrypi4-64 = "overlay2"
#BALENA_STORAGE_raspberrypi400-64 = "overlay2"
#BALENA_STORAGE_revpi-connect = "overlay2"

# More info meta-resin/README.md
#TARGET_REPOSITORY ?= ""
#TARGET_TAG ?= ""

# for the moment, we disable vc4 graphics for all but the 64 bits machines
#DISABLE_VC4GRAPHICS = "1"

# RPI BSP uses uncompressed kernel images by default
KERNEL_IMAGETYPE="zImage"
KERNEL_BOOTCMD="bootz"

# When u-boot is enabled we need to use the "Image" format and the "booti"
# command to load the kernel for rpi3 64 bits
#KERNEL_IMAGETYPE_raspberrypi3-64="Image.gz"
#KERNEL_BOOTCMD_raspberrypi3-64 = "booti"
#KERNEL_IMAGETYPE_raspberrypi4-64="Image.gz"
#KERNEL_BOOTCMD_raspberrypi4-64 = "booti"

# RPI Use u-boot. This needs to be 1 as we use u-boot
RPI_USE_U_BOOT = "1"

# Set this to 1 if development image is desired
DEVELOPMENT_IMAGE = "1"

# Set this to make build system generate resinhup bundles
#RESINHUP ?= "yes"

# Set this to change the supervisor tag used
#SUPERVISOR_TAG ?= "master"

# Compress final raw image
#RESIN_RAW_IMG_COMPRESSION ?= "xz"

# Parallelism Options
BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"

# Resin specific distros
DISTRO ?= "resin-systemd"

# Custom downloads directory
#DL_DIR ?= "${TOPDIR}/downloads"

# Custom sstate directory
#SSTATE_DIR ?= "${TOPDIR}/sstate-cache"

# Inheriting this class has shown to speed up builds due to significantly lower
# amounts of data stored in the data cache as well as on disk.
# http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-classes-rm-work
#INHERIT += "rm_work"

# Remove the old image before the new one is generated to save disk space when RM_OLD_IMAGE is set to 1, this is an easy way to keep the DEPLOY_DIR_IMAGE clean.
RM_OLD_IMAGE = "1"

# Additional image features
USER_CLASSES ?= "buildstats image-mklibs image-prelink"

# By default disable interactive patch resolution (tasks will just fail instead):
PATCHRESOLVE = "noop"

# Disk Space Monitoring during the build
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    ABORT,${TMPDIR},100M,1K

alexgg · February 15, 2021, 7:07pm

Hi, the problem is that you include DISTRO twice:

DISTRO ?= "poky"
...
DISTRO ?= "resin-systemd"

The bitbake ?= operator means assing only if not already defined. So the first assignment to poky works and the second one doesn’t. Try removing the first and let me know.

mmelnick · February 15, 2021, 7:07pm

@alexgg - simple but I wouldn’t have figured it out. The build’s continuing for now, I’m guessing this won’t be my last error but I’ll plan to hack away with any errors I encounter until I get completely stuck on something.

Thanks so much for taking a look, this was super helpful.

~Michael

mmelnick · February 22, 2021, 11:04pm

Hi all,

Still working on this build and I’ve been running up against an issue where a specific u-boot file (u-boot.itb) isn’t ending up getting copied correctly. I realize this may be down in the weeds of this particular BSP, but just in case this is a known problem with u-boot / resin u-boot, the details are:

ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/u-boot/v2020.07+gitAUTOINC+2f5fbb5b39-r0/temp/log.do_deploy.2934308

Log data follows:

| DEBUG: Executing python function sstate_task_prefunc

| DEBUG: Python function sstate_task_prefunc finished

| DEBUG: Executing shell function do_deploy

| install: cannot stat '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/u-boot/v2020.07+gitAUTOINC+2f5fbb5b39-r0/build/u-boot.itb': No such file or directory

| WARNING: exit code 1 from a shell command.

| ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/u-boot/v2020.07+gitAUTOINC+2f5fbb5b39-r0/temp/run.do_deploy.2934308' failed with exit code 1:

| install: cannot stat '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/u-boot/v2020.07+gitAUTOINC+2f5fbb5b39-r0/build/u-boot.itb': No such file or directory

| WARNING: exit code 1 from a shell command.

|

ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena-nanopi-rockchip64/recipes-bsp/u-boot/u-boot_2020.07.bb:do_deploy) failed with exit code '1'

This error is getting thrown during do_deploy, however presumably the problem may be happening previously in do_install. I’ve hunted through the do_install script and it neither throws any errors, nor shows an obvious place where this could be happening (it does rm some files during cleanup, including u-boot.itb, but commenting this line out doesn’t fix the problem.

For reference, the do_install and do_deploy that ultimately fail come from here:

github.com

dimtass/meta-nanopi-rockchip64/blob/master/recipes-bsp/u-boot/u-boot-rockchip64.inc

DESCRIPTION="Upstream's U-boot configured for allwinner devices"
AUTHOR = "Dimitris Tassopoulos <dimtass@gmail.com>"

require u-boot-common.inc

LIC_FILES_CHKSUM ?= "file://Licenses/README;md5=30503fd321432fc713238f582193b78e"

SRC_URI += " \
            file://${SOC_FAMILY}-boot/boot.cmd \
            file://rkEnv.txt \
"

do_configure() {
    set -x
    cd ${S}
    ${WORKDIR}/armbian-patcher.sh ${WORKDIR}/patches-${SOC_FAMILY}-${UBOOT_VERSION}
    oe_runmake -C ${S} ${UBOOT_CONFIG} O=${B}
}

do_compile() {

This file has been truncated. show original

Given that I’m not getting an error during the previous (do_install) state, and getting an error in the following state, I’m not sure where else to look. A search of the entire build folder doesn’t turn up a u-boot.itb file, so it doesn’t just appear to be a case of it being copied to the wrong place either.

Thanks in advance,

~Michael

alexgg · March 2, 2021, 12:41pm

Hi Michael, following up on our conversation, an itb image is a U-Boot FIT image, an uimage format that can combine kernel zimage and dtb. We don’t use it in BalenaOS, and to build it it first needs to generate the u-boot binaries which are actually used. Try to remove the copy from the do_deploy function with an append to the BSP’s u-boot recipe and let us know if that works. I have also created Replace ssh github URIs with http · Issue #2124 · balena-os/meta-balena · GitHub so that ssh github authentication stops bothering. I have linked the issue here so we’ll update you once it’s solved.

gelbal · March 5, 2021, 2:23pm

Hi Michael, did you have a chance to apply Alex’s suggestion? Did that work?

Cheers…

mmelnick · March 5, 2021, 4:01pm

Hi all,

Thanks for checking in. Sorry for the delay - I’d hoped that after this my next post would be both thank you and “it works!” but we’re not quite there yet.

Commenting out the itb worked great and now we build through ~91% completion. Now something’s failing in the kernel build where it’s failing to stat System.map and config, after calling for a make mrproper. I’m still investigating this one and trying to figure out where the source of this particular do_install is and hope to have better news about that shortly.

@alexgg - thanks for adding the issue. I did get setup with an SSH key on this AWS instance to help alleviate this. Good news is that the build is no longer throwing that warning, so that’s one less issue to worry about.

alexgg · March 5, 2021, 5:49pm

Hi Michael, glad to know it’s a step closer. Paste the kernel build error output and we may be able to suggest something to make the debugging easier.

mmelnick · March 5, 2021, 5:54pm

Hey @alexgg , thanks for the offer. Here’s what I’m running into, and it follows a build error where it says the build tree isn’t clean and to run make mrproper. That succeeds in pushing me a little further, but haven’t found the source of this one yet:

NOTE: Executing Tasks
ERROR: linux-stable-5.7.17-r0 do_install: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/linux-stable/5.7.17-r0/temp/run.do_install.2476139' failed with exit code 1:
grep: .config: No such file or directory
install: cannot stat 'System.map': No such file or directory
WARNING: exit code 1 from a shell command.

ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/linux-stable/5.7.17-r0/temp/log.do_install.2476139
Log data follows:
| DEBUG: Executing python function package_get_auto_pr
| DEBUG: Python function package_get_auto_pr finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/gcc/libgcc_9.3.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/python/python3_3.8.2.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-bsp/u-boot/u-boot-tools_2020.01.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-support/lzo/lzo_2.10.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/gcc/gcc-cross_9.3.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-gnome/gtk-doc/gtk-doc_1.32.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/zlib/zlib_1.2.11.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-extended/xz/xz_5.2.4.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/automake/automake_1.16.1.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/readline/readline_8.0.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/flex/flex_2.6.4.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/binutils/binutils-cross_2.34.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/glibc/glibc_2.31.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/gettext/gettext-minimal-native_0.20.1.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-support/gmp/gmp_6.2.0.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-support/libmpc/libmpc_1.1.0.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-support/lzop/lzop_1.04.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-connectivity/openssl/openssl_1.1.1i.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/bison/bison_3.5.4.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-support/mpfr/mpfr_4.0.2.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['pkgconfig-native', 'libgcc', 'python3-native', 'u-boot-tools-native', 'lzo-native', 'autoconf-native', 'gcc-cross-aarch64', 'kmod-native', 'bc-native', 'gtk-doc-native', 'libtool-native', 'kern-tools-native', 'patch-native', 'zlib-native', 'linux-libc-headers', 'texinfo-dummy-native', 'xz-native', 'pseudo-native', 'automake-native', 'readline-native', 'flex-native', 'binutils-cross-aarch64', 'glibc', 'gettext-minimal-native', 'gnu-config-native', 'gmp-native', 'libmpc-native', 'lzop-native', 'openssl-native', 'bison-native', 'mpfr-native', 'quilt-native', 'm4-native', 'attr-native', 'sqlite3-native', 'bzip2-native', 'gdbm-native', 'util-linux-native', 'libffi-native', 'libtirpc-native', 'libnsl2-native', 'ncurses-native', 'libpcre2-native', 'libcap-ng-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_install
| grep: .config: No such file or directory
| NOTE: no modules to install
| install: cannot stat 'System.map': No such file or directory
| WARNING: exit code 1 from a shell command.
| ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/linux-stable/5.7.17-r0/temp/run.do_install.2476139' failed with exit code 1:
| grep: .config: No such file or directory
| install: cannot stat 'System.map': No such file or directory
| WARNING: exit code 1 from a shell command.
|
ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena-nanopi-rockchip64/recipes-kernel/linux/linux-stable_5.7.bb:do_install) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1790 tasks of which 1789 didn't need to be rerun and 1 failed.

alexgg · March 5, 2021, 6:21pm

Hi Michael, I think the installation of .config and System.map is part of the kernel class in poky, see kernel.bbclass « classes « meta - poky - Poky Build Tool and Metadata

Those files should be created when the kernel is built so what might be happening is that the kernel build is failing. You need to go looking for the log files which should be in the kernel build directory, inside temp/log.do_compile

You can find out which is the building directory to check log files with:

source layers/poky/oe-init-build-env build
MACHINE=nanopi_neo3 bitbake linux-stable -e | grep ^WORKDIR=

If the temp folder is empty you might need to add the following snippet to your conf/local.conf so that build artifacts are not removed:

RM_WORK_EXCLUDE += " \
    linux-stable \
"

And after doing that run:

MACHINE=nanopi_neo3 bitbake linux-stable -c cleanall
MACHINE=nanopi_neo3 bitbake linux-stable

To regenerate the logs.

Let us know if you find the kernel build logs.

mmelnick · March 5, 2021, 8:45pm

Hey @alexgg,

This was super helpful and got me substantially further. Given what you told me above I’d have expected the log not to be there, but the log existed and didn’t have any obvious failures. I’m attaching it here just in case you’re interested. log.do_compile.log (161.6 KB)

At any rate - adding the RM_WORK_EXCLUDE to my .conf worked, and I was able to build and get to the next phase.

The next error was as follows, and asks for another make mrproper, full log below:

ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb:do_configure) failed with exit code '1'

ERROR: kernel-modules-headers-1.0-r0 do_compile: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.597383' failed with exit code 2:

Running modules_prepare...

make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

***

*** The source tree is not clean, please run 'make ARCH=arm64 mrproper'

*** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source

***

make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1

make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2

make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

make: *** [Makefile:180: sub-make] Error 2

WARNING: exit code 2 from a shell command.

ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/log.do_compile.597383

Log data follows:

| DEBUG: Executing shell function do_compile

| Running modules_prepare...

| make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

| ***

| *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'

| *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source

| ***

| make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1

| make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2

| make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

| make: *** [Makefile:180: sub-make] Error 2

| WARNING: exit code 2 from a shell command.

| ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.597383' failed with exit code 2:

| Running modules_prepare...

| make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

| ***

| *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'

| *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source

| ***

| make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1

| make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2

| make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'

| make: *** [Makefile:180: sub-make] Error 2

| WARNING: exit code 2 from a shell command.

|

ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-devtools/kernel-modules-headers/kernel-modules-headers.bb:do_compile) failed with exit code '1'

NOTE: Tasks Summary: Attempted 3410 tasks of which 3381 didn't need to be rerun and 2 failed.

Happily, that came with its own solution, and I could move forward after running the above. In the following re-build I’m seeing these warnings and they may be the last bit of problem I’m hitting:

WARNING: kernel-image-initramfs-1.0-r0 do_package: Manifest /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-resin-image-initramfs.packagedata not found in nanopi_neo3 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

WARNING: kernel-modules-headers-1.0-r0 do_package: Manifest /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-resin-image-initramfs.packagedata not found in nanopi_neo3 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

I did run into a sizing error:

ERROR: resin-image-1.0-r0 do_image_docker: The rootfs size 368640(K) exceeds IMAGE_ROOTFS_MAXSIZE: 319488(K)

ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/temp/log.do_image_docker.960182

ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-core/images/resin-image.bb:do_image_docker) failed with exit code '1'

but I assumed this was really a simple threshold - I added IMAGE_ROOTFS_MAXSIZE = "500000" to recipes-core/images/resin-image.bbappend and that seemed to be more than sufficient. I see a previous changelog that indicates this should be dynamic though - did I break this elsewhere somehow?

Now I seem to really be within the build completing, but it’s not quite there. I’m seeing this error:
| Allocating group tables: done
| Writing inode tables: done
| Creating journal (8192 blocks): done
| Copying files into the device: __populate_fs: Could not allocate block in ext2 filesystem while writing file “Image”
| mkfs.ext4: Could not allocate block in ext2 filesystem while populating file system
| Deleted: sha256:e25e84397823c824d1823092d9c5b1b32326572d299c32dd4ce7bb2bd130d6c1
| Deleted: sha256:b39ac24be79db4c4188f6498a176d79d9e8d37f6c82a0d03e81b801265aa87ae
| Deleted: sha256:3c1c6f550ee188271fd448b2ad4830902d60994b86ae25727ebe1db432560670
| WARNING: exit code 1 from a shell command.

with the full log attached here with more preamble to this log.do_image_hostapp_ext4.995922.log (23.2 KB)

Any further wisdom you could pass along?

Thanks again for your help!

mmelnick · March 8, 2021, 11:03am

@alexgg - so I had a little time to tinker with this, and if the above is really just a hard size limit, it appears that adding IMAGE_ROOTFS_SIZE = "500000" to my local.conf gets past it. Not so sure - it seems like with a clean build even with this set I hit the same error, but when I rebuild it moves past. Building resin-image fails a bit later in trying to find an actual file, which might be helpful to have the Balena point of view on:

ERROR: resin-image-1.0-r0 do_image_hostapp_ext4: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/temp/run.do_image_hostapp_ext4.39581' failed with exit code 1:
0+0 records in
0+0 records out
0 bytes copied, 1.8953e-05 s, 0.0 kB/s
File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210307143008.rootfs.docker
WARNING: exit code 1 from a shell command.

ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/temp/log.do_image_hostapp_ext4.39581
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: 367832.000000 = 367832 * 1.000000
| DEBUG: 367832.000000 = max(367832.000000, 319488)[367832.000000] + 0
| DEBUG: 367832.000000 = int(367832.000000)
| DEBUG: 368640 = aligned(367832)
| DEBUG: returning 368640
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.4.2.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-devtools/kernel-modules-headers/kernel-modules-headers.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.44.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/coreutils/coreutils_8.31.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-containers/mkfs-hostapp-native/mkfs-hostapp-native.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/opkg/opkg_0.4.2.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-core/glibc/cross-localedef-native_2.31.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-openembedded/meta-oe/recipes-devtools/jq/jq_1.6.bb:do_populate_sysroot', '/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-kernel/linux/kernel-headers-test.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['opkg-utils-native', 'update-rc.d-native', 'kernel-modules-headers', 'makedevs-native', 'prelink-native', 'mklibs-native', 'pigz-native', 'depmodwrapper-cross', 'coreutils-native', 'mkfs-hostapp-native', 'qemuwrapper-cross', 'ldconfig-native', 'opkg-native', 'cross-localedef-native', 'jq-native', 'kernel-headers-test', 'pseudo-native', 'systemd-systemctl-native', 'xz-native', 'quilt-native', 'shadow-native', 'gnu-config-native', 'elfutils-native', 'autoconf-native', 'automake-native', 'libtool-native', 'binutils-native', 'gettext-minimal-native', 'patch-native', 'kmod-native', 'attr-native', 'texinfo-dummy-native', 'balena-native', 'e2fsprogs-native', 'hostapp-update-native', 'nss-native', 'qemu-native', 'glibc', 'elfutils', 'util-linux', 'linux-libc-headers', 'libgcc', 'linux-stable', 'gcc-runtime', 'openssl', 'openssl-native', 'debianutils-native', 'perl-native', 'zlib-native', 'shared-mime-info-native', 'libsolv-native', 'libarchive-native', 'pkgconfig-native', 'onig-native', 'm4-native', 'flex-native', 'python3-native', 'gtk-doc-native', 'go-native', 'util-linux-native', 'sqlite3-native', 'nspr-native', 'glib-2.0-native', 'zlib', 'ncurses', 'libcap-ng', 'opkg-utils', 'libxcrypt', 'bash-completion', 'cpio-native', 'kern-tools-native', 'gdbm-native', 'db-native', 'itstool-native', 'libxml2-native', 'expat-native', 'ninja-native', 'cmake-native', 'bzip2-native', 'lzo-native', 'libnsl2-native', 'libffi-native', 'libtirpc-native', 'readline-native', 'libcap-ng-native', 'libpcre2-native', 'ncurses-native', 'libpcre-native', 'meson-native', 'gettext-native', 're2c-native', 'curl-native', 'python3-setuptools-native', 'unzip-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_hostapp_ext4
| 0+0 records in
| 0+0 records out
| 0 bytes copied, 1.8953e-05 s, 0.0 kB/s
| File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210307143008.rootfs.docker
| WARNING: exit code 1 from a shell command.
| ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/temp/run.do_image_hostapp_ext4.39581' failed with exit code 1:
| 0+0 records in
| 0+0 records out
| 0 bytes copied, 1.8953e-05 s, 0.0 kB/s
| File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210307143008.rootfs.docker
| WARNING: exit code 1 from a shell command.
|
ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-core/images/resin-image.bb:do_image_hostapp_ext4) failed with exit code '1'
NOTE: Tasks Summary: Attempted 4119 tasks of which 4118 didn't need to be rerun and 1 failed.

This might be due to the size limit not building the file, hard to say.

alexgg · March 8, 2021, 2:26pm

Hi Michael, let’s take a step back and double-check some of the steps.

Kernel build. You mention this was solved when adding the RM_WORK_EXCLUDE, however that is unlikely as all that variable does is remove the build artifacts from the build directory. But as you say the log.do_compile shows a correct build so both System.map and .config should be available. To double check that the kernel builds properly please do:

MACHINE=nanopi_neo3 bitbake linux-stable -c c leanall
MACHINE=nanopi_neo3 bitbake linux-stable

If the above succeeds we can move pass this.
The second error is about the kernel-modules-headers package. Do I understand correctly that you manually ran a make mrproper? Again, let’s make sure this package builds with:

MACHINE=nanopi_neo3 bitbake  kernel-modules-headers -c c leanall
MACHINE=nanopi_neo3 bitbake kernel-modules-headers

If the kernel headers are not building you can workaround it by adding BALENA_DISABLE_KERNEL_HEADER = "1" you your local.conf but eventually we will have to go back to understand what the problem is.

About the rootfs size, increasing the size if the local.conf as you have done is fine. The default rootfs is 300MB and for the device to be officially supported we would need to look into why that is not enough for this device type. It is likely that there are things in the rootfs that are not required. However, for the time being you can just increase it as you have done.

The last error is:

 File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210307143008.rootfs.docker

And that means that the creation of the hostapp container failed. What happens is that BalenaOS creates a container image from the rootfs and copies that to the image file. This container is generated in meta-balena/image_types_resin.bbclass at 1fb5d77d3eab4bf76e922cdc0a3677cafc48b338 · balena-os/meta-balena · GitHub.

To see what’s going on here you need to look at the log file for the resin-image package. As before, to make sure the log file are not erased add the resin-image package to the list of rm_work excluded packages:

RM_WORK_EXCLUDE += " \
    linux-stable \
    resin-image \
"

Then rebuild:

MACHINE=nanopi_neo3 bitbake  resin-image -c cleanall
MACHINE=nanopi_neo3 bitbake resin-image

Check for the path to the resin-image workdir:

MACHINE=nanopi_neo3 bitbake resin-image -e | grep ^WORKDIR=

And in the ${WORKDIR}/temp you will find a log.do_image_docker log file that might give us some clues.

mmelnick · March 8, 2021, 2:43pm

@alexgg - thanks for being so thorough and generous with your time. I’m going to work on this piece by piece and see what I can find out. I’ll have an update in a bit.

mmelnick · March 8, 2021, 4:11pm

Okay @alexgg, in order of the above:

Hadn’t realized this about RM_WORK_EXCLUDE, but indeed, this is running without problem now. Only output that’s concerning is? WARNING: linux-stable-5.7.17-r0 do_kernel_resin_checkconfig: Checking for CONFIG_NF_TABLES_SET=m in the kernel configs failed for nf_tables.

That’s right - I had run a manual make mrproper. As before, after running -c cleanall and then bitbake kernel-modules-headers I hit the same error as before:

  ERROR: kernel-modules-headers-1.0-r0 do_compile: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.662123' failed with exit code 2:
  Running modules_prepare...
  make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  ***
  *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
  *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
  ***
  make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
  make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
  make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  make: *** [Makefile:180: sub-make] Error 2
  WARNING: exit code 2 from a shell command.

  ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/log.do_compile.662123
  Log data follows:
  | DEBUG: Executing shell function do_compile
  | Running modules_prepare...
  | make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  | ***
  | *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
  | *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
  | ***
  | make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
  | make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
  | make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  | make: *** [Makefile:180: sub-make] Error 2
  | WARNING: exit code 2 from a shell command.
  | ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.662123' failed with exit code 2:
  | Running modules_prepare...
  | make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  | ***
  | *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
  | *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
  | ***
  | make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
  | make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
  | make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
  | make: *** [Makefile:180: sub-make] Error 2
  | WARNING: exit code 2 from a shell command.
  |
  ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-devtools/kernel-modules-headers/kernel-modules-headers.bb:do_compile) failed with exit code '1'
  NOTE: Tasks Summary: Attempted 1756 tasks of which 1748 didn't need to be rerun and 1 failed.

Running the request make mrproper successfully builds the kernel-modules-headers with the exception of this warning:

WARNING: kernel-modules-headers-1.0-r0 do_package: Manifest /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-resin-image-initramfs.packagedata not found in nanopi_neo3 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

Is it preferable to have to deal with this via the manual make mrproper or to use BALENA_DISABLE_KERNEL_HEADER?

My only suspicion about what may be different here with regards to size is this piece from the BSP repository noting how this is based on Armbian:

Also, by default in this image all the Linux firmware files are added in the image. If you want to save ~500MB then you can comment out the next line, but then you need to add the specific firmware for your MACHINE with your own recipe:
IMAGE_INSTALL += “armbian-firmware”
GitHub - dimtass/meta-nanopi-rockchip64

Perhaps not relevant but FYI.

Okay, last piece here, the resin-image build. Here’s what I’ve found:

From a clean (post cleanall) build of resin-image, I get the size errors via mkfs:

| Allocating group tables: done
| Writing inode tables: done
| Creating journal (8192 blocks): done
| Copying files into the device: __populate_fs: Could not allocate block in ext2 filesystem while writing file "Image"
| mkfs.ext4: Could not allocate block in ext2 filesystem while populating file system
| Deleted: sha256:e25e84397823c824d1823092d9c5b1b32326572d299c32dd4ce7bb2bd130d6c1
| Deleted: sha256:b39ac24be79db4c4188f6498a176d79d9e8d37f6c82a0d03e81b801265aa87ae
| Deleted: sha256:3c1c6f550ee188271fd448b2ad4830902d60994b86ae25727ebe1db432560670
| WARNING: exit code 1 from a shell command.

what I thought was fixing it before (adding IMAGE_ROOTFS_MAXSIZE = "500000") apparently wasn’t a solution, it just looked like it if I reran bitbake after this error. (same is true for a higher threshold value). The followup error is the one you referenced as failing during creation of the hostapp container.

Maybe this needs to be fixed at the first error to have success. When I see the second error, for example: File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210308160352.rootfs.docker

if I search for a *.rootfs.docker file I do find one, just not the name it’s looking for:
tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210308160036.rootfs.docker,

I don’t see anything going wrong in the log.do_image_docker either, but I’ll upload here in case you see something I don’t. log.do_image_docker.log (428.8 KB)

I am seeing errors in a separate log that seem more like potenital leads, relevant pieces here and full log attached:

time="2021-03-08T15:19:57.801374230Z" level=warning msg="Your kernel does not support swap memory limit"
time="2021-03-08T15:19:57.801392164Z" level=warning msg="Your kernel does not support cgroup rt period"
time="2021-03-08T15:19:57.801397138Z" level=warning msg="Your kernel does not support cgroup rt runtime"
time="2021-03-08T15:19:57.801401818Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2021-03-08T15:19:57.801406421Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2021-03-08T15:19:57.801560553Z" level=info msg="Loading containers: start."
time="2021-03-08T15:19:57.853110327Z" level=info msg="Default bridge (balena0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2021-03-08T15:19:57.883484527Z" level=info msg="Loading containers: done."
time="2021-03-08T15:19:57.906245671Z" level=warning msg="failed to retrieve balena-engine-init version: exec: \"balena-engine-init\": executable file not found in $PATH"

log.do_image_hostapp_ext4.796325.log (23.2 KB)

Thanks again (and again, and again)

~Michael

alexgg · March 8, 2021, 4:32pm

Hi Michael,

Hadn’t realized this about RM_WORK_EXCLUDE, but indeed, this is running without problem now. Only output that’s concerning is? WARNING: linux-stable-5.7.17-r0 do_kernel_resin_checkconfig: Checking for CONFIG_NF_TABLES_SET=m in the kernel configs failed for nf_tables.

The warning is nothing critical, it just means we have to adjust some of our common kernel class configurations to your kernel.

For the kernel headers error, let’s go for the BALENA_DISABLE_KERNEL_HEADER variable for now. We will have to come back to this later on but kernel headers are not part of the image itself.

alexgg · March 8, 2021, 4:35pm

The armbian-firmware is probably what the increase in size is about. We would have to be specific and add only the required firmware, but that is an optimization that can be left for later. However, the comment says it will add around ~500M to the size, and the failure you see is still about the partition being too small.

Could you try to increase IMAGE_ROOTFS_MAXSIZE to something obscene like 1048576 see if the image builds successfully?

mmelnick · March 8, 2021, 6:02pm

Hey again -

2 pieces:

After adding BALENA_DISABLE_KERNEL_HEADER that runs to completion with the following warning, that I hope is maybe relevant to the later problem:

WARNING: kernel-modules-headers-1.0-r0 do_package: Manifest /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-resin-image-initramfs.packagedata not found in nanopi_neo3 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

I’d been using IMAGE_ROOTFS_SIZE and not IMAGE_ROOTS_MAXSIZE when I’d raised earlier limits. In the spirit of being as obscene as possible, I ran this as:

IMAGE_ROOTFS_SIZE = 100000000
IMAGE_ROOTS_MAXSIZE = 10485760

after earlier failures and still got:

Copying files into the device: __populate_fs: Could not allocate block in ext2 filesystem while writing file "Image"
mkfs.ext4: Could not allocate block in ext2 filesystem while populating file system
Deleted: sha256:e25e84397823c824d1823092d9c5b1b32326572d299c32dd4ce7bb2bd130d6c1
Deleted: sha256:b39ac24be79db4c4188f6498a176d79d9e8d37f6c82a0d03e81b801265aa87ae
Deleted: sha256:3c1c6f550ee188271fd448b2ad4830902d60994b86ae25727ebe1db432560670
WARNING: exit code 1 from a shell command.