Building BalenaOS for Rockchip 3328 / NanoPi Neo3

Okay @alexgg, in order of the above:

  • Hadn’t realized this about RM_WORK_EXCLUDE, but indeed, this is running without problem now. Only output that’s concerning is? WARNING: linux-stable-5.7.17-r0 do_kernel_resin_checkconfig: Checking for CONFIG_NF_TABLES_SET=m in the kernel configs failed for nf_tables.

  • That’s right - I had run a manual make mrproper. As before, after running -c cleanall and then bitbake kernel-modules-headers I hit the same error as before:

      ERROR: kernel-modules-headers-1.0-r0 do_compile: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.662123' failed with exit code 2:
      Running modules_prepare...
      make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      ***
      *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
      *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
      ***
      make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
      make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
      make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      make: *** [Makefile:180: sub-make] Error 2
      WARNING: exit code 2 from a shell command.
    
      ERROR: Logfile of failure stored in: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/log.do_compile.662123
      Log data follows:
      | DEBUG: Executing shell function do_compile
      | Running modules_prepare...
      | make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      | ***
      | *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
      | *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
      | ***
      | make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
      | make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
      | make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      | make: *** [Makefile:180: sub-make] Error 2
      | WARNING: exit code 2 from a shell command.
      | ERROR: Execution of '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/temp/run.do_compile.662123' failed with exit code 2:
      | Running modules_prepare...
      | make[1]: Entering directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      | ***
      | *** The source tree is not clean, please run 'make ARCH=arm64 mrproper'
      | *** in /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source
      | ***
      | make[2]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:533: outputmakefile] Error 1
      | make[1]: *** [/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/Makefile:695: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work-shared/nanopi-neo3/kernel-source/include/config/auto.conf.cmd] Error 2
      | make[1]: Leaving directory '/home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/kernel-modules-headers/1.0-r0/kernel_modules_headers'
      | make: *** [Makefile:180: sub-make] Error 2
      | WARNING: exit code 2 from a shell command.
      |
      ERROR: Task (/home/ubuntu/development/yocto/balena-nanopineo3/build/../layers/meta-balena/meta-balena-common/recipes-devtools/kernel-modules-headers/kernel-modules-headers.bb:do_compile) failed with exit code '1'
      NOTE: Tasks Summary: Attempted 1756 tasks of which 1748 didn't need to be rerun and 1 failed.
    

Running the request make mrproper successfully builds the kernel-modules-headers with the exception of this warning:

WARNING: kernel-modules-headers-1.0-r0 do_package: Manifest /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-resin-image-initramfs.packagedata not found in nanopi_neo3 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

Is it preferable to have to deal with this via the manual make mrproper or to use BALENA_DISABLE_KERNEL_HEADER?

  • My only suspicion about what may be different here with regards to size is this piece from the BSP repository noting how this is based on Armbian:

    Also, by default in this image all the Linux firmware files are added in the image. If you want to save ~500MB then you can comment out the next line, but then you need to add the specific firmware for your MACHINE with your own recipe:
    IMAGE_INSTALL += “armbian-firmware”
    GitHub - dimtass/meta-nanopi-rockchip64

Perhaps not relevant but FYI.

Okay, last piece here, the resin-image build. Here’s what I’ve found:

From a clean (post cleanall) build of resin-image, I get the size errors via mkfs:

| Allocating group tables: done
| Writing inode tables: done
| Creating journal (8192 blocks): done
| Copying files into the device: __populate_fs: Could not allocate block in ext2 filesystem while writing file "Image"
| mkfs.ext4: Could not allocate block in ext2 filesystem while populating file system
| Deleted: sha256:e25e84397823c824d1823092d9c5b1b32326572d299c32dd4ce7bb2bd130d6c1
| Deleted: sha256:b39ac24be79db4c4188f6498a176d79d9e8d37f6c82a0d03e81b801265aa87ae
| Deleted: sha256:3c1c6f550ee188271fd448b2ad4830902d60994b86ae25727ebe1db432560670
| WARNING: exit code 1 from a shell command.

what I thought was fixing it before (adding IMAGE_ROOTFS_MAXSIZE = "500000") apparently wasn’t a solution, it just looked like it if I reran bitbake after this error. (same is true for a higher threshold value). The followup error is the one you referenced as failing during creation of the hostapp container.

Maybe this needs to be fixed at the first error to have success. When I see the second error, for example: File does not exist: /home/ubuntu/development/yocto/balena-nanopineo3/build/tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210308160352.rootfs.docker

if I search for a *.rootfs.docker file I do find one, just not the name it’s looking for:
tmp/work/nanopi_neo3-poky-linux/resin-image/1.0-r0/deploy-resin-image-image-complete/resin-image-nanopi-neo3-20210308160036.rootfs.docker,

I don’t see anything going wrong in the log.do_image_docker either, but I’ll upload here in case you see something I don’t. log.do_image_docker.log (428.8 KB)

I am seeing errors in a separate log that seem more like potenital leads, relevant pieces here and full log attached:

time="2021-03-08T15:19:57.801374230Z" level=warning msg="Your kernel does not support swap memory limit"
time="2021-03-08T15:19:57.801392164Z" level=warning msg="Your kernel does not support cgroup rt period"
time="2021-03-08T15:19:57.801397138Z" level=warning msg="Your kernel does not support cgroup rt runtime"
time="2021-03-08T15:19:57.801401818Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2021-03-08T15:19:57.801406421Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2021-03-08T15:19:57.801560553Z" level=info msg="Loading containers: start."
time="2021-03-08T15:19:57.853110327Z" level=info msg="Default bridge (balena0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2021-03-08T15:19:57.883484527Z" level=info msg="Loading containers: done."
time="2021-03-08T15:19:57.906245671Z" level=warning msg="failed to retrieve balena-engine-init version: exec: \"balena-engine-init\": executable file not found in $PATH"

log.do_image_hostapp_ext4.796325.log (23.2 KB)

Thanks again (and again, and again)

~Michael