Hi all
I’m currently porting balenaOS to a new device and so far everything builds properly. U-Boot worked with very minor changes but I am having issues getting through the kernel boot.
The specific issue I have occurs after the switch_root from the finish script here. After that point I believe the Mobyinit is what should be run as per this doc, but there is almost immediately a kernel panic. The error shown is:
[init][INFO] Switching root to '/rootfs'... [ 671.690614] EXT4-fs (mmcblk2p2): re-mounted. Opts: (null) ERRO[0000] mkdir /balena/overlay2/check-overlayfs-support679356590: no space left on device storage-driver=overlay2 1970/01/01 00:11:11 error loading layer store:error initializing graphdriver: driver not supported
I think the issue is with the re-mount of mmcblk2p2 (resin-rootA) while resin-state is not re-mounted at that point. I dropped to the initramfs shell using the shell kernel bootarg to confirm that partitions were mounted by the initrdscripts, and resin-rootA and resin-state were both mounted correctly. Before the finish script is run, the output of mount is:
rootfs on / type rootfs (rw) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) devtmpfs on /dev type devtmpfs (rw,relatime,size=1397848k,nr_inodes=349462,mode=755) tmpfs on /rootfs/tmp type tmpfs (rw,nosuid,nodev,mode=755) /dev/disk/by-state/resin-rootA on /rootfs type ext4 (ro,relatime,data=ordered) /dev/disk/by-state/resin-state on /rootfs/mnt/state type ext4 (rw,relatime,data=ordered) /dev/disk/by-state/resin-state on /rootfs/etc/machine-id type ext4 (rw,relatime,data=ordered)
I do not see anything here that is particularly unusual - but the kernel still remounts resin-rootA after the finish script (specifically the switch_root) is executed.
More specifically, the command that seems to fail is the SupportsOverlay test in balena-engine here: https://github.com/balena-os/balena-engine/blob/master/daemon/graphdriver/overlayutils/overlayutils.go#L39. Creating the temp directory doesn’t seem to be possible as it is doing so on the read-only filesystem resin-rootA (which is remounted after switch_root). It isn’t possible to debug past this due to a kernel panic directly after the error loading layer store:error initializing graphdriver message.
Is this something that could be modified (in this case, simply removing the creation of the tempdir?) to fix the issue?
For more info - I am running the kernel passing root=LABEL=resin-rootA as an argument instead of UUID. Using UUID caused other issues with creating the by-state symlinks (not sure why, did not investigate this too much). This caused the rootfs shell script to not find the resin-rootA partition by its old UUID or the /dev/disk/by-state/active link (since it didn’t exist)
Any help is much appreciated