balenaOS compulab kernel not booting on IOT-GATE-IMX8

Hi, I’m trying to boot balena OS kernel (balena-compulab v2.51.1+rev5) from USB but it stops booting with the following log:

IOT-GATE-iMX8 => run usb_ul
Trying to boot from USB …
resetting USB…
Bus usb@32e40000: USB EHCI 1.00
Bus usb@32e50000: USB EHCI 1.00
scanning bus usb@32e40000 for devices… 2 USB Device(s) found
scanning bus usb@32e50000 for devices… 5 USB Device(s) found
scanning usb for storage devices… 1 Storage Device(s) found
IOT-GATE-iMX8 => setenv image Image.gz
IOT-GATE-iMX8 => setenv zip_addr 0x42480000
IOT-GATE-iMX8 => setenv loadimage “load ${iface} ${dev}:${part} ${zip_addr} ${image}; unzip ${zip_addr} ${loadaddr};”
IOT-GATE-iMX8 => setenv loadfdt “load ${iface} ${dev}:${part} ${fdt_addr} sb-iotgimx8-can.dtb”
IOT-GATE-iMX8 => setenv bootcmd “run loadimage; run loadfdt; booti ${loadaddr} - ${fdt_addr};”
IOT-GATE-iMX8 => printenv
autoload=off
baudrate=115200
boot_dev_str=USB
boot_try_msg=echo Trying to boot from ${boot_dev_str} …;
bootargs=console=ttymxc2,115200 earlycon=ec_imx6q,0x30880000,115200 net.ifnames=0 root=/dev/sda2 rootwait rw
bootargs_common=console=ttymxc2,115200 earlycon=ec_imx6q,0x30880000,115200 net.ifnames=0
bootcmd=run loadimage; run loadfdt; booti 0x40480000 - 0x43000000;
bootcmd_mfg=run mfgtool_args; if iminfo ${initrd_addr}; then booti ${loadaddr} ${initrd_addr} ${fdt_addr};else echo “Run fastboot …”; fastboot 0; fi
bootdelay=2
bootdev=2
bootscript=echo Running bootscript from ${boot_dev_str} …; source;
dev=0
emmc_dev=1
emmc_ul=setenv boot_dev_str eMMC; run boot_try_msg; setenv iface mmc; setenv dev 2; setenv part 1;setenv bootargs ${bootargs_common} root=/dev/mmcblk2p2 ${root_opt};
eth1addr=00:01:c0:2d:f3:a7
ethaddr=00:01:c0:2e:0c:b9
ethprime=FEC
fdt_addr=0x43000000
fdt_file=sb-iotgimx8-ied.dtb
fdt_high=0xffffffffffffffff
fdtcontroladdr=fe93bee0
iface=usb
image=Image.gz
initrd_addr=0x43800000
initrd_high=0xffffffffffffffff
loadaddr=0x40480000
loadfdt=load usb 0:1 0x43000000 sb-iotgimx8-can.dtb
loadimage=load usb 0:1 0x42480000 Image.gz; unzip 0x42480000 0x40480000;
mfgtool_args=setenv bootargs console=${console},${baudrate} rdinit=/linuxrc g_mass_storage.stall=0 g_mass_storage.removable=1 g_mass_storage.idVendor=0x066F g_mass_storage.idProduct=0x37FF g_mass_storage.iSerialNumber="" clk_ignore_unused
part=1
root_opt=rootwait rw
script=boot.scr
sd_dev=0
serial#=1211005-00220
serialpn=IOT-GATE-iMX8-D4-NA32-WB-JS7600G-XL-TIC-FARS4-FBCAN-FCDIO-V100-211024-0001c02df3a7
ulbootscript=load ${iface} ${dev}:${part} ${loadaddr} ${script};
ulfdt=load ${iface} ${dev}:${part} ${fdt_addr} ${fdt_file};
ulimage=load ${iface} ${dev}:${part} ${loadaddr} ${image}
usb_ul=setenv boot_dev_str USB; run boot_try_msg; usb reset; setenv iface usb; setenv dev 0; setenv part 1; setenv bootargs ${bootargs_common} root=/dev/sda2 ${root_opt};
zip_addr=0x42480000

Environment size: 1968/4092 bytes
IOT-GATE-iMX8 => run bootcmd
13335791 bytes read in 738 ms (17.2 MiB/s)
Uncompressed size: 29628928 = 0x1C41A00
40893 bytes read in 4 ms (9.7 MiB/s)

Flattened Device Tree blob at 43000000

Booting using the fdt blob at 0x43000000
Using Device Tree in place at 0000000043000000, end 000000004300cfbc
Found /vpu_g1@38300000 node
Modify /vpu_g1@38300000:status disabled
Found /vpu_g2@38310000 node
Modify /vpu_g2@38310000:status disabled
Found /vpu_h1@38320000 node
Modify /vpu_h1@38320000:status disabled

Starting kernel …

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.98-cl-som-imx8-1.1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Oct 25 19:46:57 UTC 2021
[ 0.000000] Boot CPU: AArch64 Processor [410fd034]
[ 0.000000] Machine model: CompuLab SB-IOTGIMX8 board + CAN bus extension
[ 0.000000] earlycon: ec_imx6q0 at MMIO 0x0000000030880000 (options ‘115200’)
[ 0.000000] bootconsole [ec_imx6q0] enabled
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] Reserved memory: created CMA memory pool at 0x0000000078000000, size 640 MiB
[ 0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x13ffbd500-0x13ffbf2ff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000040000000-0x000000013fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000013fffffff]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.1
[ 0.000000] random: get_random_bytes called from start_kernel+0xa8/0x410 with crng_init=0
[ 0.000000] percpu: Embedded 21 pages/cpu @ffff8000fff64000 s47640 r8192 d30184 u86016
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for ARM erratum 845719
[ 0.000000] Speculative Store Bypass Disable mitigation not required
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1032192
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: console=ttymxc2,115200 earlycon=ec_imx6q,0x30880000,115200 net.ifnames=0 root=/dev/sda2 rootwait rw
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
[ 0.000000] log_buf_len min size: 16384 bytes
[ 0.000000] log_buf_len: 32768 bytes
[ 0.000000] early log buf free: 13480(82%)
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] software IO TLB: mapped [mem 0xfbfff000-0xfffff000] (64MB)
[ 0.000000] Memory: 3377448K/4194304K available (12092K kernel code, 1158K rwdata, 3512K rodata, 12096K init, 306K bss, 161496K reserved, 655360K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB)
[ 0.000000] .text : 0xffff000008080000 - 0xffff000008c50000 ( 12096 KB)
[ 0.000000] .rodata : 0xffff000008c50000 - 0xffff000008fd0000 ( 3584 KB)
[ 0.000000] .init : 0xffff000008fd0000 - 0xffff000009ba0000 ( 12096 KB)
[ 0.000000] .data : 0xffff000009ba0000 - 0xffff000009cc1a00 ( 1159 KB)
[ 0.000000] .bss : 0xffff000009cc1a00 - 0xffff000009d0e570 ( 307 KB)
[ 0.000000] fixed : 0xffff7dfffe7fb000 - 0xffff7dfffec00000 ( 4116 KB)
[ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB)
[ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum)
[ 0.000000] 0xffff7e0000000000 - 0xffff7e0004000000 ( 64 MB actual)
[ 0.000000] memory : 0xffff800000000000 - 0xffff800100000000 ( 4096 MB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: no VLPI support, no direct LPI support
[ 0.000000] ITS: No ITS available, not enabling LPIs
[ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000038880000
[ 0.000000] clk: failed to reparent enet_axi_cg to sys_pll1_800m: -22

Hi @Federicod, is this log from trying to bood the stock flash image from the Balena Cloud?

As as I can see here, the device tree on the USB drive is for the “SB-IOTGIMX8 board + CAN bus extension”, and I guess “SB” refers to the Single-Board Computer instead, may this be the cause?

The kernel version 4.14.98-cl-som-imx8-1.1 seems to refer to the cl-som, which would be using the same SoC, a differend SOM it seems: CL-SOM-iMX8 | NXP i.MX 8 | System-on-Module | Computer-on-Module | CompuLab

Or can the CAN bus extension being present or not make a difference here? I guess not, as it stops quite early during boot of the SOM.

Is the device booting other Images like an original Yocto or Debian Image from USB successfully?

Another thing: As you show in the log which I quoted, you are trying to boot some kernel 4.14.98-cl-som-imx8-1.1 which is pretty old. This looks like an old BSP kernel from Compulab and not like the current Kernel (linux-compulab-5.4) which gets built when I built when I run a build of this repo myself:

BTW: @alexgg: What is the difference to this repo, it looks similar? GitHub - balena-os/balena-compulab

My bad, the repo I was using refers to the compulab imx8 som, the one you linked refers to the full iot gateway which is my case. The later one boots correctly with kernel 5.4.

Thanks