Compulab IoT DIN Boot Failure

Product support balenaOS
1

Hello,

After a successful flash of Balena OS to a Compulab IOT DIN 8GB device, the device fails to boot. I have tried both with an image downloaded from Balena Hub and from the Balena CLI. Details:

  • Balena OS: 6.5.53+rev23
  • Device: Compulab IOTDIN-iMX8P 1G-8G DRAM
  • Hardware Device: IOTDIN-IMX8P-C1800QM-D8-N64-WB-JEG25G-XL

Boot logs:

pca9450@25 [ldo4][u] = 1v8
DDRINFO: EEPROM VALID DATA [ [ cafecafe ] = ff070018 8 
DDRINFO(D): Micron 8192MB @ 4000 MHz
DDRINFO: start DRAM init
DDRINFO: DRAM rate 4000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
DDRINFO(M): mr5-8 [ 0xff070018 ]
DDRINFO(E): mr5-8 [ 0xff070018 ]
SEC0:  RNG instantiated
Trying to boot from BOOTROM
Boot Stage: Primary boot
image offset 0x0, pagesize 0x200, ivt offset 0x0
NOTICE:  BL31: v2.6(release):lf-5.15.71-2.2.2-0-gf4540f956
NOTICE:  BL31: Built : 06:10:48, Apr 13 2023

[    0.181440] imx-drm display-subsystem: no available port
[    0.320560] rtc-abx80x 1-0069: Create sysfs group: 0
[    0.333299] pca953x 4-0020: failed writing register
[    0.339120] pca953x 4-0021: failed writing register
[    0.344885] pca953x 4-0022: failed writing register
[    0.350637] pca953x 4-0023: failed writing register
Starting version 250.5+
[   13.720355] fsl-aud2htx 30cb0000.aud2htx: failed to pcm register
[   13.732519] imx-bus-devfreq 32700000.interconnect: failed to fetch clk: -2
[   13.740642] fsl-aud2htx 30cb0000.aud2htx: failed to pcm register
[   13.808466] fsl-aud2htx 30cb0000.aud2htx: failed to pcm register
[   13.826666] fsl-aud2htx 30cb0000.aud2htx: failed to pcm register
[   13.844361] fsl-aud2htx 30cb0000.aud2htx: failed to pcm register
[   14.010826] tpm tpm0: A TPM error (256) occurred attempting the self test
[   14.077167] Bluetooth: request_firmware(firmware) failed, error code = -2
[   14.091742] Bluetooth: Failed to download firmware!
[   14.099582] Bluetooth: Downloading firmware failed!
[   14.608621] caam_jr 30903000.jr: failed to flush job ring 1

UUU Logs:

[INFO] Provisioning process will start now.
[INFO] Will build flash container for aarch64...
[+] Building 1.0s (12/12) FINISHED                                                                                                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                 0.0s
 => => transferring dockerfile: 738B                                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/balenalib/aarch64-ubuntu:focal-run-20221215                                                                                                               0.9s
 => [internal] load .dockerignore                                                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                                      0.0s
 => [1/7] FROM docker.io/balenalib/aarch64-ubuntu:focal-run-20221215@sha256:d8e3e3e4e9aaaa845463c8980ad5b4950204c166d555d50245214d4ca3fa51be                                                         0.0s
 => [internal] load build context                                                                                                                                                                    0.0s
 => => transferring context: 348B                                                                                                                                                                    0.0s
 => CACHED [2/7] WORKDIR /usr/src/app/                                                                                                                                                               0.0s
 => CACHED [3/7] RUN     apt-get update && apt-get install -y libusb-1.0-0-dev libbz2-dev libzstd-dev libtinyxml2-dev pkg-config cmake libssl-dev g++ zlib1g-dev git usbutils file &&     git clone  0.0s
 => CACHED [4/7] COPY ./container/flash_iot.sh /usr/src/app/                                                                                                                                         0.0s
 => CACHED [5/7] COPY ./container/helpers /usr/src/app/                                                                                                                                              0.0s
 => CACHED [6/7] COPY ./README.md /usr/src/app/                                                                                                                                                      0.0s
 => CACHED [7/7] COPY ./container/imx-boot /usr/src/app/imx-boot                                                                                                                                     0.0s
 => exporting to image                                                                                                                                                                               0.0s
 => => exporting layers                                                                                                                                                                              0.0s
 => => writing image sha256:ebc9004b2f4df3ca284c7e5fd1615d0590860bcb1c5388fc09e22b2011dd51ac                                                                                                         0.0s
 => => naming to docker.io/library/uuu-image                                                                                                                                                         0.0s

 1 warning found (use docker --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 4)
[INFO] imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk has been extracted
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.201-10-g311ee9b

Success 1    Failure 0                                                  
                                                                         
                                                                         
2:13-1417100 2/ 2 [Done                                  ] SDPS: done                                                               


[INFO] imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk has been loaded... Waiting for a couple seconds to allow fastboot to run...
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.201-10-g311ee9b

Build in config:
	Pctl	 Chip		 Vid	 Pid	 BcdVersion	 Serial_No
	==================================================
	SDPS:	 MX8QXP		 0x1fc9	 0x012f	 [0x0002..0xffff]
	SDPS:	 MX8QM		 0x1fc9	 0x0129	 [0x0002..0xffff]
	SDPS:	 MX8DXL		 0x1fc9	 0x0147
	SDPS:	 MX28		 0x15a2	 0x004f
	SDPS:	 MX815		 0x1fc9	 0x013e
	SDPS:	 MX865		 0x1fc9	 0x0146
	SDPS:	 MX8ULP		 0x1fc9	 0x014a
	SDPS:	 MX8ULP		 0x1fc9	 0x014b
	SDPS:	 MX93		 0x1fc9	 0x014e
	SDPS:	 MX91		 0x1fc9	 0x0159
	SDPS:	 MX95		 0x1fc9	 0x015d
	SDPS:	 MX95		 0x1fc9	 0x015c
	SDPS:	 MX943		 0x1fc9	 0x0027
	SDP:	 MX7D		 0x15a2	 0x0076
	SDP:	 MX6Q		 0x15a2	 0x0054
	SDP:	 MX6D		 0x15a2	 0x0061
	SDP:	 MX6SL		 0x15a2	 0x0063
	SDP:	 MX6SX		 0x15a2	 0x0071
	SDP:	 MX6UL		 0x15a2	 0x007d
	SDP:	 MX6ULL		 0x15a2	 0x0080
	SDP:	 MX6SLL		 0x1fc9	 0x0128
	SDP:	 MX7ULP		 0x1fc9	 0x0126
	SDP:	 MXRT106X	 0x1fc9	 0x0135
	SDP:	 MX8MM		 0x1fc9	 0x0134
	SDP:	 MX8MQ		 0x1fc9	 0x012b
	SDPU:	 SPL		 0x0525	 0xb4a4	 [0x0000..0x04ff]
	SDPV:	 SPL1		 0x0525	 0xb4a4	 [0x0500..0x9998]
	SDPV:	 SPL1		 0x1fc9	 0x0151	 [0x0500..0x9998]
	SDPU:	 SPL		 0x0525	 0xb4a4	 [0x9999..0x9999]
	SDPU:	 SPL		 0x3016	 0x1001	 [0x0000..0x04ff]
	SDPV:	 SPL1		 0x3016	 0x1001	 [0x0500..0x9998]
	FBK:	 		 0x066f	 0x9afe
	FBK:	 		 0x066f	 0x9bff
	FBK:	 		 0x1fc9	 0x0153
	FB:	 		 0x0525	 0xa4a5
	FB:	 		 0x18d1	 0x0d02
	FB:	 		 0x3016	 0x0001
	FB:	 		 0x1fc9	 0x0152
	FB:	 		 0x0483	 0x0afb

Run built-in script:
 
uuu_version 1.4.149

# @_flash.bin            | bootloader, which can extract from wic image
# @_image   [_flash.bin] | wic image burn to emmc.


# This command will be run when i.MX6/7 i.MX8MM, i.MX8MQ
SDP: boot -f /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk -scanlimited 0x800000

# This command will be run when ROM support stream mode
# i.MX8QXP, i.MX8QM
SDPS: boot -scanterm -f /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk -scanlimited 0x800000

# These commands will be run when use SPL and will be skipped if no spl
# SDPU will be deprecated. please use SDPV instead of SDPU
# {
SDPU: delay 1000
SDPU: write -f /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk -offset 0x57c00
SDPU: jump -scanlimited 0x800000
# }

# These commands will be run when use SPL and will be skipped if no spl
# if (SPL support SDPV)
# {
SDPV: delay 1000
SDPV: write -f /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk -skipspl -scanterm -scanlimited 0x800000
SDPV: jump -scanlimited 0x800000
# }


FB: ucmd setenv fastboot_dev mmc
FB: ucmd setenv mmcdev ${emmc_dev}
FB: ucmd mmc dev ${emmc_dev}
FB: flash -raw2sparse all /usr/src/app/balena-cloud-test-iotdin-imx8p-d1d8-6.5.53+rev23-v17.0.3.img
FB: flash -scanterm -scanlimited 0x800000 bootloader /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk
FB: ucmd if env exists emmc_ack; then ; else setenv emmc_ack 0; fi;
FB: ucmd mmc partconf ${emmc_dev} ${emmc_ack} 1 0
FB: done


Wait for Known USB Device Appear...
New USB Device Attached at 2:13-141710005B7E2347
2:13-141710005B7E2347>Start Cmd:FB: ucmd setenv fastboot_dev mmc
2:13-141710005B7E2347>Okay (0.001s)
2:13-141710005B7E2347>Start Cmd:FB: ucmd setenv mmcdev ${emmc_dev}
2:13-141710005B7E2347>Okay (0.066s)
2:13-141710005B7E2347>Start Cmd:FB: ucmd mmc dev ${emmc_dev}
2:13-141710005B7E2347>Okay (0.029s)
2:13-141710005B7E2347>Start Cmd:FB: flash -raw2sparse all /usr/src/app/balena-cloud-test-iotdin-imx8p-d1d8-6.5.53+rev23-v17.0.3.img
100%2:13-141710005B7E2347>Okay (57.83s)
2:13-141710005B7E2347>Start Cmd:FB: flash -scanterm -scanlimited 0x800000 bootloader /usr/src/app//imx-boot-iotdin-imx8p-d1d8-sd.bin-flash_evk
0x400000002:13-141710005B7E2347>Okay (0.137s)
2:13-141710005B7E2347>Start Cmd:FB: ucmd if env exists emmc_ack; then ; else setenv emmc_ack 0; fi;
2:13-141710005B7E2347>Okay (0.001s)
2:13-141710005B7E2347>Start Cmd:FB: ucmd mmc partconf ${emmc_dev} ${emmc_ack} 1 0
2:13-141710005B7E2347>Okay (0.001s)
2:13-141710005B7E2347>Start Cmd:FB: done
2:13-141710005B7E2347>Okay (0s)



[INFO] Finished writing balenaOS image!
[INFO] Please remove programming cable from the PC, power off the board and then power it back on.
[INFO] Cleanup complete

Looks like the wrong device tree. However, I don’t believe I have the wrong OS image.

Any suggestions for further debugging steps or alternate images to try?

Thanks!

2

Also tried with latest OS (6.10.15+rev2), same result

3

Hi, looking at the boot log you pasted above it seems the device does boot. It won’t start getty so there won’t be any login prompt over the serial console if that is what you are expecting.

Does the device not appear as online in the dashboard?

Alternatively you can enable development mode in the config.json of the image before flashing it. See Configuration | balena about how the option is to be used. This change needs to be done in the config.json which you can find if you mount the first partition of the OS image file. See Configuration | balena #about-configjson about info on config.json

To access the config.json you would “sudo losetup --show -fP <OS_image>“ followed by mounting the created /dev/loopXp1 from the previous command. Then make the config.json changes and then unmount the loop partition, then “losetup -d /dev/loopX“ and then flash the image again.

4

Hi, thanks for the confirmation.

Yes the device never shows up in the dashboard. Flashing the development version on the device, I can access the device, as expected. However, the I don’t have network access. nmcli states “Error: No Wi-Fi device found“ when trying to connect. dmesg shows:

root@6888439:~# dmesg | grep -A 5 -B 5 -i wifi
[   13.390694] btmrvl_sdio mmc1:0001:2: Direct firmware load for mrvl/sdsd8997_combo_v4.bin failed with error -2
[   13.390720] Bluetooth: request_firmware(firmware) failed, error code = -2
[   13.416493] Bluetooth: Failed to download firmware!
[   13.422037] Bluetooth: Downloading firmware failed!
[   13.459754] zram: Can't change algorithm for initialized device
[   13.462409] mwifiex_sdio mmc1:0001:1: Direct firmware load for mrvl/sdsd8997_combo_v4.bin failed with error -2
[   13.462431] mwifiex_sdio mmc1:0001:1: Failed to get firmware mrvl/sdsd8997_combo_v4.bin
[   13.462436] mwifiex_sdio mmc1:0001:1: info: _mwifiex_fw_dpc: unregister device
[   13.476433] usbcore: registered new interface driver option
[   13.478641] usbserial: USB Serial support registered for GSM modem (1-port)
[   13.478938] option 3-1.4:1.0: GSM modem (1-port) converter detected
[   13.497542] usbcore: registered new interface driver cdc_wdm
[   13.508360] usb 3-1.4: GSM modem (1-port) converter now attached to ttyUSB0

Is the firmware for the NXP 88W8997 wifi chipset not in the OS? Any further troubleshooting steps?

Thanks

5

Hi,

is this the wifi chipset that gets sold with the board?

On our side we don’t have a wifi chipset on the hw we use to validate the OS releases with so I am asking to see which is the typical chipset used by this board so that we can add it and validate wifi on it

6

Yes, that is the chipset used. See: https://www.compulab.com/products/iot-gateways/iot-din-imx8plus-industrial-iot-gateway/#specs (also see dmesg above).

This specific “DIN” model differs from most of their offerings, which use Intel chipsets.

Is there a rough ETA for a solution on Balena’s side? Would there be any considerations if I were to add the firmware to the image myself? I’ll try that today.

Thanks!

7

Hi,

while we can add the module ourselves I would rather ask the vendor to add the missing modules in their BSP which we can update in our repo.

In the meantime, for testing you could just add the missing firmware yourself and if the driver is present then you should have a working device.

Let me know if you need guidance on how to do that or if it works in case you managed to add it.

8

Hi,

I added the firmware to the device and it’s working. Looking for guidance on how to add it to the image to be able to re-use it. Thanks!

9

Hi Josh,

this is something we are currently looking at to have it added by default in future versions.

In the meantime you can keep an eye out for it in the Changelog balena-iotdin-imx8p/CHANGELOG.md at master · balena-os/balena-iotdin-imx8p · GitHub

10

This has been resolved with the latest Balena version, “balenaOS 6.10.20+rev1”. The NXP wifi chipset firmware is now included for the Compulab IOTDIN-iMX8P 1G-8G DRAM, .