CM4 Module not booting with PCIe SATA Card

Product support balenaOS
1

Hi,

we have some trouble with running our CM4 modules on the official IO Board with a PCIe SATA Card.
The SATA Card has an ASM1061 Chipset.

With the actual Raspberry Pi OS and also with Ubuntu Server 22.04 the SATA Card is recognized and the system boots complete.

With the balenaOS 2.94.4+rev1 the boot hangs.
I also tried to copy the start… and fixes… files from the running Raspberry Pi OS boot partition and also copied the bcm2711-rpi-cm4.dtb

I tried also the version 2.88.4-dev with the boot files start…, fixes and the cm4.dtb
But it stucks at “Starting kernel …”

Without the card placed in the PCIe slot it boots normal.

Does anybody have an idea?

Thanks

2

Hi Patrick,

In your ticket you mentioned that there’s one device that runs fine; it’s quite strange that only one device has no issues while others fail. I assume it’s running the exact same software as the ones that don’t boot? Is there anything different about it? (e.g. power supply, cable, environment, plugged in peripherals?). What happens if you re-flash it?
Also, how do you power/boot the CM4 when it’s not in the PCIe slot?

3

Hi builder555,

thank you for your answer.

We use our own board for the CM4 module which is based on the official Raspberry CM4 IO board.

The problem also exists with the official CM4 IO board.
This is supplied with power via a 12V power supply.

The boot process without PCIe SATA Card works fine.
With the PCIe SATA card inserted into the PCIe slot on the CM4 board, the system does not boot anymore.

It had worked in all systems until Thursday. Only since last Friday it does not work anymore.
The system which was already running in test mode, we have not restarted since this problem. Since there is already a software in continuous operation is tested.

Even with a new Fleet with no other software and default configuration, the same problem occurs.

With a Raspberry PI OS or Ubuntu 22.04 on the CM4 module everything works.

4

Do you have access to more logs ?

Maybe have another PCIe card that you can try ? (I’d like to distinguish between a PCIe port issue and a specific issue with that card).

6

Can you also test with a development variant of balenaos (it should produce more logs)?

7

Hi,

I tried to reproduce this and I was just not able to. My setup was:

  • 4GB CM4 lite
  • official CM4 IO board
  • 12V 3A barrel jack PSU
  • a PCIe SATA card with ASM1061 chip exposing 4 SATA ports
  • SanDisk Extreme Pro 32GB SD card
  • balenaOS 2.94.4+rev1 for the “Raspberry Pi CM4 IO Board” device type

I have performed a series of cca 10 reboots and the board just comes up fine every time.

The way you are describing your issue makes it look quite strange - “It had worked in all systems until Thursday. Only since last Friday it does not work anymore.” It is very unlikely that there is a timer somewhere that would make it stop working in a particular moment. Have you maybe performed some changes inbetween that seem unrelated on first sight? Maybe updated the CM4 firmware? Performed an OS upgrade? Plugged in a new peripheral?

Also do you have drives connected to the card? How are they powered? Would the system boot with just the card and no drives plugged in?

In order to debug further, would you be able to collect kernel logs from the serial console? I would suggest flashing the balenaOS 2.94.4+rev1 development variant, booting it up without the card attached, enabling UART through the “Device configuration” dashboard tab, changing console=null to console=serial0,115200 in /mnt/boot/cmdline.txt, rebooting and confirming you can see the kernel logs. If that is the case, try booting with the card plugged in and look for differences or error messages in the logs.

9

And one more thing, for the testing, please try to keep the files in the boot partition (start*, fixup* and DTB) as they are in the balenaOS image.

11

Thank you very much for your feedback.

I was able to find out a few more things in the meantime.

In the system that ran without problems so far, a CM4 module Rev 1.0 was installed.
Here was originally also the BalenaOS 2.88.4+rev0 flashed.

This was later updated to version 2.94.4+rev1 via the dashboard.

For the other systems that had problems, we reflashed the integrated memory on Friday with the actual 2.94.4+rev1 version. The old 2.88.4+rev0 is not running an this modules because they are on Rev 1.1
Interestingly, the modules now boot up again but without recognizing the SATA card.
Why it didn’t work at times is still a mystery.

We were able to work around the problem by using other SATA cards that also work with BalenaOS.
We now use cards with the ASM1064 chipset.

With the use of the Rasperry Pi OS and Ubuntu there were no problems.

Unfortunately I don’t have the time to explore this further.