I have two Raspberry 4s running balenaOS 2.69.1+rev1 + supervisor 12.3.5.
I’ve been having a lot of issues with SD cards in this project, and have switched over to SanDisk Extreme Pro (64gb, 32gb) cards, but the issues persist. Using the official power supply with both devices.
One of the raspberries barely comes online, and produces the green led 4 flashes four times error, with a brand new card. In the past, several other cards eventually produced the same result. I have no access to this device, it’s at a friend’s house, we tried reflashing and it only works for half an hour or so at a time.
I’ve had the other raspberry produce the same result with most cards in the past, and I recently had a corrupted file system with a new SanDisk card after less than a month of use (it was pretty hard to tell, because the logs just flashed on screen for a second or so before the device rebooted). Repairing the filesystem on another device made it work again.
I’ve been writing these issues off as being caused by cheaper SD cards and power sources, but that doesn’t seem to be the case anymore, upgrading those components more or less gives me the same results. What logs can I look at to figure out what’s up? What checks can I run to eliminate hardware issues? What am I dealing with?
Caught the misbehaving one online, the health checks run fine:
check_balenaOS Succeeded Supported balenaOS 2.x detected
check_container_engine Succeeded No container_engine issues detected
check_localdisk Succeeded No localdisk issues detected
check_memory Succeeded 80% memory available
check_networking Succeeded No networking issues detected
check_os_rollback Succeeded No OS rollbacks detected
check_service_restarts Succeeded No services are restarting unexpectedly
check_supervisor Succeeded Supervisor is running & healthy
check_temperature Succeeded No temperature issues detected
check_timesync Succeeded Time is synchronized
check_under_voltage Succeeded No under-voltage events detected
Checks version: 4.20.24
But the diag seems to fail. It gives me a whooping 1.5 megabytes of output, that probably contains the story of my entire life and the addresses of everyone I have met but here it goes.
The other one also fails, but output is different:
I have some issues with the internet connection this one is using, so the check_networking sometimes fails
Jokes aside, if these contain some information that shouldn’t go public (keys, creds, …) let me know, so I can act accrodingly
Hi kunakos
Thanks for sharing the logs. Some answers here:
The only private data these logs contain re regarding IP addresses, containers, volumes… There are no passwords or certificates.
There are differences in the logs, but they don’t seem to be related to the issue. Maybe there’s some hardware differences that are causing different logs?
I don’t see any issue related to the memory cards. But I do see problems connecting to balena’s VPN. This is causing device reboots.
I’m going to investigate if this is related with the boot process and the lack of RTC in the RPi, but I’m not sure at this moment.
Keep tuned.
Hi @kunakos
My colleague did check into the timesync / RTC possibility and we confirmed that, based on the logs you’ve provided, it should not be related to the issue you’re experiencing.
However, since the four blinks indicates no boot code is found, I was wondering if you could describe the process you use for flashing?
What tool do you use for flashing your SD cards?
What partitioning scheme do use?
Anything else about the process you think might be valuable to share…
I’d be curious if that process leads us or other forum users to think of something else for you to look into.
but here it goes.
