Hi my name is Tim, I’ve recently joined a project with a fleet of BalenaFin boards. I sent this to store@balena.io as well, but no response yet.
We have about 14 devices so far, 6 up and running, and looking to expand that to more, but we’ve had 6 have had the same failure, and two out in the field we suspect have died the same way. We’re running out of working boards! Here’s what happens:
Unit goes dead and won’t come online.
Plugging the unit into USB on my mac gives a ’need to format disk’ error.
Balena etcher says the disk is only 32.1MB and isn’t enough space to install an image. Should be 8GB.
5 of the units are v1.1.0 units, but we have had 1 v1.1.1 fail as well.
We are powering the boards with a Lithium Iron Phosphate 12v 7.5Ah battery. I checked the voltage range is in range, and it should well be.
There’s a few mentions of this in the forums, maybe power related, but without much more information.
Does anyone have more information on why these are failing? We obviously need to stop loosing them!
If I understand correctly, your 6 of 14 devices are not showing up on dashboard? But are able to be re-flashed?
Also, are there any logs you are able to share (maybe via connecting to device locally). Apologies for the delay via store@balena.io; I have also dropped a note to our required teams to help you on the issue.
Hi @nitish, no it’s not the dashboard that’s the problem, it’s that we have 5 devices that can’t be flashed. Balena etcher says the disk is only 32.1MB and isn’t enough space to install an image. (sorry not 6, one was found to be a different issue thankfully)
We have seen this distinctive 32.1MB issue with the Fin on some rare occasions. It sounds like you’ve been extremely unlucky to be able to trigger it 5 times! It’s an issue within the eMMC flash storage itself and happens to SD Cards by the same manufacturer (SanDisk) too. It may be triggered by power supply glitches, undervoltage events, physical stress (flexing or bending) or over-heating. We are working to understand the exact conditions, but it’s hard to pin down. Perhaps the common element in your case is the battery?
Unfortunately we haven’t found a reliable method to restore these devices once they are in that state. This symptom with SanDisk flash has been reported all over the Internet but we haven’t seen a fix that works yet.
Can you answer the following questions, and I’ll see what we can do for you:
Are all the failed units v1.1.0, or has the v1.1.1 definitely failed as well?
Does the battery have any low voltage cut-off? In other words, will it stop discharging if it gets below a certain state of charge?
How do you recharge the battery? With the Fin still plugged in?
Are the Fins subjected to any particularly hot conditions, or harsh vibrations?
I’m sorry you’ve had such a terrible run so far! Hopefully together we can come up with a resolution.
Tanks @hraftery, we suspect it might be our power supply/solar controller board we have causing the problems but unfortunately it’s hard for us to confirm due to the original creator of the board no longer being involved. It’s plugged into the Belana Fin Pins whatever they are called(!) (not the Pi GPIO pin array)
We are charging via solar, which is why we have a solar charge controller to manage the power between the solar panel, battery and the FIN board.
Yes there should be a low charge cut off, but if there isn’t, will the Fin board not cope with low voltage?
Is there anything on the v1.1.1 board that would make it more robust to power problems?
Our one v1.1.1 board that was corrupted happened after we suspect the power board slipped off the pins during testing.
Normally our boards are secured in a weather tight case, stationary on a pole, so not moving or vibrations. We’re in New Zealand, so not subjected to extreme temperatures, cold or hot.
Do you have any other recommendations for solar charging remote deployments with the Fin?
If we were to move to a completely separate solar charge controller, would you recommend a voltage regulator of some sort between the battery and the Fin board?
Is that where power is supplied to the Fin? If so then that could indeed be an issue. It is not safe to supply power via that header.
Thanks for the extra info, it’s very useful to identify patterns and narrow down the issue so we don’t get repeats. In response:
The Fin is not rated below 6V (or below 5V via the HAT pins). It can cope, but given we suspect power supply fluctuations are implicated it would be useful to isolate that possibility.
Nothing on the v1.1.1 makes it more robust to power problems, but the more data on it in relation to the 32MB issue the better! The only change in v1.1.1 provides USB bus stability in hot environments.
No specific recommendations on solar charge arrangements, but it’s a fairly common application. As long as it powers the Fin via the barrel jack or Pheonix connector, and ideally, has a conditioned 6V to 24V output, you should be fine. There’s no need for a regulator, but standard voltage conditioning and under-voltage protection would be good for your battery and your system as a whole.
Let me know the details on where your power is supplied to the Fin and we’ll go from there. We’re keen to solve this 32MB corruption issue for you.
Thanks for that info!
Yes that’s right, our power management board is connected to the Co-Processor I/O connector. But checking the board it looks like it supplies power via cable to the ‘Phoenix power in’ port. I can email you some photos if that would help (sorry can’t post them here).
