And we are upgrading to 128 GB Samsung EVO Plus cards in our Raspberry Pi 4 devices.
My question is if there are any tips or configuration settings that can be used to minimize the write cycles and extend the life of the SD cards?
We are only running a single container with a fork of the balena browser block. It simply loads a URL in Chrome that runs an Angular PWA.
Are there any Chrome configurations that I need to be aware of to limit how much data chrome writes to the disk for logs?
Does anyone have experience with “industrial” SD cards?
These devices are used for digital signage, so they are running in the field 24/7. They are not all easily accessible, so anything that we can do to extend their lifespan or reduce the likelihood of corruptions would be helpful.
I realize that SD cards are not ideal for this, and we are looking into other hardware options going forward, but we already have a sizeable Raspberry Pi fleet. Any tips on extending SD card life would be appreciated.
I haven’t experimented with the RasPiKey, just read about them. I want to though.
I wonder if it’s time we try and get some answers to the long-running question about which hardware to choose for the RPI 4 price. It seems like many of us are in that gap where we want the RPI4 priced entry point but the SD card reliability just leaves us a bit uncomfortable. I am thinking we could identify a couple of contenders, then develop a series of tests to run on each (a ‘Balena Games’):
RPI4 with SD card
RPI4 with RasPiKey
Another brand, something like the BeagleBone but would need to be cheaper to really be a contender?
Our own custom build, I am thinking we could pick a cheap carrier board and put a RPI compute module on top
All ideas welcome
Tests could be as simple or extravagant as we like:
Read/write speeds
SD card wear after hammering a card or two with writes
Wi-fi ranges
Hardware performance in high temperature and low temperature environments
Weather tolerance (relates to cases)
Ability to run on low power
Underwater
Ability to run in the stomach of a bear
Ability to skydive
No limits on ideas, could be a lot of fun seeing these things put through their paces.
Hopefully there will be a clear winner, although either way the amount of info we could gain from each test and device would most certainly help in decision making.
Based on the winner(s), we could then discuss a suitable case design under it’s own set of tests, and I can put together a 3D printable version (RasPiKey for example will likely need more space to enclose the card; the custom carrier board build will need it’s own case etc.).
Based on posts I have read, I am thinking @keenanjohnson@steelrat@radexter@the-real-kenna among others might also be interested in this, and can likely share a lot of experience on potential contenders and designs.
If there is enough interest and design input then I will try and take on what I can.
I for sure am curious about this - thanks for thinking of me!
I guess I want to narrow the end-game of this if we’re to start running tests… thinking back to mv1's request, this would need to be something that could extend the lifespan of already deployed devices, for which I might think of slightly different solutions than devices still-to-be-deployed.
So some limitations of already-deployed devices are:
they already have a case, so space limitations might restrict solutions
cost of adding to the device would need to be minimized, instead of something that could be considered as part of designing the product
we probably can’t swap to something totally new (like BeagleBone devices) but would need to rely on the SBC that’s already deployed
preferably something that doesn’t require physical access to the device (like using the ARM version of sdtool [1] to enable TMP_WRITE_PROTECT on the SD card, in use-cases where that’s possible)
In the case of already-deployed devices, like @mv1’s situation, I feel like my best thought is to replace the SD cards with those which are rated for more writes over their lifetime. Usually I think of the ones typically labelled “endurance” because they are intended for a high number of writes, aimed at the surveillance video sector. [2][3]
That said @maggie0002, I recently stumbled into a really nice break down of read/write speeds across SD cards [4], done pretty recently (March 2022). It was fun to just see that many cards tested on that many SBCs! I do wish he’d made the data manipulatable, so that I could for instance compare the performance of an SD card across multiple SBCs (for instance RPi4 vs BB Black). But alas, for that there is manual scrolling involved.
Also, there has been discussion a few times inside balena to do some SD card benchmarking just for funsies (and to share as part of a blogpost) so that we can make better recommendations in general. I do hope it happens someday (soon) so that we can do a bit more than speculate. But if you are up for more than just an endurance challenge, I’m game to define something of a wider scope and see what we can contribute from the balena-side. Let me know what you think.
Thanks for the reply! While I am certainly open to experimenting with other devices for future deployments, you are right that my primary focus is extending the life of the card in already deployed devices. We have over 1000 Raspberry Pi 4’s deployed in the field, so replacing the devices is not really an option at this point. Much of the fleet was deployed around 2 years ago with the basic 32 GB SanDisk cards that come with the CanaKit. Surprisingly, they have held up pretty well, but we are seeing more and more failures recently. We are replacing them with upgraded SD cards going forward.
SD card I/O performance isn’t really a concern. I’m mostly looking for advice on what can be done to minimize the writing to the card as much as possible. For example if there are any chrome flags or settings I should be using to reduce how much it accesses the drive. Or if there are any configuration settings I should be using.
Indeed, I hijacked the thread a little with a detour conversation, apologies about that.
To the original question, I am not sure of much more that can be done to reduce writes. Balena is already really optimised for reducing writes. I see from the Balena Browser project @phil-d-wilson is listed in the commits, maybe he can be more helpful on the project specific things?
The RasPiKey though is likely a viable option, I haven’t tried them but I have read they do fit ok with a lot of existing cases, if the case has an exposed SD card slot like the official raspberry pi cases (although it will stick out the side, so perhaps not ideal for outdoor use). My comment on a new case was for the instances when you want to enclose the SD card inside a case. When you start looking at the larger and industry level SD cards, the price difference isn’t always that great between the RasPiKey and SD card (although the one’s that @the-real-kenna found seem particularly cheap which is nice).
If you do go for the SD card, then I always opt for the largest card I can justify buying, as the wear levelling means they will last longer, but looks like you are already ahead of me on that one. It does create an interesting business question using SD cards. Do you spend more on industry level cards and run them until they fail, or do you budget for an SD card swap every X months or years? Considering you have been running for 2 years, you have a good idea of how long you can get out of the cards, you could budget for a total card swap after 18 months which would mean reduced downtime. Would be curious to know if you find it more cost effective to get better cards, or just to budget the standard SanDisk cost card replacement as part of the operating costs. I guess the only way to know for sure would be to try half with the SanDisk cards and half with industry cards and see where the most value lay, although I would want to account for environmental costs throwing away that many cards too.
The comments on testing new hardware was me latching on to your comment:
looking into other hardware options going forward
If there is interest then will be sure to start a new thread on it.
Thank you, this is good to know. I mostly just wanted to make sure I haven’t missed something that needed to be configured. I’ll continue to look into if anything needs to be done to reduce writes by Chromium or if that’s even an issue.
I’m going to look into the RasPiKey as well. Even if they cost a bit more than a high-end SD card, the reduced cost of having to replace failing SD cards would make them more than worth it. My only concerns there are if they have been tested for longevity and being able to secure a stead supply. I’ll see if I can get a few for testing and post my results.
I’ve started another thread about exploring alternative devices. I’m happy to stick with Raspberry Pi as long as I can get a decent lifespan out of the cards and the RasPiKey may help there, but it’s good to know what the options are.
