for our resin.os based product we would like to create partition where we will store content. This partition should survive SW update (content should remain). I hacked little bit extending of data partition (limit it to 20G instead of whole remain space) and create content partition. Problem is that when I update system from usb content partition is lost and recreated again (I also have similar script like for expand data partition). I figured out that data and state partions are in extended block which is always recreated during SW update so it means that my content partition is always lost. I was thinking about creating new primary partition and limit again data partition but unsure if this is way to go. Any pointers which direction should follow. Thanks a lot. (If necessary I can share scripts)
Can you elaborate a bit, please?
When you say “SW update” do you mean your application update (when you push the new version to resin git) or the host OS update (using the corresponding dashboard feature)?
Do you know that resin already provides you with /data partition that should survive both?
Please check this and let me know if it works for you.
SW update I mean doing system update using usb stick and reflashing new system (basically putting fres copy of image). Are you sure that /data survive system update? Looking at scripts and from my testing data partition is recreated on every system update. Thanks.
What device are you using?
I read now that it’s the device with the internal memory, like a NUC?
I didn’t mention this scenario (host OS update using the resin dashboard is the different thing).
When we find some issue in core system we need to have possibility to update (via USB stick) or via resinHUP (in future). So that’s the main usecase for us. Thanks.
OK, so when resinHUP supports OS v2 you should be OK with the /data partition.
I’ve asked about the USB reflash now, will get back to you when I have more info.
Just jumping in here, as Marek and I are working together on this. We’re unable to use the /data partition for a number of different reasons, one of which is the sensitivity of the content we’re storing - we’re contractually required to keep it logically separate from other parts of the application, and eventually we’ll be adding encryption to the partition.
Additionally, we have around 200GB of data there, and if we need to “purge” the application (which does happen from time to time) then we can’t afford to lose that data.
Sure, we’re using docker-in-docker, together with docker-compose to enable a multi-container architecture. The quickest (and safest) way to force a complete rebuild of containers is with a purge.
The current Implementation of the partition table is using MBR which only supports 4 primary partitions. All of these 4 are used by the OS so adding a new partition needs a change in the partition table type too. Which is probably something that you guys changed if you already have a system working.
Secondly, it is true that flasher process will completely repartition and rewrites the internal media. Which includes data partition and anything else on it. Which means stored data in the resin-data partition (last logical partition in the extended one) will not be persistent.
There are two aspects here. Firstly I see that hostOS modifications are needed. For example your OS needs to know to mount the persistent partition and make it available to the container. Secondly the update needs to cooperate with these changes and this will be very easy when two features will be ready: resinhup in 2.X (this will allow you to update a device only by touching the rootfs partition - and maybe the state too) and host apps (whch will allow you to define a custom operating system based on resinOS which will include your changes). When these too will be ready - sadly I don’t have an ETA for them but I can assure you they are under active development - there will be one last bit that needs to be figured out: how to initially configure the device based on your partition table requirements. The easiest way would probably be using a different disk for the custom partition.
Tell me what you think about all this and if everything looks in sync with your requirements.
thanks for valuable info. As I mentioned in previous post we shrink data partition to 20G and rest use for our content partition. We will wait for resinHUP update. I think initialization can remain how we have done it now (I can share if necessary). Thanks.
One thing just to clarify your current implementation. You force the data partition to not take more than 20G. But how do you access the data? Do you use another partition or read raw from the disk?
So you used a logical partition in the extended one. This means that you had to modify the EBR on the data partition (labelled resin-data) to point to the partition that you keep persistent, right?
Yes exactly. What we did is basically change resindataexpand initrd script to not took rest of the disk for data but only some amount. Then I created separate initrd script where I create partition and resize it and finally systemd mount script which will mount it to /mnt/content.
I completely understood now. Thanks for clarifying. In this case resinhup and host apps will definitely help your usecase. Do follow resinos repository for update on these.
Marek