Introducing balenaOS v7: read-only root by design

Discussions
1

Originally published at: Introducing balenaOS v7: read-only root by design

balenaOS v7 is a major release. The headline change is a single line in the changelog: the root filesystem is now mounted read-only. It sounds small, but it’s a foundational shift.

2

In earlier versions, mobynit (the tool that assembles the rootfs from container layers) used Docker’s overlay2 driver with a writable upper layer. That meant writes to paths like /etc, /usr, and /lib were technically possible at runtime, even though they were ephemeral and lost on reboot.

“lost on reboot” does not seem to be accurate? I always thought this would be the case but then a coworker went and changed /etc/chrony.conf to poll more often and the change definitely persisted across many reboots.

This was confusing at first but everything points at it being completely expected behavior. The upper layer of the overlay2 appears to be a normal writable ext4 directory.

And uh, the lower layer also appears to be writable, just in a more illegal way (source)

Changes to the underlying filesystems while part of a mounted overlay filesystem are not allowed. If the underlying filesystem is changed, the behavior of the overlay is undefined, though it will not result in a crash or deadlock.

3

Is the plan for hostapp-extensions to become a supported path for the sort of modifications currently being done by remounting the root filesystem and modifying files in place? I’m aware its not an officially endorsed path currently but we have a container which is responsible for modifying some balenaOS configuration to support our requirements - hostapp-extensions look like a great path for properly facilitating that in a way which has much less risk of causing the device to become unbootable.

4

Thanks for those points @dequis , the post has been updated to avoid any confusion.

5

Hi Jon,

Really good question.

As @rosswesleyporter puts it, we sometimes describe hostapp-extensions as a “sidecar” to balenaOS. It’s a way for us to selectively add functions to balenaOS without disrupting other things. As the post suggests, we’ll use that to add drivers, container runtimes, etc. In the more distant future we may create a mechanism for users to add their own extensions, but that is not part of the current effort.