I have an update to share.
The user/group quota piggybacks on file ownership to calculate disk usage on a per user/group basis. You can use that from inside a container because you can create files as any user/group inside the namespace.
The problem is of course that we don’t have user namespaces enabled and so two containers that use say user id 2 to store all their files will step on each other toes when working with limits. On top of that if you want to have multiple limits for different volumes you have to store the files as different user ids etc, which is very cumbersome.
XFS supports a feature called project quotas. A project is yet another piece of metadata that each file has and it’s just an id, similar to user id or group id. The advantage is that it’s not associated with anything so you don’t have to jump through hoops to tag your files. Docker integrated that to support limits per volume.
Now, etx4 also supports project quotas since linux 4.5 so the way forward for balenaOS would be to enable that feature in our filesystem and then add support for that in docker and as an extension to balenaEngine. The code shouldn’t be very different.
The right way forward is to add Project quota support for ext4 in docker.
There is already a feature request and some work on that aspect. See https://github.com/moby/moby/issues/29364
There is one unknown at the moment. All this work is for containers and not volumes (aren’t sure yet if it applies to volumes as well). But I guess if we get storage limits in containers, there should be a way forward for applications to utilise that.
As you can imagine, this is a rather larger piece of work and I can’t provide any timelines or even if we will fix this upstream or not. If upstream does add the ext4 project quota feature in docker, we will surely pull it in balenaEngine.