Device Start End Sectors Size Type
/dev/mmcblk1p1 1024 5119 4096 2M Linux reserved
/dev/mmcblk1p2 5120 87039 81920 40M Linux filesystem
/dev/mmcblk1p3 87040 742399 655360 320M Linux reserved
/dev/mmcblk1p4 742400 1397759 655360 320M Linux reserved
/dev/mmcblk1p5 1397760 1438719 40960 20M Linux reserved
/dev/mmcblk1p6 1438720 7732223 6293504 3G Linux reserved
The error I get in BalenaCloud log:
Failed to download image 'registry2.balena-cloud.com/v2/xxxxxxxxxxxxxxxxxxxxxx@sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' due to 'failed to register layer: Error processing tar file(exit status 1): write /usr/sbin/genl: no space left on device'
journalctl -a -f -u balena-supervisor
Sep 07 12:41:26 ecddf19 balena-supervisor[1228]: [event] Event: Image download error {"error":{"message":"failed to register layer: Error processing tar file(exit status 1): write /usr/sbin/genl: no space left on device","stack":"Error: failed to register layer: Error processing tar file(exit status 1): write /usr/sbin/genl: no space left on device\n at Stream.<anonymous> (/usr/src/app/dist/app.js:10:2309272)\n at Stream.emit (events.js:310:20)\n at drain (/usr/src/app/dist/app.js:2:306286)\n at Stream.stream.queue.stream.push (/usr/src/app/dist/app.js:2:306693)\n at Parser.parser.onToken (/usr/src/app/dist/app.js:10:256101)\n at Parser.proto.write (/usr/src/app/dist/app.js:10:704892)\n at Stream.<anonymous> (/usr/src/app/dist/app.js:10:254565)\n at Stream.stream.write (/usr/src/app/dist/app.js:2:306562)\n at IncomingMessage.ondata (_stream_readable.js:695:22)\n at IncomingMessage.emit (events.js:310:20)\n at addChunk (_stream_readable.js:286:12)\n at readableAddChunk (_stream_readable.js:268:9)\n at IncomingMessage.Readable.push (_stream_readable.js:209:10)\n at HTTPParser.parserOnBody (_http_common.js:132:24)\n at Socket.socketOnData (_http_client.js:476:22)\n at Socket.emit (events.js:310:20)"},"image":{"name":"registry2.balena-cloud.com/v2/xxxxxxxxxxxxxxxxxxxxxx@sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","appId":XXXXXXX,"serviceId":XXXXXX,"serviceName":"main","imageId":XXXXXXX,"releaseId":XXXXXX,"dependent":0}}
Sep 07 12:41:28 ecddf19 balena-supervisor[1228]: [error] Scheduling another update attempt in 16000ms due to failure: Error: Failed to apply state transition steps. Steps:["fetch"]
Sep 07 12:41:28 ecddf19 balena-supervisor[1228]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
Sep 07 12:41:28 ecddf19 balena-supervisor[1228]: [error] Device state apply error Error: Failed to apply state transition steps. Steps:["fetch"]
Sep 07 12:41:28 ecddf19 balena-supervisor[1228]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
I have also purged data from within BalenaCloud with no change,
The device is a STM32MP1 based board so maybe I have to do some more changes in my meta-layer. I have notices some renaming from resin to balena between the versions and also that the rootA and rootB size is increased from 300MB to 312MB.
I finaly solved it temporary by running resize2fs /dev/mmcblk1p6. This expanded the filesystem on my data partition and the device can download the container.
So I guess I have to do some changes in my custom yocto layer to fit the change made in BalenaOS 2.59.0 that start omiting fs check and resize if partition is mounted.
I can override the resin-filesystem-expand - script, but I guess there is a better ways like not mounting the partition before running the script? Any suggestions?
hi, is this a Balena supported device type? If not and you are building a custom OS, have you considered contributing this device so that it can be added as a community supported device to balenaCloud (see Device Types and Versioning - Balena Documentation)?
Once the device is a community supported device we will be in a better position to support you.
Hi, it’s not the first time I get this question from your team and maybe it’s time to contributing and add it as a community device.
The problem is that the board we are using is customized for a specialized commercial B2B product.
We are using a System-On-Module but there are development boards with the SOM mounted. Maybe adding one of those development boards could be of interest for the community. Unfortunately our board is not 100% compatible with the development board as we have removed functions to use IOs for other product specific functions so it will not always help us in our development, maybe it just gives us more work.
hi, I think the way forward here would be for your team to contribute the dev board support as a community device, and then to maintain your own layer with product specific customizations.
This would allow us to offer support on the community device, for example when having to adapt it to new balenaOS releases. Otherwise it is difficult for us to provide support as it is effectively a customOS.
Having same problem with Dunfell and later BalenaOS. The problem is that the partition is already mounted when the mounting service is started.
As it is already mounted, the expanding data partition service does not run.
So I guess the solution is to find where partitions get mounted earlier. Any suggestion, where to look?
I think /data is really needed by balenaOS, hence its directly mounted at boot. Probably thats a wrong vector, but I thought I might push that idea here: How about changing the data partition size during the Yocto image creation? If I see correctly from this Karo module (btw, they are actually quite near my location… didn’t know, cool ) - they come with 4 GB eMMC. So changing the data partition to its nominal size and burning it “bit-for-bit” to the eMMC would take away this hassle. But it would, most certainly, increase the deployment time by a bit. So its more of a temporary solution until something better is found… ( in my yocto config I used to play around with the IMAGE_ROOTFS_SIZE to achieve something like that - but I don’t know about how balenaOS creates its partitions, so… it could be more difficult,…)
I know BalenaOS need /data later on but there is a balena service to handle the mounting. As the partition is already mounted it can’t resize the partition and just continues.
As you wrote, resizing the partition in Yocto could be a workaround but I don’t think it’s the correct way to solve it. I think the best way to solve it is to prevent kernel to automount the partition and let the balena service resize and then mount it. Maybe let the service unmount before the resize script could also be a solution.
I completely concur.
I think that balenaEngine and the supervisor need the partition.
But I don’t know how normally resizing is handled at the moment.
I just glanced over the balena-meta layers and did not directly find it, but might have overlooked it.
Maybe @alexgg can shine a light on that.
There is a resin-filesystem-expand script that is started by resin-filesystem-expand.service on boot.
From BalenaOS 2.59.0 the script was changed to "Omit fs check and resize if partition is mounted"
) - they come with 4 GB eMMC. So changing the data partition to its nominal size and burning it “bit-for-bit” to the eMMC would take away this hassle. But it would, most certainly, increase the deployment time by a bit. So its more of a temporary solution until something better is found… ( in my yocto config I used to play around with the IMAGE_ROOTFS_SIZE to achieve something like that - but I don’t know about how balenaOS creates its partitions, so… it could be more difficult,…)