Balena daemon won't start - not enough space?

Product support
1

Hi there! Pretty new here. I have an issue where I can’t deploy to my raspberry pi 4 anymore. Services are stuck at previous versions and exited, can’t restart them, and after logging in to the remote host and looking around I found that the Balena Engine is not running. When I tried restarting it manually, I got:
Error starting daemon: failed to get temp dir to generate runtime scripts: mkdir /var/lib/docker/gen-runtimes865678124: no space left on device

df -h gives me:

devtmpfs                           822M     0  822M   0% /dev
/dev/disk/by-partuuid/8617dc5c-02  300M  281M     0 100% /mnt/sysroot/active
/dev/disk/by-label/resin-state      19M  227K   17M   2% /mnt/state
overlay                            300M  281M     0 100% /
tmpfs                              955M     0  955M   0% /dev/shm
tmpfs                              955M   37M  918M   4% /run
tmpfs                              955M     0  955M   0% /sys/fs/cgroup
tmpfs                              955M     0  955M   0% /tmp
/dev/mmcblk0p1                      40M  8.3M   32M  21% /mnt/boot
tmpfs                              955M  100K  955M   1% /var/volatile
/dev/mmcblk0p6                      15G  9.0G  4.6G  67% /mnt/data

I can see two volumes that are full, but I’m afraid I can’t figure out what those are and how to fix this :smiley:

Any ideas? HALP :smiley:

2

Hi,

Could you grant us support access to the device and tell us the device uuid please? We can have a look at the device.

4

Hi! I have granted access, and the UUID is: b6e747162c11079213b77a9d8ead25b1

Is it okay to share UUIDs in public forums?

5

Hello, the device UUID is perfectly fine to share in public. Just a question: are you trying to deploy a very big release to this device, according to the df -h output above you should have around 4,5G of free space

7

Hi! Nope, release is the same size as before, four images in total, way less than what’s available.

[Info]        ┌────────────┬────────────┬────────────┐
[Info]        │ Service    │ Image Size │ Build Time │
[Info]        ├────────────┼────────────┼────────────┤
[Info]        │ gatekeeper │ 271.93 MB  │ 8 seconds  │
[Info]        ├────────────┼────────────┼────────────┤
[Info]        │ ui         │ 283.52 MB  │ 8 seconds  │
[Info]        ├────────────┼────────────┼────────────┤
[Info]        │ things     │ 271.14 MB  │ 8 seconds  │
[Info]        ├────────────┼────────────┼────────────┤
[Info]        │ mongo      │ 346.00 MB  │ 2 seconds  │
[Info]        └────────────┴────────────┴────────────┘
9

Hello,

We’ve had a look at the device in question. There is more than enough space on the data partition, which is where the /var/lib/docker directory is mapped, and the error seems to suggest that it’s trying to map this directory to the RO root rather than the data partition. Having had a look at the diagnostics for the device, it also shows very slow media access. It’s possible that this is down to a corrupt SD card, which would also explain the strange disk space/mapping error. What kind of SD card is this? We generally recommend the SanDisk Extreme Pro range, as we’ve tested this heavily and they show the best results for the Pi series.

It might be well worth trying another SD card to see if this is the issue.

Best regards,

Heds

11

Thanks for looking into this! I’m using a Samsung card, but I have been using it for quite a while now, so it could be faulty. I will try another (newer and faster) Samsung to see if it works and report back.

12

Great, thank you! Let us know if you see the same issue with the new card, and we’ll take another look at the device.

Best regards,

Heds