# balenaFin stuck **URL:** \ **Category:** balenaFin\ **Created:** [January 15, 2020, 4:58pm UTC](https://forums.balena.io/t/balenafin-stuck/56947 "2020-01-15T16:58:17Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 15, 2020, 4:58pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/1 "2020-01-15T16:58:17Z")
I have a BalenaFin 1.1 stuck at the very end of an application update. Updated 3 out of 4 containers, 4th container is stuck at the “Installed” status. Hard and soft reboots and subsequent application updates have failed to have any effect. The 3 good containers are running. Been stuck this way for a couple of days. Been having similar problems with Pi3/4s and moved over to the fin to eliminate SD card considerations. diagnostics produces: “Error reported when querying checks data: ssh client socket error while initiating SSH connection” Device is pingable throughout. Anyone have any ideas? ---
**Author:** ![hedss](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/hedss/32/210_2.png) [@hedss](https://forums.balena.io/u/hedss)\ **Post date:** [January 16, 2020, 1:08pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/4 "2020-01-16T13:08:10Z")
Hi, So by the sounds of it, you’re not able to SSH into the device from the Dashboard or balena CLI? If this isn’t the case, then if you could supply us with the device dashboard URL, we’ll have a look at the device for you. If it is the case that you’re not able to SSH to the device, is there another device on the same network? Best regards, Heds ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 1:10pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/6 "2020-01-16T13:10:10Z")
979bfd593d1f9a1f2bf4892e882dd51f ---
**Author:** ![hedss](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/hedss/32/210_2.png) [@hedss](https://forums.balena.io/u/hedss)\ **Post date:** [January 16, 2020, 1:38pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/7 "2020-01-16T13:38:52Z")
Thank you. It seems that all services are running normally here looking at the Dashboard (none are stuck at installed and all are showing the latest application release). I do see something strange in that the Supervisor service seems to have got into a bad state (and is not connecting properly). It’s also taken me several attempts to SSH in, which usually corresponds with a very bad network connection, however once in the device is very responsive. Currently, the device is in a bad state due to the Supervisor. I’d like to try and sort the Supervisor out, but this might result in me having to stop balenaEngine and the services as well. Would this be acceptable? Best regards, Heds ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 1:49pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/9 "2020-01-16T13:49:43Z")
Go for it. I need to figure this out. ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 5:43pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/17 "2020-01-16T17:43:58Z")
Hi. Some operations are timing out and taking too long, including a restart of the balenaEngine. Can this device be restarted? As for the reason why, my guess is an SD card issue. System logs show `balenad` hanging for at least two minutes while trying to do I/O (and that’s also why it is timing out on a restart). I saw timeouts on I/O-heavy operations as well, and you mentioned SD card issues. If it’s possible, could you try another SD card? From experience, we usually recommend the Sandisk Extreme Pro. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 5:55pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/19 "2020-01-16T17:55:35Z")
This device is a fin - so … SD card? It’s been power cycled. ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 6:05pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/20 "2020-01-16T18:05:03Z")
Sorry, yes it’s a Fin. I’ll get in touch with the right people and update you when we have something. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 6:08pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/22 "2020-01-16T18:08:24Z")
I’m not looking so much to fix this individual devices as to learn how to debug this situation myself. Personally, I suspected network connectivity but you see otherwise? If it’s any help, the device has 6 i2c and 2 GPIO devices (4gpio pins) in use. The GPIO/i2c stuff all seems to work correctly individually. I can unplug all that if it helps. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 6:14pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/23 "2020-01-16T18:14:22Z")
I don’t think it’s a power thing either - the fin is on the standard supply and the 4, 3b and 3b+ are all on decent 2.5A supplies that have handled similar loads for me in the past and I don’t get any blinky power LED. ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 6:25pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/24 "2020-01-16T18:25:36Z")
Different pieces of the stack are disagreeing on what is the actual state of the supervisor, and that is leading to some odd issues. Part of the that seems to be caused by slow I/O, which may trigger some timeouts. I don’t think i2c or GPIO are related. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 6:26pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/25 "2020-01-16T18:26:09Z")
IO as in network IO or storage IO? ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 6:26pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/26 "2020-01-16T18:26:45Z")
Storage I/O ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 6:29pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/27 "2020-01-16T18:29:25Z")
When I tried to restart the engine, `balenad` was killed by the kernel due to blocking way too long while doing disk I/O, so that’s _an_ issue. I don’t know yet if that’s _the_ issue. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 6:30pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/28 "2020-01-16T18:30:35Z")
Container issue? 1 python-buster, 1 python-stretch, 2 golangs? Camera presence? (1) ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 16, 2020, 8:54pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/30 "2020-01-16T20:54:24Z")
What are you looking at to determine this, and can I see the same info? ---
**Author:** ![Ereski](https://avatars.discourse-cdn.com/v4/letter/e/958977/32.png) [@Ereski](https://forums.balena.io/u/Ereski)\ **Post date:** [January 16, 2020, 9:12pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/31 "2020-01-16T21:12:25Z")
Both the standard `dmesg` and `journactl` commands should give you info if a task is killed due to extreme situations, such as out of memory or blocking too long inside the kernel (such as due to very slow IO). `journalctl` is a standard utility in systemd machines, and it usually gives a lot of good information for debugging. ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 17, 2020, 6:20pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/33 "2020-01-17T18:20:50Z")
Ideas? ---
**Author:** ![garethtdavies](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/garethtdavies/32/2669_2.png) [@garethtdavies](https://forums.balena.io/u/garethtdavies)\ **Post date:** [January 17, 2020, 9:59pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/36 "2020-01-17T21:59:32Z")
Hi @rodley, as noted above this may be related to a storage issue. One of our engineers suggested a way to get a ball park estimation for the state of the eMMC on the Fin, which might provide some more insight into this. If you want to try then here are the instructions: - Install mmc utils (apt-get install mmc-utils) - Run `mmc extcsd read /dev/mmcblk0` - There will be a field called `EXT_CSD_DEVICE_LIFE_TIME_EST_TYP_B` - The value of that field gives you an idea of the wear on the memory in the following way: 0x01 ~ \<10% of maximum wear 0x02 ~ \<20% of maximum wear … ---
**Author:** ![rodley](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rodley/32/2344_2.png) [@rodley](https://forums.balena.io/u/rodley)\ **Post date:** [January 18, 2020, 5:52pm UTC](https://forums.balena.io/t/balenafin-stuck/56947/39 "2020-01-18T17:52:22Z")
Here’s my action plan: - Reflash Pi3B, 3B+, 4B and Fin as development versions with local mode enabled using recommended sd cards (except fin) - Run the eMMC diagnostics above on fin to estimate wear (should be very low as device has been very lightly used) - Report back here with wear results. - Push some application updates locally and see if they “take” quickly, unlike the current situation where updates take many hours. - If updates go wonky, we’ve eliminated the firewall/router/internet connection as issue. Check dmesg and journalctl - get back on forum for further help. Sound good? [Next page](https://forums.balena.io/t/balenafin-stuck/56947.md?page=2)