i have a problem updating our sensors. 1 of 3 sensors in a group updated successful, the other two are loading the updates up to 100% and afterwards restarts the process. Our mobile data volume is consumed three times now. It is a quite small update.
edit: it is only one container that is unable to update - even if there is no change. We tried delta update and normal
5372def0269fbc6dcc80d42e8743044c
support access granted
I’ve got this same bug, tested both delta and full updates.
08.08.18 19:58:44 (+1000) Service is already running 'main sha256:e4e4c7f772ee596dc5644670db8233f19232bc6ba59dac0b92b7f96528e2daac'
08.08.18 19:58:47 (+1000) Downloading image 'registry2.resin.io/v2/ca32b3e850deec371ad2c3df97ccf6ac@sha256:683410e357301e587c77481a0c70a616a1ee1140fe09b13bd0dd11e365c1baf0'
and repeats
UUID: dcea8b0b6e97154dfecb4f09c929bc34
Support Access granted
I’m going to try and push a mostly empty update to see if it’s specific to the build 49daf401cc13cc935cdf2048641215a5797209ee
In my case the stuck device was able to update to the next release (For the resin guys this is 22712a844d0926e13eceef912e464459a7d7b446).
I’ll refrain from purposefully restarting the device in case the resin guys want to look at the device which they are welcome to do.
what did you do to overcome the issue?
^This
I tried pushing a mostly empty update. it is always stuck with one container - even when there is no change at all in that container compared the one which is working.
The problem was solved with the kind help of Gergely in the Resin Chat.
His closing statement:
Checked the supervisor, the version running on your device should have all our current delta improvements available, so would recommend using that. If the image is still processing, there should not be any downloads done until that is finished, so that shouldn’t max out your network traffic.
We didn’t do anything on the devices, haven’t changed anything else.
Would say probably, that:
- hope this won’t happen again, then it’s all good
- if it happens again, ping us before switching off deltas and such, to see what is really happening. As I think delta downloads should have still worked even if the network disconnects often, as it resumes. But maybe there are degrees of network “badness”, some that are not dealt with well. That would be something we’d like to see in action to be able to address if possible.
Not sure how big the delta was either, unfortunately. Looked at your releases, and looks like the majority was built from cache, so would expect that deltas work well. If the image is large and there are this many, the delta generation might take a bit though, hence the initial “looping” possibly.
It seems to me that the network connection was very slow overall and the frequent “restarting” is something that happens when the delta updates go through step by step.
Maybe it would be nice for Resin to add a “counter” or something similar to make this more transparent, as the 100% do NOT seem to indicate real update progress in this case.
What we did in the end was mainly repairing the data connection, topping up on data volume and waiting. One device finished pretty quickly without Delta updates enabled. Another one took a longer time with Delta Updates enabled - going from 0% to 100% and back again 5-10 times until it was finally finished.
It might be worthwhile to investigate in the future, exactly HOW delta updates are built and exactly WHY they became so time consuming to deploy in our case.
FYI Pablo from looked into our occurrence and it was due to the supervisor crashing.
The common element here seems to be incomplete / interrupted updates.
