The problem was solved with the kind help of Gergely in the Resin Chat.
His closing statement:
Checked the supervisor, the version running on your device should have all our current delta improvements available, so would recommend using that. If the image is still processing, there should not be any downloads done until that is finished, so that shouldn’t max out your network traffic.
We didn’t do anything on the devices, haven’t changed anything else.
Would say probably, that:
- hope this won’t happen again, then it’s all good
- if it happens again, ping us before switching off deltas and such, to see what is really happening. As I think delta downloads should have still worked even if the network disconnects often, as it resumes. But maybe there are degrees of network “badness”, some that are not dealt with well. That would be something we’d like to see in action to be able to address if possible.
Not sure how big the delta was either, unfortunately. Looked at your releases, and looks like the majority was built from cache, so would expect that deltas work well. If the image is large and there are this many, the delta generation might take a bit though, hence the initial “looping” possibly.
It seems to me that the network connection was very slow overall and the frequent “restarting” is something that happens when the delta updates go through step by step.
Maybe it would be nice for Resin to add a “counter” or something similar to make this more transparent, as the 100% do NOT seem to indicate real update progress in this case.
What we did in the end was mainly repairing the data connection, topping up on data volume and waiting. One device finished pretty quickly without Delta updates enabled. Another one took a longer time with Delta Updates enabled - going from 0% to 100% and back again 5-10 times until it was finally finished.
It might be worthwhile to investigate in the future, exactly HOW delta updates are built and exactly WHY they became so time consuming to deploy in our case.