[========> ] 16%[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/50dd9b0eed9438dfa07fca9a6290602d:latest
Thanks for reporting this, we have been looking into this issues lately, but still haven’t managed to find a permanent solution. The problem is caused by the garbage collector in the builder, which may remove images too aggressively causing other builds to fail with an error like the one above. For the time being, the only mitigation we can recommend is to re-trigger the build, as that should make it go through eventually.
A ticket is attached to this support thread and we will get back to you as soon as we manage to fully solve the problem, apologies for the bumpiness in the meantime.
Thank you Balena Cloud team for the feedback on this issue. I wanted to mention that I’ve observed this behaviour on git push and balena push actions for weeks now, but it has become particularly bad lately. As @SplitIce mentioned, it’s at the point now where I know that most build attempts will fail.
It goes without saying that this is a frustrating impact to productivity since it is at the end of the build process that this happens. Here are my failures from just this afternoon, of these I have had two successes.
Edit: after 3 more builds I’ve run out of time today and have given up. Since I saw mention of this recently, I’ll add that my builds happen fairly regularly (each day at present), and they typically finish well within 10 minutes. You can look up internal details using my examples below.
Is this issue under active investigation?
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/4cdf86a529592ac4f408ea1634ba21d3:latest
[Info] Built on arm03
[Error] Not deploying release.
Remote build failed
Info] Generating image deltas from release b22a419b70c9e716c9d48ff657703d95 (id: 1466717)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/5d3424f367ae15e0c76b2aeb4ace85b7:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
[Info] Generating image deltas from release b22a419b70c9e716c9d48ff657703d95 (id: 1466717)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/ba85849c48fc92bc73a4a87048849dcc:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
[Info] Generating image deltas from release b22a419b70c9e716c9d48ff657703d95 (id: 1466717)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/08db03baf41dc32328e33a84859cf17b:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
[Info] Generating image deltas from release 99176b8d32f7c4330ac1e4fbb65a8265 (id: 1467042)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/03856c294dc8d2cbf3479e1ad788e893:latest
[Info] Built on arm03
[Error] Not deploying release.
Remote build failed
[Info] Generating image deltas from release 99176b8d32f7c4330ac1e4fbb65a8265 (id: 1467042)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/1a261913f23646dbf1f6a5f34c584b86:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
[Info] Generating image deltas from release 99176b8d32f7c4330ac1e4fbb65a8265 (id: 1467042)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/d8a692ebd338ef495c53ceee785d0caa:latest
[Info] Built on arm03
[Error] Not deploying release.
[============> ] 25%Remote build failed
[Info] Generating image deltas from release 7d07b890d1abcc37b5e00442e8eeb9cf (id: 1467061)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/b7fed7591fca42772e708a7638496965:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
[Info] Generating image deltas from release 7d07b890d1abcc37b5e00442e8eeb9cf (id: 1467061)
[Success] Successfully generated image deltas
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - No such image: registry2.balena-cloud.com/v2/f42766c0501ac0a751e61b0c528e67ae:latest
[Info] Built on arm03
[Error] Not deploying release.
[=========================> ] 50%Remote build failed
Hi @SplitIce, @exposedwiring,
I’d like to share with you some internal feedback from the team regarding the builder failures.
The failures are caused by the builder Garbage Collection, which does run from time to time. These GC runs caused the missing image errors that users may have encountered. Among the affected images, must have been your own images as well.
We are actively working to move the builders to a new infrastructure and includes more fixes to avoid the missing images issue, which will be completed soon. We don’t have an ETA for that change of infrastructure though, but hopefully, we can get it done fairly easily and fast.
In the meanwhile, the only workaround we can suggest is trying to build your images again.
We’ll keep you posted regarding this issue,
Georgia