Occasional "unknown blob" while uploading release

Product support
1

Use case:

Automatically upload release when test has been passed successful

Issue:

I have seen similar behavior as described in There was a problem uploading the images to the registry.

Jenkins jobs are setup to compile and test software. When all is okay, the software is combined into a resin docker image and uploaded to the resin backend.

Sometimes this upload process fails with the error message “Unknown blob”. The size of image is 2.3GB. Likelihood is approximately once per 20 upload.

To recover from this, then just upload the image again and all is good.

4

Hi @aliasbits,

I am having this problems too. In the past, the ratio of failures was about 1/20 as you said. However, since a few months ago, this happens to me much more frequently.

In my case, the image size is about 1GB, but it is a microservices app with 10 services. Most of them share the same base image, but I as I am using resin deploy to upload the images to resin I need to upload all of these images, which increases the size up to 10GB (I cannot use their builders because I am using CI for tests in other servers).

Do you know why this happens? Is it a problem of my network connection? Ot is it a problem of resin servers? Any solutions?

Thanks in advance!

9

Hi jcozar

I do not know why this happens. I have made a retry solution which will redeploy if unknown blob is detected.

Not the optimal solution, but the best right know.

12

hi @jcozar @aliasbits,

This issue is often caused by network hiccups. We are monitoring the problem and are working on making deploys more reliable.

Thanks for bringing this to our attention!

Best, Kostas

14

I experience the issue here, too.
A mild annoyance, not more.

Looking forward to a fix.

16

I suffer from this as well. Image size around 4GB so each deploy failure is a serious waste of time.

18

Hello, today we released balena-cli v9.14.5 that we expect is able to mitigate this issue and also help with transient network issues from failing the deploy. Please do let us know if you find the issue persists after an update to the new version.