Docker compose: no such image

A pre-built image referenced in the docker compose config is not found.
This used to work. Did anything change? Is docker hub no longer supported for external images?

docker-compose.yml extract:

version: '2'

services:
  db:
    image: arm64v8/postgres:12-alpine
    ...

  web:
    build: services/web
    ...

Log output:

[Info]                  Uploading images
[Error]                 An error occured: (HTTP code 404) no such image - no such image: arm64v8/postgres:12-alpine: No such image: arm64v8/postgres:12-alpine

This image should be pulled:
https://hub.docker.com/layers/arm64v8/postgres/12-alpine/images/sha256-7afb318d4d27afce4ecfabee202d66e099b88668f1b719218e597dcb3d5c601a

Hi there!
Docker hub is supported for external images.
Does that issue persist? It could be a transient DockerHub issue, as I was unable to reproduce.

Indeed, just now it worked.

I have observed this issue several times already. I will keep an eye on it.

Thank you for letting us know, and please let us know if this happens for you again.

Best regards,

Heds

I’ve observed this same issue multiple times.

Now it also occurred for an image of one of the docker-compose services that was (docker-compose-)built in the same (balena-)build.

Log (I inserted some new-lines to make it more readable):

[zstack-zigbee-reader]  Successfully built 832dc9928038
...
[Info]                  Uploading images==========================================] 100%
[===>                                               ]
7%[Error]                 An error occured: (HTTP code 404) no such image - no such image: 832dc9928038:
No such image: 832dc9928038:latest

Hi @ftes, just to confirm does this issue resolve itself later or are you able to reliably reproduce this? If you can reproduce, could you send us an example so we can try here?

The issue occurs sporadically. Sometimes it happens, sometime it does not.

I have observed it both for images that should be pulled from hub.docker.com and for locally built images.

I don’t think an example will help due to the sporadic nature.

Thanks for the update; I am going to go ahead and create an issue for this so the builder team can take a look at it :+1:

It just happened again:

[Info]                  Uploading images==========================================] 100%
[========>                                          ] 16%[Error]                 An error occured: (HTTP code 404) no such image - no such image: arm64v8/postgres:12-alpine: No such image: arm64v8/postgres:12-alpine
[Info]                  Built on arm03
[Error]                 Not deploying release.

Maybe this is related? I often experience a “build has been cancelled” error:

[Error]                 This build has been cancelled
[Info]                  Built on arm01
[Error]                 Not deploying release.

Hi @ftes are you using git push or balena push. if you are using balena push you can set the DEBUG=1 environment variable, which might give us some more information to go on:

Linux or macOS:

DEBUG=1 balena push ...

Windows “cmd.exe” command prompt:

set DEBUG=1
balena push ...

Sorry for getting back to you so late.

I am using git push (running on TravisCI).

I just saw the error again, after some months of seeing no issues.

Hi there – thanks for letting us know about this. We’re investigating this, and hope to have an update for you shortly.

All the best,
Hugh

Same here.
From the last 5+ hours not even one deploy passes.

An error occured: (HTTP code 404) no such image - no such image

Hi @daniel.rusev can you provide us the logs with the errors? Is it again on the build step or is this from the device trying to pull an image?

Hi @mbalamat,
Yes sure.

Its on the build step

[Info] Generating image deltas from release f5c10bba083ce4f2cc97e863d9833dcc8702afab (id: 1362019)
[Warning] Failed to generate deltas due to an internal error; will be generated on-demand
[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - no such image: efcc3ae100ff: No such image: efcc3ae100ff:latest
[Info] Built on arm03
[Error] Not deploying release.

Ohh I see some delta generation error, I will route this internally. For now can you try disabling the delta updates from the device’s configuration page and tell us if the issue persists?

Hey @mbalamat,

Thanks for the quick response !

I’ve also initiate deployment with disabled delta updates :
All services says: [Success] Successfully built service

The result :

[Info] Uploading images
[Error] An error occured: (HTTP code 404) no such image - no such image: 754d150b4452: No such image: 754d150b4452:latest
[Info] Built on arm01
[Error] Not deploying release.

If there is anything I can help with, please let me know.

Thanks,

Daniel

Is there a way for us to reproduce this in order to figure out what’s wrong? Also can you provide the full log of the build if that’s possible?

also if you use the balena-cli are you able to get the build to complete successfully?

Hey,

From this morning everything is going on smoothly. Every action is successful.

Thanks,

Daniel