Balena Cloud builds - Only first multi container seems to cache

Product support
1

Some rather odd behaviour. When I have three components to my Dockerfile (2 build processes and 1 final process) like so the cache doesn’t work on the second one:

FROM alpine as 1 <-- this one always builds from Balena cache. 

...
WORKDIR /test
...

FROM alpine as 2 <-- this one starts building from Balena cache, but then stops after the WORKDIR stage

...

WORKDIR /test

... 

FROM alpine <-- this uses cache ok. 

COPY --from=1 ...
COPY --from=2 ...

Only the first of the build steps uses the cache when I push to Balena Cloud via the CLI. I have yarn builds in each so it adds a lot of time to the build process. When I reverse the order of the Dockerfile, i.e. I put the process from 2 at the top of the file, then that one builds ok and the one then second in the Dockerfile (process 1) doesn’t use the cache, so seems to be related to the order in the file rather than the content of each build step.

Here is some outputs from the builds process:

[Info]              Pulling previous images for caching purposes...
[balena-device-ui]  Step 1/24 : FROM node:16.13.2-alpine3.15 as UI
[balena-device-ui]   ---> 492edd4910ad
[balena-device-ui]  Step 2/24 : ENV NODE_ENV=development
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 2db69c8ad597
[balena-device-ui]  Step 3/24 : WORKDIR /ui
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 821bdd41ce46
[balena-device-ui]  Step 4/24 : COPY ui/package.json package.json
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 4a7623ded13c
[balena-device-ui]  Step 5/24 : COPY ui/yarn.lock yarn.lock
[balena-device-ui]  Using cache
[balena-device-ui]   ---> f9bafe3ef9a0
[balena-device-ui]  Step 6/24 : RUN yarn install --network-timeout 600000 --frozen-lockfile
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 3ef0d01d00ef
[balena-device-ui]  Step 7/24 : COPY ui .
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 2e5007829e24
[balena-device-ui]  Step 8/24 : RUN tar -cf /ui.tar .
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 18817a55e14c
[balena-device-ui]  Step 9/24 : FROM node:16.13.2-alpine3.15 as ExpressJS
[balena-device-ui]   ---> 492edd4910ad
[balena-device-ui]  Step 10/24 : ENV NODE_ENV=development
[balena-device-ui]  Using cache
[balena-device-ui]   ---> 2db69c8ad597
[balena-device-ui]  Step 11/24 : WORKDIR /ui
[balena-device-ui]   ---> Running in 23821a77e031
[balena-device-ui]  Removing intermediate container 23821a77e031
[balena-device-ui]   ---> a39244ecdb00
[balena-device-ui]  Step 12/24 : COPY expressjs/package.json package.json
[balena-device-ui]   ---> 9e237514e687
[balena-device-ui]  Step 13/24 : COPY expressjs/yarn.lock yarn.lock
[balena-device-ui]   ---> e530efb1b940
1 Like
3

This is interesting, @maggie0002. Have you tried and replicated this since then? Do you maybe have a sample dockerfile I can try and test this with?

5

I haven’t tested it since the post. Unfortunately I do not use those same Dockerfiles anymore either, they evolved in to only 1 multi-stage process. It should be fairly easy to replicate through, if I recall correctly I had used a simple example like the one I provided above for testing:

FROM alpine as 1 <-- this one always builds from Balena cache. 

COPY `add a test copy`
RUN `or maybe an install process here`
WORKDIR /test
COPY `add a test copy`
RUN `or maybe an install process here`

FROM alpine as 2 <-- this one starts building from Balena cache, but then stops after the WORKDIR stage

COPY `add a test copy`
RUN `or maybe an install process here`
WORKDIR /test
COPY `add a test copy`
RUN `or maybe an install process here`

FROM alpine <-- this uses cache ok. 

COPY --from=1 ...
COPY --from=2 ...