Docker multi-stage builds all targets

Product support
1

We have a project with a multi-stage dockerfile that includes targets for both our arm64 IoT devices and x86 tooling (cross compiler image).

When I submit this to Balena cloud builders using the balena CLI, it looks like all of the targets in the multi-stage dockerfile are being built - even if they are not a dependency of the target specified in our docker-compose file. This is both slow and actually fails due to an incompatibility with the target platform for our x86 cross compiler install scripts.

According to docker, if you are using buildkit, then only the stages that the target stage depends on are built. Do the Balena cloud builders use buildkit?

The following is a simplified version of our dockerfile:

ARG BASE_OS=linux
ARG BASE_ARCH=aarch64

#-------------------------------------------------------------------------------
# Install the setup directory

FROM ubuntu:bionic AS tools
COPY . /setup
RUN /setup/download_some_stuff.sh

#-------------------------------------------------------------------------------
# Base Image

FROM --platform=$BASE_OS/$BASE_ARCH ubuntu:bionic AS base
COPY --from=tools /setup /setup
RUN /setup/install_base_packages.sh

#-------------------------------------------------------------------------------
# Target Sysroot (for cross compilation)

FROM base AS sysroot-tar
RUN /setup/scripts/balena/export-sysroot.sh
RUN /setup/blobs/tegra/extract-xavier-sysroot.sh /sysroot.tar /target-root


#-------------------------------------------------------------------------------
# Cross compiler image

FROM ubuntu:bionic AS crossbuild-x86
COPY --from=sysroot-tar /target-root /target-root
COPY --from=tools /setup /setup
RUN /setup/install_cross_compiler_stuff.sh

#-------------------------------------------------------------------------------
# Main Balena Application Image

FROM base AS application-base
RUN /setup/remove_unnecessary_files.sh && rm -rf /setup

#-------------------------------------------------------------------------------
# Much smaller final image

FROM scratch AS application
COPY --from=application-base / /
CMD ["/system-init"]

We specify “application” as our target in the docker-compose.yml file but the crossbuild-x86 target is also being built. If buildkit was being used I would expect that only the following should be built:

  • tools
  • base
  • application-base
  • application

Is there any way I can ensure that only the required targets are being built by the Balena cloud builders?

2

Hi, unfortunately we don’t yet use buildkit on the builders - the balenaEngine does have the support enabled but until recently dockerode, which we use to interface with it, did not support it.

1 Like