Multiple device types within a single fleet

I have a fleet of raspberry pi4’s that work great.

Recently I’ve started to experiment with adding raspberry pi5. Therefore i created an image based for the pi5 and that also seems to work.

However I couldn’t get the vcgencmd utility to work. So therefore I was thinking maybe the docker file is out of date. And yes - even when I use FROM balenalib/%%BALENA_MACHINE_NAME%%-ubuntu-node:20.12-jammy-build-20240429 AS base in my Dockerfile.template I noticed that the builds are still Step 1/37 : FROM balenalib/raspberrypi4-64-ubuntu-node:20.12-jammy-build-20240429 AS base

Now I am guessing the builder does not build multiple images even when multiple device architectures are in the fleet. This seems to be a unsupported feature a few years ago (Suggested workarounds for multi-architecture fleets?)

What is the correct way to address this situation? In other word; How do I properly deploy multiple device types within a single fleet?

Hello @bas that’s a good question!

In the meantime I ask internally, what difference are you expecting to have if the base image runs for Pi 5 than Pi 4? Is there anything specific? (e.g. computation power of the device type?)

Thanks

Well the pi5 supports ssd via nve ports. So more reliable and faster storage is the main reason we are looking into this. Boot time goes from 1.5 minutes to 35 seconds. Image updates are faster too.

Also it should be better at GPU tasks (though I haven’t gotten that working yet) which drive our UI.

We now run up two 3d printers of one pi4, but are considering upping that to 4. Also adding camara’s. So yeah could really use the performance boost.

So supporting this in one fleet would be awesome.

1 Like

Thanks for sharing more context @bas

Let me ask internally!

Let’s stay connected

@bas after speaking internally, if the concern is that the base image for the pi4 will perform differently on a pi5, that is very unlikely.

There shouldn’t be any difference at all, especially for Ubuntu. We probably don’t even inject the RPi proprietary tools.

Related with the base images, all of the raspberry Pi X base images are essentially copies of balenalib/aarch64-debian so it all ends up being included.

NVMe and GPU are handled from the OS, so there is not much to do from the application containers.

Did you find any difference of perfomance?

Ok that’s good to hear. I do wonder if Balena will ever officially support multiple images targeted at multiple devices types inside a fleet but indeed I was mainly concerned about our own fleet. So if that point it void, fine.

The only remaining difference that I can’t seem to figure out is the that the vcgencmd utility does not work on a 5.
vcgencmd is installed in raspberrypi4-64 but maybe it is too old. Installing a more recent version did not work for me… But it is a minor point and I will be advising our team to switch to pi5 soon. We are about to jump into production and ramp up devices, so better do that now.

hello @bas
perhaps you can share more details?

  • what version of vcgencmd command is running in the container? you can find out using vcgencmd version but opening a terminal in the respective service. more details here
  • what specific command do want to run with vcgencmd?
  • you mentioned that it did not work. can you share what was your expected outcome and what actually happens? perhaps you have some error logs or something else that can help us better understand?

thanks

1 Like

I install package ‘libraspberrypi-bin’

on a pi4 this works

controller@fa94442da258:/project/apps/backend$ vcgencmd version
May  9 2023 12:16:34 
Copyright (c) 2012 Broadcom
version 30aa0d70ab280427ba04ebc718c81d4350b9d394 (clean) (release) (start)
controller@fa94442da258:/project/apps/backend$ vcgencmd display_power 1
display_power=1
controller@fa94442da258:/project/apps/backend$ 

on a pi5 not so much

controller@6a421640ad3d:/project/apps/backend$ vcgencmd version
VCHI initialization failed
controller@6a421640ad3d:/project/apps/backend$ vcgencmd display_power 1
VCHI initialization failed
controller@6a421640ad3d:/project/apps/backend$ 

Our app only uses vcgencmd get_throttled atm moment as a health check.

Dockerfile

## Base image
# FROM balenalib/%%BALENA_MACHINE_NAME%%-ubuntu-node:18.14-jammy-build-20240105 AS base 
FROM balenalib/%%BALENA_MACHINE_NAME%%-ubuntu-node:20.12-jammy-build-20240429 AS base

RUN npm install -g npm@10.8.0 pnpm@8
RUN apt update && apt install --no-install-recommends -y \
  # to be able to fetch from private repos
  git openssh-client \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* \
  && rm -Rf /root/.cache

## Cache image

FROM base AS cache 

WORKDIR /cache/

COPY --chown=1001:1001 project/.npmrc .npmrc
COPY --chown=1001:1001 project/.env.production .env.production
COPY --chown=1001:1001 project/package.json package.json
COPY --chown=1001:1001 project/pnpm-lock.yaml pnpm-lock.yaml
COPY --chown=1001:1001 project/pnpm-workspace.yaml pnpm-workspace.yaml
COPY --chown=1001:1001 project/apps/backend/package.json apps/backend/package.json
COPY --chown=1001:1001 project/apps/frontend/package.json apps/frontend/package.json
COPY --chown=1001:1001 project/apps/nfc-reader/package.json apps/nfc-reader/package.json
COPY --chown=1001:1001 project/packages/@3dlayermaker-common/package.json packages/@3dlayermaker-common/package.json
COPY --chown=1001:1001 project/packages/@3dlayermaker-mcu-profiles/package.json packages/@3dlayermaker-mcu-profiles/package.json
COPY --chown=1001:1001 project/packages/@3dlayermaker-printers/package.json packages/@3dlayermaker-printers/package.json
COPY --chown=1001:1001 project/packages/@3dlayermaker-qrcode/package.json packages/@3dlayermaker-qrcode/package.json
COPY --chown=1001:1001 project/packages/tsconfig/package.json packages/tsconfig/package.json

COPY --chown=1001:1001 ssh-agent.js ssh-agent.js
RUN mkdir -p /root/.ssh && chmod 700 /root/.ssh
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts

# Filter currently does not work properly with v8 default config, you have to implicitly set dedupe-peer-dependents to false to have that work. For more info and progress please refer to https://github.com/pnpm/pnpm/issues/6300
RUN pnpm config --location=project set dedupe-peer-dependents=false

# this does deploy-key mappings and then an pnpm install
# COPY /.balena/secrets/ssh-private-keys /run/secrets/ssh-private-keys
RUN set -e && node ssh-agent.js "$(cat '/run/secrets/ssh-private-keys')" 

## Application image

FROM balenalib/%%BALENA_MACHINE_NAME%%-ubuntu-node:20.12-jammy-run-20240429 AS app 

RUN apt update && apt install --no-install-recommends -y \
  # to be able to use vcgencmd
  libraspberrypi-bin \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* \
  && rm -Rf /root/.cache

RUN groupadd --gid 1001 controller \
  && useradd --uid 1001 --gid controller --shell /bin/bash -m controller

# VCHI (VideoCore Host Interface) is a low-level inter-process communication interface used on some platforms that incorporate VideoCore multimedia processors.
# VideoCore is a series of SoC (System on Chip) processors designed by Broadcom, and it's primarily known for its use in multimedia applications, such as
# those found in mobile devices, digital set-top boxes, and the popular Raspberry Pi microcomputers.
# The VCHI interface is part of the broader architecture that allows the host CPU (such as an ARM processor in the Raspberry Pi) to communicate with the 
# VideoCore co-processor. This communication is crucial for initializing and managing multimedia tasks like video playback, 3D graphics 
# rendering, and camera input processing.
#
# In order to access the vcgencmd utiluty, the user needs to be a member of the video group.
#
RUN usermod -aG video controller

RUN npm install -g npm@10.8.0 pnpm@8

COPY --chown=1001:1001 --from=cache /cache/ /project
COPY --chown=1001:1001 project /project

# Inform the container that udev support is active or required, which could influence the behavior
# of contained applications or services in terms of device management
ENV UDEV=1

WORKDIR /project/apps/backend
COPY --chown=1001:1001 docker-entrypoint.sh /
RUN mkdir -p /project/apps/backend/data \
  && chown -R 1001:1001 /project/apps/backend/data

USER 1001
ENV NODE_ENV=production

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["start:docker"]

after I switched from ubuntu to debian and installed GitHub - raspberrypi/utils: A collection of scripts and simple applications manually I am able to use the vcgencmd.

1 Like