open-balena server on aarch64/arm64 possibility

Hi,

I’m new to balena and I “inherited” a system running in AWS, on an x86_64 instance.
I thought I’ll see if it is possible to migrate it to an Oracle Cloud’s Ampere (aarch64) instance.

However, I “ended up” with the “amd64” version of the balena/* images and for me it doesn’t seem that the arm64 images would exist.
(Nor found “system requirements” page where this [and other] requirements would be listed)

[6362d3f847b1] arm64    openbalena-cert-provider (latest) [35MB]
[ea6ffc89bfc0] arm64    openbalena-haproxy (latest) [22.8MB]
[c03be620fd3b] arm64    redis (alpine) [30.3MB]
[d3156fec8bcb] arm64    alpine (latest) [7.46MB]
[afed37f34212] amd64*   balena/open-balena-s3 (v2.13.10) [1.67GB]
[1733d3e22921] amd64*   balena/open-balena-db (v5.1.2) [376MB]
[ab09155c411b] amd64*   balena/open-balena-api (v0.209.2) [975MB]
[4557a3af0c41] amd64*   balena/open-balena-registry (v2.25.3) [751MB]
[d73322e957fc] amd64*   balena/open-balena-vpn (v11.4.6) [781MB]

Is there any approach to “reach my goal” and if so, what is the best?

Thanks!

Hello,

As far as I know, there’s no images for openBalena on aarch64 at the moment.

Why would you need to migrate to aarch64, is it just to migrate to Oracle cloud?
I’m not familiar with Oractle cloud, don’t they provide amd64 servers as well?

Hi! It indeed looked like for me that there are no arm64 balena images.
Since I don’t see any technical reason why is that, the way better questions are what is the “show-stopper” or whether it is intentional or not, because:

There are “mentions” of arm64 here and there, for example at:

open-balena-api/CHANGELOG.md at master · balena-io/open-balena-api · GitHub

has

 [6.9.6 - 2022-12-11]
 ### open-balena-base-14.1.3 - 2022-12-08
 * Build systemd variant for both amd64 and arm64 [Kyle Harding]

or if it is “intentional” then clarify this, like:

balena compose should complain on an aarch64 before pulling the amd64 images and then failing trying to run them

openBalena - Home

should reflect that the system requirement is an x86_64 (also adding minimum RAM requirement, ~2GB since we’re here)

And to answer your question/give more info on this case:
There are 3 servers in AWS, one is an aarch64 (used only to build the raspberry images for the ~80 PI devices occasionally).

Oracle has an “always free” part of their services, one can use an Ampere A1 [aarch64] 4*OCPU + 24GB RAM + 200 GB storage for free.
Since the builder needs aarch64 it would be an ideal situation to run balena too together on such an instance.
(and the other server on another, for example as two 2*OCPU + 12 GB instances)
[Oracle also have x86_64, but the always free one is just an AMD 1/4 OCPU + 2GB - which is way less usable in our case.]

So if it is clarified that there are no arm64 images (and won’t be), then my choice is either to grab the source and build the images myself or just go for an x86_64 instance.
Even though I like challenges, the second option looks way easier and faster to do… :expressionless:

There’s no intention to prevent running on aarch64, but there were no intention to make it work either.

Thing is we didn’t had the use-case before. Open-balena is the core of the balena-cloud infrastructure, as balena-cloud mostly runs on amd64 hardware and we have no reason to migrate to aarch64, we didn’t work on it (and won’t until it proves to be beneficial to do so).

Some services might actually runs on aarch64, other might have been intentionally made to run on aarch64 because it’s advantageous, but some might have requirements that makes it impractical.

If you decide to try, I’ll be following your results with great interest.

Actually I’m not just new to balena but to docker too, so it is like a blind and deaf guy trying to land with a Boeing 747, but after some patching, it seems to run fine.
I’ve started with the 3.8.0 so I went with the versions included in, namely:

    open-balena-api-0.209.2
    open-balena-db-5.1.2
    open-balena-registry-2.25.3
    open-balena-s3-2.13.10
    open-balena-vpn-11.4.6

For some reason, the dockerfile in:

    open-balena-registry-2.25.3 requires open-balena-base:v13.3.2
    open-balena-s3-2.13.10 requires open-balena-base:v13.4.0
    open-balena-vpn-11.4.6 requires open-balena-base:v13.3.1

and since I wasn’t really knowing what does what, I’ve built all 3 base image - where I had to “backport” the arm64 “awareness” (confd/nodejs) which are already in v14.1.3

I also had to add “ARG TARGETARCH” in “open-balena-s3-2.13.10”, also fix that the “GO_SHA256” variable is for the amd64 image.
(Question: is there any importance to check the sha256 here? It is not checked at the confd/nodejs download…)

I don’t know how to check if things are working, I’m trying to build the CLI, where npm install balena-cli still gives a lot of

22062 warn tar ENOENT: no such file or directory, open '/root/.nvm/versions/node/v12.22.12/lib/node_modules/.staging/fs-extra-2ed77084/lib/empty/index.js'
22063 warn tar ENOENT: no such file or directory, open '/root/.nvm/versions/node/v12.22.12/lib/node_modules/.staging/type-fest-8396db36/ts41/snake-case.d.ts'
22064 warn tar ENOENT: no such file or directory, open '/root/.nvm/versions/node/v12.22.12/lib/node_modules/.staging/typed-error-8cc29938/test/mocha.opts'

messages, and:
24063 error Error while executing:
24063 error /usr/bin/git ls-remote -h -t ssh://git@github.com/balena-io-modules/unbzip2-stream.git
[ node.js - npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/node_modules/cordova.git - Stack Overflow ]


# uname -a
Linux infoscreen-balena 5.15.0-1025-oracle #31-Ubuntu SMP Fri Nov 25 17:03:15 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

# ver
Distrib.: Ubuntu 22.04.1 LTS
Kernel: 5.15.0-1025-oracle #31-Ubuntu SMP Fri Nov 25 17:03:15 UTC 2022
Hardware: 1 x ARM Neoverse-N1 r3p1 (1), BogoMIPS: 50.00

# docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED       STATUS       PORTS                                                                                                             NAMES
8defaf0b2a05   openbalena-haproxy                    "/docker-entrypoint.…"   6 hours ago   Up 6 hours   0.0.0.0:80->80/tcp, :::80->80/tcp, 222/tcp, 3128/tcp, 5432/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 6379/tcp   openbalena-haproxy-1
28b81c3c1eb2   balena/open-balena-vpn:v11.4.6        "/usr/bin/entry.sh"      6 hours ago   Up 6 hours   80/tcp, 443/tcp, 3128/tcp                                                                                         openbalena-vpn-1
ac45f3d24579   balena/open-balena-registry:v2.25.3   "/usr/bin/entry.sh"      6 hours ago   Up 6 hours   80/tcp                                                                                                            openbalena-registry-1
958837c39b60   balena/open-balena-api:v0.209.2       "/usr/bin/entry.sh"      6 hours ago   Up 6 hours   80/tcp                                                                                                            openbalena-api-1
bf8b9c38eaba   openbalena-cert-provider              "/entry.sh /usr/src/…"   6 hours ago   Up 6 hours   80/tcp                                                                                                            openbalena-cert-provider-1
dac96801f2c6   balena/open-balena-s3:v2.13.10        "/usr/bin/entry.sh"      6 hours ago   Up 6 hours   80/tcp                                                                                                            openbalena-s3-1
714e2f2e67f3   balena/open-balena-db:v5.1.2          "/balena-entrypoint.…"   6 hours ago   Up 6 hours   5432/tcp                                                                                                          openbalena-db-1
dbae2c35386d   redis:alpine                          "docker-entrypoint.s…"   6 hours ago   Up 6 hours   6379/tcp                                                                                                          openbalena-redis-1
# docker images
REPOSITORY                    TAG        IMAGE ID       CREATED       SIZE
balena/open-balena-vpn        v11.4.6    4fea46f805f2   6 hours ago   750MB
<none>                        <none>     d74965a6eff2   6 hours ago   702MB
<none>                        <none>     be9de1c19d6d   6 hours ago   838MB
balena/open-balena-api        v0.209.2   50b094c01480   6 hours ago   944MB
balena/open-balena-s3         v2.13.10   4f99378c4cf1   6 hours ago   1.52GB
<none>                        <none>     d27c59874b19   6 hours ago   693MB
<none>                        <none>     39ef1311488e   6 hours ago   693MB
<none>                        <none>     1a02741c461d   6 hours ago   693MB
<none>                        <none>     193b88baf440   6 hours ago   693MB
<none>                        <none>     a719fefd43ca   7 hours ago   693MB
balena/open-balena-registry   v2.25.3    d32e6c84883c   7 hours ago   719MB
balena/open-balena-db         v5.1.2     01ef8cbf9afe   9 hours ago   357MB
balena/open-balena-base       v13.4.0    fbbf064b11cd   9 hours ago   693MB
balena/open-balena-base       v13.3.2    e64f2cf4d02a   9 hours ago   693MB
balena/open-balena-base       v13.3.1    af9d2dc0cf0b   9 hours ago   693MB
openbalena-cert-provider      latest     6362d3f847b1   3 days ago    35MB
openbalena-haproxy            latest     ea6ffc89bfc0   3 days ago    22.8MB
postgres                      14         25dc4447751f   5 days ago    357MB
debian                        bullseye   917ea2fdcbd6   6 days ago    118MB
redis                         alpine     c03be620fd3b   11 days ago   30.3MB
alpine                        latest     d3156fec8bcb   5 weeks ago   7.46MB

CLI works too - had to compile under a “non-root” account, and used an earlier version.
So seemingly everything is fine.

This is quite impressive.
Did you connect devices to your installation ?

It goes without saying but keep in mind that your installation is experimental if you intent on managing production device with it.