In the readme it states that Open Balena supports a subset of devices:
Supported devices: Raspberry Pi family, the Intel NUC, the NVIDIA Jetson TX2, and the balenaFin
What is exactly the limitation? Is this just to encourage people using the paid version or is there a technical impediment? Since the project is open source, how would one add more supported devices?
Similar questions:
Good question. The device information appears to be stored in the device types table and populated by open-balena-api migrations like src/migrations/00013-add-device-types.sql. The table contains many more devices, and I have seen openBalena expend considerable effort loading the massive contract column that appears to contain the device definitions:
select name, slug, length(contract) from "device type" order by name;
name | slug | length
--------------------------------------------------------+----------------------------------------+--------
Aetina N310 TX2 | n310-tx2 | 2875
Aetina N510 TX2 | n510-tx2 | 2875
AIO 3288C | aio-3288c | 11455
AM571X EVM | am571x-evm | 13522
Apalis iMX6q | apalis-imx6q | 7695
Asus Tinker Board | asus-tinker-board | 4929
Asus Tinker Board S | asus-tinker-board-s | 6066
ASUS Tinker Edge T | asus-tinker-edge-t | 4902
Auvidea CNX100 Xavier NX | cnx100-xavier-nx | 5903
Auvidea JN30B Nano | jn30b-nano | 7305
Balena Fin (CM3) | fincm3 | 4028
BananaPi-M1+ | bananapi-m1-plus | 10028
BeagleBoard-XM | beagleboard-xm | 9096
BeagleBone Black | beaglebone-black | 10482
BeagleBone Green | beaglebone-green | 11926
BeagleBone Green Gateway | beaglebone-green-gateway | 20456
BeagleBone Green Wireless | beaglebone-green-wifi | 20451
Colibri iMX6dl | colibri-imx6dl | 7799
Compulab IOT-gate-imx8 | iot-gate-imx8 | 5128
Compulab MX8M | cl-som-imx8 | 6643
Coral Dev Board | coral-dev | 1369
CTI Astro TX2 G+ | astro-tx2 | 4346
CTI Orbitty TX2 | orbitty-tx2 | 4353
CTI Photon Nano | photon-nano | 5844
CTI Photon TX2 NX | photon-tx2-nx | 5844
CTI Photon Xavier NX | photon-xavier-nx | 5723
CTI Spacely TX2 | spacely-tx2 | 4353
Cybertan ZE250 | cybertan-ze250 | 8285
Digi ConnectCore 8X SBC Pro | ccimx8x-sbc-pro | 28709
Edge Device Builder | edge | 1172
Etcher Pro | etcher-pro | 4905
FireFly rk3288 | firefly-rk3288 | 11470
Floyd Nano BB02A eMMC | floyd-nano | 3220
Generic | generic | 1178
Generic AARCH64 | generic-aarch64 | 3164
Generic AMD64 (x86-64) | generic-amd64 | 1789
Generic AMD64 (x86-64) with disk encryption | generic-amd64-fde | 1802
Generic ARMv7-a HF | generic-armv7ahf | 2233
Generic x86_64 | genericx86-64-ext | 6276
Hummingboard | hummingboard | 3729
Intel Edison | intel-edison | 4912
Intel NUC | intel-nuc | 4927
I-Pi SMARC PX30 SD-CARD | smarc-px30 | 3162
Microsoft Surface 6 | surface-pro-6 | 1831
Microsoft Surface Go | surface-go | 1827
NanoPC-T4 | nanopc-t4 | 4750
Nanopi Neo Air | nanopi-neo-air | 3054
NanoPi R2C | nanopi-r2c | 12933
NanoPi R2S | nanopi-r2s | 2025
Nitrogen 6x | nitrogen6x | 5048
Nitrogen 6X Quad 2GB | nitrogen6xq2g | 5063
Nitrogen8M Mini SBC | nitrogen8mm | 6818
NPE X500 M3 | npe-x500-m3 | 17425
Nvidia blackboard TX2 | blackboard-tx2 | 2513
Nvidia Jetson Nano 2GB Devkit SD | jetson-nano-2gb-devkit | 3258
Nvidia Jetson Nano eMMC | jetson-nano-emmc | 3234
Nvidia Jetson Nano SD-CARD | jetson-nano | 3231
Nvidia Jetson TX1 | jetson-tx1 | 2321
Nvidia Jetson TX2 | jetson-tx2 | 2881
Nvidia Jetson TX2 NX (with Xavier NX Devkit) | jetson-tx2-nx-devkit | 2925
Nvidia Jetson Xavier | jetson-xavier | 3121
Nvidia Jetson Xavier NX Devkit eMMC | jetson-xavier-nx-devkit-emmc | 3166
Nvidia Jetson Xavier NX Devkit SD-CARD | jetson-xavier-nx-devkit | 3161
Nvidia Jetson Xavier NX Devkit SD Seeed ReSpeaker-2Mic | jetson-xavier-nx-devkit-seeed-2mic-hat | 3207
ODROID-C1+ | odroid-c1 | 16182
ODROID-XU4 | odroid-xu4 | 15810
Orange Pi Lite | orange-pi-lite | 11459
Orange Pi One | orange-pi-one | 12093
Orange Pi Plus2 | orangepi-plus2 | 13052
Orange Pi Zero | orange-pi-zero | 12096
Parallella | parallella | 84688
PocketBeagle | beaglebone-pocket | 12577
QEMU X86 32bit | qemux86 | 15723
QEMU X86 64bit | qemux86-64 | 18858
Radxa Zero | radxa-zero | 1358
Raspberry Pi 2 | raspberry-pi2 | 6833
Raspberry Pi 3 | raspberrypi3 | 7182
Raspberry Pi 3 (using 64bit OS) | raspberrypi3-64 | 7497
Raspberry Pi 400 | raspberrypi400-64 | 8242
Raspberry Pi 4 (using 64bit OS) | raspberrypi4-64 | 6393
Raspberry Pi CM4 IO Board | raspberrypicm4-ioboard | 6401
Raspberry Pi (v1 / Zero / Zero W) | raspberry-pi | 5787
Raspberry Pi Zero 2 W (64bit) | raspberrypi0-2w-64 | 5785
Revolution Pi Connect | revpi-connect | 4601
Revolution Pi Core 3 | revpi-core-3 | 12102
ROCK Pi 4B | rockpi-4b-rk3399 | 1373
ROCKPro64 | rockpro64 | 3101
Rocktech ISG 503 | isg-503 | 6856
RushUp Kitra 520 | kitra520 | 1777
RushUp Kitra 710 | kitra710 | 1371
Samsung Artik 10 | artik10 | 4091
Samsung Artik 520 | artik5 | 3844
Samsung Artik 530 | artik530 | 6449
Samsung Artik 530s 1G | artik533s | 6723
Samsung Artik 710 | artik710 | 2420
Seeed ODYSSEY-X86 | odyssey-x86 | 3869
Siemens IOT2000 | iot2000 | 7201
Technologic TS-4900 | ts4900 | 13765
UP Board | up-board | 1597
UP Core | up-core | 6373
UP Core Plus | up-core-plus | 6509
UP Squared | up-squared | 5436
Variscite DART-6UL | imx6ul-var-dart | 6528
Variscite DART-MX8M | imx8m-var-dart | 3310
Variscite DART-MX8M Mini | imx8mm-var-dart | 4379
Variscite VAR-SOM-MX6 | var-som-mx6 | 5447
Variscite VAR-SOM-MX7 | imx7-var-som | 3785
VIA VAB 820-quad | via-vab820-quad | 1964
Zynq ZC702 | zc702-zynq7 | 104609
The contract contains most of the parts that I would consider critical:
{"slug":"imx7-var-som","version":"1","type":"hw.device-type","aliases":["imx7-var-som"],"name":"Variscite VAR-SOM-MX7","assets":{"logo":{"url":"d
ata:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0<LONG_STRING>3ZnPgo=","name":"logo"}},"data":{"arch":"armv7hf","hdmi":false,"le
d":true,"connectivity":{"bluetooth":true,"wifi":true},"storage":{"internal":true},"media":{"installation":"sdcard"},"is_private":false}}
If there is some fundamental limitation on the openBalena side, I would expect it to involve architecture or maybe some limitation of the devices’ BalenaOS images.
Just to layer in here - we have included the ability to add and manage custom device types in open-balena-admin. Just be aware that for openbalena to accept apps for this device type, you need to also add a “device type slug” if you are using a more recent version of openbalena (also possible to do with open-balena-admin). We have been pushing apps for custom device types to our openbalena server for a while and it works great!
Got the following answer on the github issue about the same topic:
There’s no limitation – in theory openBalena supports the same set of devices balenaCloud does. In practice, we cannot advertise support for all of them because it’s currently impractical to test them all out like we do for balenaCloud. This will hopefully change in the future. What device are you interested in?
how would one add more supported devices?
Adding support for new devices is non-trivial – irrespective of whether it’s for openBalena or balenaCloud. The first step would be to get balenaOS running on the device and the quickest way to do that is to start from a balenaOS variant that appears to be most compatible to the device you want to add support for. You can get quite far by masquerading as a device that is currently supported as it trivially solves the problem of connecting your custom balenaOS to the backend. That’s all off the top my head. For more information, the best approach is to use the forums where you can get hold of the OS team to help you along the way.
I can see how to add a device type, but it does not seem to be working. Maybe i need to add a device type slug first? How to do that?
What is the error you are seeing when trying to add a device type? Device type aliases were not added until open-balena-api v0.185.0, so depending which version you are running you may not see that option in open-balena-admin.
I’m not sure if you meant you were adding a device-type in the open-balena-admin UI managed by harmoni.io, or if you were adding it to openBalena generally…
If the latter, I wanted you to know that we are actually in the process of updating some of our documentation around contributing device-type support. Could you tell me which page you’re referencing for it? The current documentation we keep fully updated is this one: meta-balena/contributing-device-support.md at master · balena-os/meta-balena · GitHub, but I know of some others (including this one) which we are working to deprecate and move.
So in short, maybe the info you need is in the main document I referenced. Else, can you share more about how the work you’ve done so far is “not working” and maybe we can narrow down the step you’re struggling with. Feedback around how to make this process more clear / easier is really helpful, and it is good timing for it, so any comments you have or places you are stumbling is good insight for us!
I’ve managed to build the OS and it’s running fine. Now I want to hook up my device to my open-balena instance. When I create my custom device type using open-balena-admin, I’m getting some errors (will create a github issue for this), but it seems to create the device type.
What would be the correct way to add my custom device type to open-balena? Do I follow steps 2 and 3 from the link you provided, sending to contract over to Balena, even for open-balena support?
Update:
You need to have contracts in place for your custom device in the GitHub - balena-io/contracts: Balena.io Base Contracts repository. However, you can override this using the following env variables in open-balena:
CONTRACTS_PUBLIC_REPO_OWNERCONTRACTS_PUBLIC_REPO_NAMECONTRACTS_PUBLIC_REPO_BRANCHCONTRACTS_PRIVATE_REPO_OWNERCONTRACTS_PRIVATE_REPO_NAMECONTRACTS_PRIVATE_REPO_BRANCHCONTRACTS_PRIVATE_REPO_TOKENOpen Balena will update the contracts periodically and add new devices automatically.
However the api does not support all commands with custom devices. More on this, here: Support for custom device types · Issue #474 · balena-io/open-balena-api · GitHub
@mpous Yes! Thanks to @markdegroot I pushed this change yesterday, however this was just to suppress an incorrect error message, as otherwise the device was being created successfully via open-balena-admin.
@markdegroot can you clarify what specifically isn’t working after you add the custom device type via the admin interface? We have been adding custom device types, creating devices that use them and pushing applications to them without issue for a while now - curious if you are trying to do something else? We didn’t have to mess with contracts to do any of the above.
I’ve made some progress. Will write an update soon™
So you’ve managed to build BalenaOS for your custom device and would like to connect this device to OpenBalena? Here’s how I managed to get this working:
In order to use custom device types in OpenBalena we need to tell OpenBalena about these device types. For this to work, we need to provide two different types of information:
For OpenBalena to even know about our custom device type, we need to provide a so-called contract for our device type. OpenBalena periodically downlods the contracts from one or two Github repositories. Once the contract has been downloaded, it is possible to create devices and fleets for this custom device type.
Like I said, OpenBalena downloads the device contracts from one or two Github repositories. Out of the box OpenBalena downloads the device type contracts from the contracts folder in GitHub - balena-io/contracts: Balena.io Base Contracts.
Additionally you can add a second Github repository. This can be done using the following optional ENV variables:
| Variable | Description |
|---|---|
| CONTRACTS_PRIVATE_REPO_OWNER | Github organization |
| CONTRACTS_PRIVATE_REPO_NAME | Github repository |
| CONTRACTS_PRIVATE_REPO_BRANCH | Github branch |
| CONTRACTS_PRIVATE_REPO_TOKEN | Github access token |
Both the public and private repository can be configured simultaniously. As you can see, the private repo can be configured with an access token and does not have to be publically accessible.
In order to configure the ENV variables, we need to update the docker-compose file for the api service. This file can be found in compose/services.yaml.
Add the following lines underneath services:/api:/environment:
CONTRACTS_PRIVATE_REPO_OWNER: ${OPENBALENA_CONTRACTS_PRIVATE_REPO_OWNER}
CONTRACTS_PRIVATE_REPO_NAME: ${OPENBALENA_CONTRACTS_PRIVATE_REPO_NAME}
CONTRACTS_PRIVATE_REPO_BRANCH: ${OPENBALENA_CONTRACTS_PRIVATE_REPO_BRANCH}
CONTRACTS_PRIVATE_REPO_TOKEN: ${OPENBALENA_CONTRACTS_PRIVATE_REPO_TOKEN}
and configure the actual values by adding to config/activate:
export OPENBALENA_CONTRACTS_PRIVATE_REPO_OWNER=...
export OPENBALENA_CONTRACTS_PRIVATE_REPO_NAME=...
export OPENBALENA_CONTRACTS_PRIVATE_REPO_BRANCH=...
export OPENBALENA_CONTRACTS_PRIVATE_REPO_TOKEN=...
The added Github repository should at least contain the folder contracts/hw.device-type. For every device there should be a folder with the device type slug as name containing at least a contract.json and a logo svg file. Use an existing device type contract as reference.
More information about adding hardware contracts can be found here: meta-balena/contributing-device-support.md at master · balena-os/meta-balena · GitHub
After configuring the public and/or private repos, restart OpenBalena using:
./scripts/compose down
./scripts/compose up -d
and check the logs for the api service using:
./scripts/compose exec api journalctl -fn100
Contracts are refreshed every 5 minutes, so after a while you should see something like this:
Jun 10 14:25:00 5d0a0b8810a2 api[1456]: [Scheduler] Running job: contractSync
Now all new devices in your contracts repo are available to OpenBalena. You can now create a fleet for your custom device type:
balena fleet create MyCustomDeviceFleet --type my-custom-device-type
However, when you want to create a device configuration or configure a device image by running:
balena config generate --fleet MyCustomDeviceFleet --version 2.12.7
this fails with the following error:
Device type slug not recognized. Perhaps misspelled?
Check available device types with "balena devices supported"
This is not due to Device type slug not recognized like the error says, but because we do not have the device type information yet. We’ll configure this next.
Besides the contracts, a lot of commands in OpenBalena require yet another json file for custom devices to work. This information is loaded from an AWS S3 bucket which also contains the BalenaOS images for all devices.
Out of the box, OpenBalena downloads the device device-type.json file from https://s3.console.aws.amazon.com/s3/buckets/resin-production-img-cloudformation. Obviously our custom image information is not present there, so just like the Contracts, we need to reconfigure this.
In order to configure the S3 bucket where device images are stored, we need to change a couple of ENV variables again:
| Variable | Description |
|---|---|
| IMAGE_STORAGE_ENDPOINT | S3 server endpoint, defaults to s3.amazonaws.com
|
| IMAGE_STORAGE_BUCKET | S3 bucket name, defaults to resin-production-img-cloudformation
|
| IMAGE_STORAGE_PREFIX | Folder inside the bucket, defaults to images
|
| IMAGE_STORAGE_ACCESS_KEY | S3 Access key, default empty |
| IMAGE_STORAGE_SECRET_KEY | S3 Secret key, default empty |
| IMAGE_STORAGE_FORCE_PATH_STYLE | Setting whether to force path style URLs for S3 objects |
For this setting there is no way to configure a secondary bucket, so the only approach is to overwrite the entire setting. Now we could set up our own AWS S3 bucket, clone the resin-production-img-cloudformation bucket (6.5 TB total, images folder is 3.8 TB) and add our own custom device, but I chose a different approach.
At the moment it’s not possible to download device images using the balena os download command, so we don’t need the actual image files. Just the device-type.json files are enough. Let’s create a local S3 bucket with just the device-type.json files for devices we might use and add our custom devices to that. OpenBalena already runs a S3 service to store the docker images, so we can use that and create a new bucket there.
In order to do this, and also set-up an easy way to copy device-type files from the resin-production-img-cloudformation bucket to our own, we’ll need to add some variables to the compose/services.yaml.
Under services:/api:/environment: replace
IMAGE_STORAGE_BUCKET: resin-production-img-cloudformation
IMAGE_STORAGE_PREFIX: images
IMAGE_STORAGE_ENDPOINT: s3.amazonaws.com
with
IMAGE_STORAGE_BUCKET: ${OPENBALENA_IMAGES_S3_BUCKET}
IMAGE_STORAGE_PREFIX: images
IMAGE_STORAGE_ENDPOINT: s3.${OPENBALENA_HOST_NAME}
IMAGE_STORAGE_FORCE_PATH_STYLE: true
IMAGE_STORAGE_ACCESS_KEY: ${OPENBALENA_S3_ACCESS_KEY}
IMAGE_STORAGE_SECRET_KEY: ${OPENBALENA_S3_SECRET_KEY}
Next, we’ll edit config/activate and change:
export OPENBALENA_S3_BUCKETS="registry-data"
to
export OPENBALENA_S3_BUCKETS="registry-data;images-data"
export OPENBALENA_IMAGES_S3_BUCKET=images-data
and restart OpenBalena with:
./scripts/compose down
./scripts/compose up -d
If you want to use OpenBalena not only with your own custom device, but also use supported BalenaOS devices, we can synchronize the device-type json files from the resin-production-img-cloudformation bucket to our own.
For this you’ll need to have an AWS (free) account and create an AWS Access key and Secret key using: Understanding and getting your AWS credentials - AWS General Reference
Edit compose/services.yaml and under services:/s3:/environment: add:
MC_HOST_s3: https://${OPENBALENA_AWS_ACCESS_KEY}:${OPENBALENA_AWS_SECRET_KEY}@s3.amazonaws.com
and add the following to config/activate (fill your keys):
export OPENBALENA_AWS_ACCESS_KEY=...
export OPENBALENA_AWS_SECRET_KEY=...
Restart OpenBalena with:
./scripts/compose down
./scripts/compose up -d
and run the following commands to syncronize for example raspberrypi4-64 device image version 2.98.33 from resin-production-img-cloudformation to your own bucket:
./scripts/compose exec s3 mc cp -r s3/resin-production-img-cloudformation/images/raspberrypi4-64/2.98.33/device-type.json localhost/images-data/images/raspberrypi4-64/2.98.33
To add your custom device to the images bucket you’ll need to add a file called device-type.json to the folder images/[my-custom-device-slug]/[my-custom-device-balenaos-version]. We can do this using the mc command we’ve used above:
cat my-custom-device-type.json | ./scripts/compose exec -T s3 mc pipe localhost/images-data/images/my-custom-device-type/my-custom-device-version/device-type.json
The my-custom-device-type.json file is generated when building the BalenaOS image. It’s called the same as the .coffee file you used and located in the same folder.
Now the command that failed above:
balena config generate --fleet MyCustomDeviceFleet --version 2.12.7
should generate a valid device configuration.
So, in summary, to add your custom device to OpenBalena you’ll need to:
@markdegroot this is very impressive work! Thank you for sharing it in such detail with the community.
Based on the above, it appears that open-balena-api sets up the initial devices and aliases in the database using the contracts stored in balena.io/contracts repo, which solves one of the bigger questions I had when developing open-balena-admin - “how did these device types get here in the first place?”. And you’ve identified the contractSync process within open-balena-api that is doing this which makes perfect sense.
I have some questions based on this that would help us with advancing the open-balena-admin project:
It looks like the contract and svg file are simply stored in the “device type” table within the openbalena database. While it would be architecturally consistent with balena cloud to create a github repo with contracts and images, and have them sync automatically via the contractSync process, it seems to also work by simply adding an entry to the “device type” table manually. And theoretically the contract and svg could be included directly as well, although we don’t yet support this in open-balena-admin. Based on your understanding of it, can you see any reason why this approach wouldn’t work? The repo seems a bit overkill to manage a json file with a handful of settings, and an image (which I don’t think is used in openbalena anyhow, although I suppose the image could be displayed in open-balena-admin.
I might be able to help automate the second piece. With a handful of tweaks implemented using a wrapper shell script, we have managed to utilize the balena jenkins build process to deploy our balena-os builds directly to our openbalena s3 server. It creates all of the files you identified that exist in the cloudformation s3 repo automatically with each build. We then have a custom process that runs on our balena-os image which uses the s3 repo for OTA updates of balena-os using the auto deployed images on the s3 server. We are planning to release this whole architecture as a community project, but it’s not ready yet (though if you are interested send me a private message and I’d be happy to share it). The device config architecture is one of the last outstanding pieces. When I look at the contents of the device-type.json file, it appears to just define the initial wifi networks and the base settings for supervisor (polling update interval, etc.). We handle these configs using helper recipes in our OS build process, but I can imagine that some users want to configure these separately for each deployment of the OS. Am I correct in speculating that the only place the device-type.json file is used is by balena-cli which just takes the few paramters in there and creates default wifi network connections in the image and then configures some default parameters for the supervisor? If so we might be able to replicate this functionality in open-balena-admin when downloading an image.