Hi @fisehara are there any further updates on progress or timeline?
Thanks,
Gerard.
Hello Gerard,
for now we have no further progress or timeline to communicated.
Best Regards
Harald
Letâs make sure Host OS updates happens for openBalena. Please upvote here:
Hello
Iâve got a few questions regarding the manual updating of balenaos and the accompanying supervisor.
I succeeded in upgrading manually using the update scripts in the OS. The only problem now is where to find the latest upgrade images, are they only part of the balena-cloud offering or is there also a possibility to open these up for openbalena?
Having it automated is probably still a bit of work, which is completely understandable, but having these images somewhere public would already help tremendously. I have found a few images on docker-hub (resinos) but it seems this isnât being used/actively maintained anymore.
Even a bit of guidance in how to build these images would be of great help.
Regards
GoogleIt
Hi @GoogleIt, are you building your own OS and looking to push it to your devices, or looking to update your devices from the public balena-cloud images?
We are using our own OS images, and have implemented automated host OS updating using openbalena by having these custom images automatically push to our openbalena s3 instance via the included jenkins build scripts (balena-yocto-scripts), with some tweaks - i.e. pointing to our local s3 environment instead of the balena-cloud environment, etc. You would also need to use a tool such as open-balena-admin to create the openbalena user that is used by the build process (balena_os), and set up a process to run in your host OS to check for updates. Once an update is detected, the included hostapp-update scripts work really well.
If you are building your own OS Iâd be happy to provide more details on all of this, if not unfortunately Iâm not sure of a way to do what you are trying to do as host OS updating is not supported by openbalena out of the box.
Hello @drcnyc
Thanks for your reply!
I would love to know more about your approach, however, I would like to do it without any modifications to the Host OS (ex. Executing a chain of commands over SSH, which I expect is whatâs happening with the balena-cloud platform)
Ultimately I would love to provide a new docker container which could be run alongside the openbalena instance to handle these types of tasks over the tunnel/ssh connection. But maybe there are drawbacks with this approach Iâm not seeing at the moment?
Would love to hear your input!
@GoogleIt, I think the challenge youâre going to run into is that the hostapp-update script needs to be provided the OS image, which is given either as a local image (-f option) or a remote image (-i option). When your device is connected to balena-cloud, this isnât an issue because the OS images are already loaded into its docker registry (more on this below) - but when you are in openbalena, your registry has no OS images - and as you pointed out, balena has stopped using the docker public registry in favor of their (private) balena-cloud registry.
From what I can tell, the way balena-cloud handles hostapp updates today is by automatically deploying newly built OS images to the balena-cloud registry using the jenkins build scripts, which are then available on all devices by default as those devices can authenticate to the balena-cloud (openbalena) instance using their API key. Then, when an OS update is requested, balena-cloud triggers the hostapp-update script to run (presumably through supervisor) pointing it to the target image in the balena-cloud registry using the -i flag.
The way we have handled it is similar, by building new OS images to our private openbalena registry using the jenkins scripts, and then including a hostapp-update wrapper script which authenticates to our private openbalena registry using the device UUID and API key. Once authenticated, we determine the image location, it runs hostapp-update, which then works in the same was as on a balena-cloud device. Our wrapper script also does other stuff such as checking for pinned OS versions, looking for latest versions on a schedule, etc. but I think thatâs aside from the problem you are trying to solve.
One idea which isnât the cleanest but could work for you, would be if you ran âbalena loginâ on your host OS, but log into the balena-cloud registry instead of you own registry (registry.balena-cloud.com?). Youâll need a balena-cloud user and password, although Iâm not sure if a regular user will give you access to device images (devices would typically log in using d_[UUID] as username and their API key as a password, but your device wonât be registered in balena-cloud so that wonât work). Then, youâll need to know / get the image âlocationâ, which is that UUID-looking string which identifies docker images in a registry. It needs to be decoded from the target version number you are looking for, below is how I handle this decoding in my hostapp update script wrapper: (TARGET_IMAGE is the version number you would be used to seeing and DEVICE_TYPE is self explanatory)
RELEASE_ID=$(curl -s -H "Authorization: Bearer $BALENA_API_KEY" "$BALENA_API_ENDPOINT/v6/release_tag?%24filter=release/any(r:r/belongs_to-application/any(bta:bta/slug%20eq%20%27balena_os/${DEVICE_TYPE}%27))%20and%20tag_key%20eq%20%27version%27%20and%20value%20eq%20%27${TARGET_IMAGE}%27" | jq -r '.d[0].release.__id')
IMAGE_ID=$(curl -s -H "Authorization: Bearer $BALENA_API_KEY" "$BALENA_API_ENDPOINT/v6/image-is_part_of-release?%24filter=is_part_of-release%20eq%20${RELEASE_ID}" | jq -r '.d[0].image.__id')
IMAGE_RECORD=$(curl -s -H "Authorization: Bearer $BALENA_API_KEY" "$BALENA_API_ENDPOINT/v6/image?%24filter=id%20eq%20${IMAGE_ID}")
IMAGE_LOCATION=$(jq -r .d[0].is_stored_at__image_location <<< "$IMAGE_RECORD")
Note that you will also need an API key for balena-cloud (to be set as BALENA_API_KEY) for the above to work. If this were your own openbalena instance, you would just use the deviceâs API key for these calls, which would only be possible if you had built your own OS images to your openbalena instance.
Now, once you have the image location (and assuming you have already logged in to the balena-cloud registry) you should be able to simply run:
hostapp-update -i $IMAGE_LOCATION
No clue if this will actually work - but hopefully it helps! If you can get this to work manually in your host OS, the next step will be figuring out a way to do it all from a container.
Allright, that cleared some things up!
Would balena be open to publishing the images to a public registry? Else would it be OK if a community member builds them and ships them to a public registry? I could spare some compute for this personally so that would also be an option.
I would then modify the script so some API calls arenât needed anymore, which could work from looking at the script if done correctly. (I have updated an OS manually before, using the publicly available resinos images)
If all of that works out, tunneling over the VPN and executing arbitrary SSH commands should be as easy as 1-2-3.
So I guess the first step is making sure the balenaOS images are available publicly (for device types that donât have a special contract) and the same for the supervisor.
If that doesnât work out, Iâll probably give the openbalena registry approach a whirl, and maybe when more people start to experiment it we can start streamlining it even more.
@GoogleIt it is definitely technically possible to move a docker image from one registry to another (and there are plenty of guides out there on how to do it), but Iâm guessing the balena team made the decision to stop using public dockerhub images for a reason and are not interested in going back. The jenkins build scripts actually still have all of the code to deploy images to public dockerhub in addition to the openbalena registry, but those must not be used by balena anymore. That is a build option that can be turned off, I donât use it either.
Alternatively you could clone the balenaos GitHub repository for your desired hardware and build them yourself / push to dockerhub, but once youâre doing that you might as well just push them to your private openbalena registry as youâve already done the hard part (building your own OS) and then you would also solve the problem of how to get them on the device as it can authenticate to your private registry using its API key. But if you prefer using public dockerhub you could do that too, as youâll have the images at your disposal. You could even just host them via a web server and wget them on the device / use the -f switch with hostapp-update.
@drcnyc did you release this at all yet?
Hi @shawaj unfortunately we havenât had time to get this packaged up properly for release, right now itâs intertwined with all sorts of other proprietary systems we use - such as our own device registration framework, our own image tagging to manage versions / pinning, and other layers / recipes specific to our hardware and application. It would also need sufficient documentation given the number of different pieces involved - from provisioning the build user in open-balena-admin (automated building / updating requires adding users to openbalena) to modifying the jenkins build scripts to not use balena cloud servers and ultimately how to manage the entire process. I would still like to do this, and will post here if and when we do - but in the meanwhile happy to provide guidance and sample code for any specific pieces from the host OS update process that I described earlier in this chain.
Hy everyone,
I was able to modify the upgrade scripts for enabling the HostOS-Upgrade for openBalena Devices.
Please see the pull request:
I use all public API Endpoints to get the host-image and supervisor-images from balenas docker registry.
For device specific openBalena HTTP-Requests, the parameters from the config.json are used to update your openBalena Environment
With best regards.
1 Like