open-balena-admin: an admin interface for openbalena

@the-real-kenna thank you for all the information - it is very helpful as we continue to develop open-balena-admin. Can you (or your engineers) provide some additional details on the following:

Pine assumes it has direct and exclusive control of the database so in theory it can change the layout of the database in completely arbitrary ways at any time.

  • Does this mean that it can actually change the table structure while running? Or are you referring to changes occurring from one release to the next? If the former, that’s an issue. If the latter, we have a version management system within open-balena-admin which based on the version of open-balena-api that is running (specified during the install process) it will modify the underlying table structure.
  • the release.semver (which is also a computed term) + release.revision pair
  • the image location (which is the basis for our registry to work & devices to download updates)
  • Are these items referring to deployment of apps / creation of new images and releases? If so, we only interface with images / releases on a read only basis.

Other than the above items, it sounds like it’s essentially pre-commit “soft” validation (as “hard” rules are enforced by the database, i.e. text field length limits, numbers vs text, foreign keys, etc.), computed fields and permissions that would need to be replicated if bypassing pine. We have some basic soft validation implemented in the front end now, but not much in terms of computed fields. Is there a full list of computed fields available anywhere?

Separately, I am happy to explore ways to integrate Pine (or any other balena best practices for that matter) into open-balena-admin - I don’t suppose this is something that balena would be willing to help with? I believe there is a community of users who are either using openbalena in production (and muddling through the limitations) or looking to do so but not able to get over the limitations - and I believe a tool such as this could bridge the gap. Either way, we will continue to develop it, and provide the latest updates to the community in the hopes that at least some members are finding it helpful despite its flaws.

@drcnyc I don’t know the answers, but will ping the folks who replied to me and ask if they can elaborate more here in-thread while I’m out of office the next few days. :slight_smile:

Hopefully we can help get you pointed in the right direction for these specific questions, but also come up with some good ways to share information better between our strategies / intents with openBalena and the community that uses and builds on top of it.

I also meant to say, the reason I’m sharing this information in part is also because we are really excited to see what you’ve created on openBalena and want to be sure it continues to be successful. I hope that comes across in these async messages, but wanted to be explicit about it just in case. I’ve personally recommended your UI to several customers already, so please know we’re excited about what you’ve built and are hoping to support you in that work remaining successful over time. :slight_smile:

Hi @drcnyc

Does this mean that it can actually change the table structure while running?

No. At the moment we only change the database schema in migrations that run when a new version of open-balena-api first starts up.

Are these items referring to deployment of apps / creation of new images and releases?

~Yes. The release.revision & image.is_stored_at__image_location fields are populated by the open-balena-api when creating a new release or image. I you are just surfacing the DB data for those columns in a read-only way then there shouldn’t be an issue. That being said that, there are the release.version & release.raw_version which are computed term that the balenaCloud dashboard surfaces in the releases page, which you would need to re-implement.

permissions that would need to be replicated if bypassing pine

Keep in mind that Pine also has permission caching & invalidation, which might be an issue if you start modifying the DB directly.

Is there a full list of computed fields available anywhere?

The lazily computed (on request) terms are:

  • release.semver, release.version, release.is_final & release.raw_version
  • device.overall_status & device.overall_progress
  • the user__has_direct_access_to__application computed resource/table

On top of these, there are also “eager”/on-write computed terms, such as:

  • the release.revision
  • the invalidation of the user’s jwt when the username/password gets updated
  • generating a slug for each application created or renamed
  • creating a unique image location for each new image

pre-commit “soft” validation (as “hard” rules are enforced by the database

There is also a group of rules which are not enforced by the database, but I’m hesitant to categorize them as soft, since violating them would result the DB to be in an invalid state. That group includes rules like like making sure devices only run releases that are of status “success”, which pine confirms with the database before it persists any uncommitted data that might invalidate those rules.
Here is a list of such rules which atm are enforced by Pine and not the DB:

A good way to get introduced to Pine or at least its exposed OData endpoints, would be through the balena-sdk, which is solely what our dashboard uses to interact with our backend.

Kind regards,
Thodoris

@thgreasi thank you!!! This is extremely helpful information and will drive the next leg of development of open-balena-admin. I’m still taking a lot of this in but below are my thoughts / action items.

No. At the moment we only change the database schema in migrations that run when a new version of open-balena-api first starts up.

Great - so we should be good on the table / column structure, it just means that we need to keep an eye on any changes to the sbvr with each new release of open-balena-api, which is exactly how we built our backwards compatibility with multiple versions. Whenever the sbvr changes, we simply need to work that change into our versioning system so we can support the new feature. But it wouldn’t break anything on running instances, this would only apply to someone who intentionally updates their open-balena-api instance, so it would just necessitate a concurrent update of open-balena-admin.

~Yes. The release.revision & image.is_stored_at__image_location fields are populated by the open-balena-api when creating a new release or image. I you are just surfacing the DB data for those columns in a read-only way then there shouldn’t be an issue. That being said that, there are the release.version & release.raw_version which are computed term that the balenaCloud dashboard surfaces in the releases page, which you would need to re-implement.

Also great - we intentionally are not involved with images and releases. Similar to balena cloud, open-balena-admin users need to use balena-cli for release deployment.

The lazily computed (on request) terms are:

  • release.semver, release.version, release.is_final & release.raw_version
  • device.overall_status & device.overall_progress
  • the user__has_direct_access_to__application computed resource/table

ACTION ITEM: These computed fields should be implemented to show on their respective pages.

On top of these, there are also “eager”/on-write computed terms, such as:

  • the release.revision
  • the invalidation of the user’s jwt when the username/password gets updated
  • generating a slug for each application created or renamed
  • creating a unique image location for each new image

Release revision is not necessary because we don’t support pushing releases via the web. We have our own invalidation process for the jwt which actually uses open-balena-api so should follow this already. For the slug, we allow users to create their own slug and make it required. And again we don’t need to worry about the image location because we don’t support pushing images via the web.

There is also a group of rules which are not enforced by the database, but I’m hesitant to categorize them as soft, since violating them would result the DB to be in an invalid state. That group includes rules like like making sure devices only run releases that are of status “success”, which pine confirms with the database before it persists any uncommitted data that might invalidate those rules.
Here is a list of such rules which atm are enforced by Pine and not the DB:

I did see those when combing through the sbvr but did not appreciate that they were not implemented at the database level - but when reading through them it makes total sense. Thank you for clarifying this!! It basically gives me a to-do list to work through implementing the same logic in our front end. Some of them seem pretty obscure and I would think unlikely for most users to encounter (i.e. if device1 is managed by device2 then application1 needs to depend on application2, public application types can not be legacy, etc. But others are more relevant like you must have a successful release tied to a device.

ACTION ITEM: These rules need to be implemented in open-balena-admin, and monitored for changes in the same way as the database structure.

This is the single most helpful contribution I have received for this project - and I can’t thank you enough for it.