Using configfs for dynamic device tree overlays on BeaglePlay

Product support balenaOS
1

I made a blog post on how to apply dynamic device tree overlays on BalenaOS for BeaglePlay…

content copied here…

I spend most of my time working on BeagleBoard.org community projects, but I do a few projects of my own building IoT devices. There are a number of good software infrastructure tools out there, but I often use Balena. The problem I have faced often is not specific at all to Balena, but I’d like to share with you my Balena solution in case it might be helpful to you.

There is no fully “standardized” way to provide device-tree overlay fragments to Arm board bootloaders in general or for Beagles in specific. The config.txt file was picked up for Raspberry Pi boards by Balena, but they have yet to adopt the extlinux.conf we rely on for the BeagleBoard.org Debian images. While it is possible to customize BalenaOS myself, I find that to be an incredible pain.

In this particular case, all I wanted to do was to disable the loading of the serdev driver attached to the CC1352 so that I could flash it with different firmware and use it with other protocols. While the upstream driver provides a firmware update mechanism, I am using my own custom CC1352 firmware and need to remove the driver to get raw UART access. I suspect this is a fairly common use-case.

Balena has an example for building an out-of-tree kernel module that makes it fairly easy to build a kernel module. This was my starting point.

For the kernel module itself, Pantelis Antoniou proposed a patchset all the way back in 2014 that still works with the mainline dt changeset. It is a shame we haven’t yet included this with the mainline kernel, perhaps in fear of abuse, but given the nature of Arm and other architectures that depend on device tree for expressing hardware that might be dynamic, it feels like critical infrastructure for system developers. I grabbed a version off of the Xilinx Github Linux tree and added the necessary MODULE_LICENSE to enable it to compile out-of-tree as a module.

I built both the kernel module and device tree binaries in separate build phases. The kernel module example was tied to a specific version of Ubuntu and I didn’t want to mess with that. There was nothing special about the version of Ubuntu I grabbed to run the device tree compiler, except that `jammy` is a bit old for my taste. For the example of applying the overlay, I chose a Debian image with the Beagle package feed enabled. All this breakdown is likely entirely unnecessary.

Applying the device tree overlay via configfs is done in start.sh for this example:

#!/bin/bash

# Load the configfs kernel module for dynamic device tree overlays
insmod /opt/lib/modules/of_configfs.ko

# Apply the device tree overlay
mount -t configfs none /sys/kernel/config
mkdir -p /sys/kernel/config/device-tree/overlays/no-cc1352-driver
cat /opt/lib/dtb/k3-am625-beagleplay-bcfserial-no-firmware.dtbo > /sys/kernel/config/device-tree/overlays/no-cc1352-driver/dtbo
cat /sys/kernel/config/device-tree/overlays/no-cc1352-driver/status # applied

# Verify the overlay applied and reload the platform serial driver
cat /proc/device-tree/bus@f0000/serial@2860000/mcu/status # disabled
echo 2860000.serial > /sys/devices/platform/bus@f0000/2860000.serial/driver/unbind
echo 2860000.serial > /sys/devices/platform/bus@f0000/2800000.serial/driver/bind

sleep infinity

Here I am not directly testing that the device tree overlay successfully applied or that the driver was disabled as a result of the device tree overlay being applied, but just showing you the cat operations that would indicate success. To complete the operations, the UART driver bind operation was performed. Note that /dev/ttyS1 may need to be generated manually with mknod and I skipped that operation here.

I have posted this example on Github. This exact version is untested as this was previously integrated into a project of mine, so please post any issues you find.

2 Likes
2

Hi @jkridner thanks for the write up - from the balenaOS perspective one of the challenges we face is how to standardize product features for over 150 different boards supported, with several different bootloaders to support.

Also, one of the pain points we have identified when adding balenaOS support for new hardware is bootloader development, as patching the source to make it balena compatible is always a pain.

What we decided was to try and avoid the pain altogether and use vendor bootloaders unmodified, and have them boot a secondary balena bootloader where we can then offer a homogenous product surface for things like device tree overlay support. As you can imagine, this effort is not completed or you would not be having these problems, but it’s a friction point we have been aware of for some time and are actively trying to improve.

3

I’m not sure if SystemReady will help, but something like that initiative might be what you need to standardize on things a bit.

The configurizer is nice for writing to some specific files used in systems. It might be nice if it could write to the bootloader directory to add the .dtb file used at boot. Of course, this could prevent boot success, so having some kind of A/B fallback is desired here. In my case, I’m comfortable verifying the .dtb before deployment with reasonably small deployments–but with large deployments, recovery might not be manageable. Still, this really is mostly something in the provisioning phase if there was a more direct way to edit the boot directory contents on the flasher image without needing to rebuild the whole thing from Yocto.

I’ve gotten into a bit of an odd cycle where once I have the first image, I just use Dockerfile to edit the contents and redeploy, but I don’t have the Docker base images to build from for BalenaOS.