USB hotplug for USB serial devices

Product support balenaOS
,
1

What is the current recommended way to handle USB serial devices being plugged in?

I am working on porting an open source IoT gateway to balenaOS for the commercial pilot of an IoT hub product. The gateway has an add-ons system which adds support for a wide range of hardware including Zigbee and Z-Wave USB dongles (e.g. the Aeotec Z-Stick Z-Wave dongle or the Digi XStick or Conbee II Zigbee dongle).

On Raspbian the add-ons can automatically detect USB serial devices that are plugged into the device at runtime and use them.

Running inside a Docker container on balenaOS I can statically map a device in docker-compose.yml if I know what path it will be mounted at, e.g.

    devices:
      - /dev/ttyACM0:/dev/ttyACM0

However, if that device is not plugged in when the Docker image starts up it will simply refuse to start and the (headless) gateway becomes inoperable.

Ideally I would like to be able to dynamically detect USB dongles when they are plugged in using USB hotplug and automatically map them into the Docker container.

If I could create a bind mount to just bind the whole of the host’s /dev directory then I could configure device_cgroup_rules to allow the kernel to access certain classes of device, and perhaps use cap_add to add SYS_RAWIO for low level access (I do not want the container to be running privileged mode since that negates most of the security benefits of using a container), e.g.

    volumes:
      - '/dev:/dev' # Mount the host's dev directory
    device_cgroup_rules:
      - 'c 188:* rmw' # Allow USB serial devices (XBee, Conbee III, Sonoff)
      - 'c 166:* rmw' # Allow ACM devices (Conbee II)
    cap_add:
        - SYS_RAWIO

However, since bind mounts are not allowed this is not possible.

I can imagine some elaborate sidecar container that runs in privileged mode and pipes serial interfaces into unix sockets, but that would require re-architecting my whole gateway application and its 100 or so adapter add-ons in order for it to run on balenaOS.

Is there a recommended way to detect USB devices being plugged in via USB hotplug and automatically map them into a container?

2

Hello @benfrancis,

Well we do have a way to let the container use the whole /dev, however I believe it does need to be privileged:

Ideally I would like to be able to dynamically detect USB dongles when they are plugged in using USB hotplug and automatically map them into the Docker container.

I think that could be doable with a set of carefully crafted udev rules with proper access rights and clever volume management.

If you start this work and report here, we can try to guide you to the best possible outcome.

Regards,

Yann

3

Hi,

I’m still struggling with this.

I thought I might have found a solution by using device_cgroup_rules in docker-compose.yml to allow allow the container cgroup to access USB serial device classes, so that device nodes created at runtime via mknod are functional:

    labels:
      io.balena.features.sysfs: '1'
    cap_add:
      ## Raw I/O access for interacting with hardware devices
      - SYS_RAWIO
    # Allow container cgroup to access USB serial device classes so that
    # device nodes created at runtime via mknod are functional.
    # Major 166 = ttyACM (USB CDC ACM devices, e.g. Zigbee dongles)
    # Major 188 = ttyUSB (USB-to-serial converters, e.g. FTDI/CH340)
    device_cgroup_rules:
      - 'c 166:* rwm'
      - 'c 188:* rwm'

I then created a usb-monitor.sh script which dynamically creates and removes /dev nodes for USB serial devices (ttyACM*, ttyUSB*) inside the container by polling /sys/class/tty/.

Whilst this made the serial ports dynamically discoverable from inside the container, when the gateway application actually tried to use them it caused an EPERM error which suggests the kernel’s device cgroup is rejecting the open() call:

[Logs]    [2026-03-26T17:21:05.903Z] [webthings-gateway] 2026-03-26 17:21:05.903 INFO   : zigbee-adapter: Probing serial ports
[Logs]    [2026-03-26T17:21:05.918Z] [webthings-gateway] 2026-03-26 17:21:05.918 ERROR  : zigbee-adapter: [Error: Error: Operation not permitted, cannot open /dev/ttyACM0]
[Logs]    [2026-03-26T17:21:05.919Z] [webthings-gateway] 2026-03-26 17:21:05.918 ERROR  : zigbee-adapter: [Error: Error: Operation not permitted, cannot open /dev/ttyACM0]
[Logs]    [2026-03-26T17:21:05.919Z] [webthings-gateway] 2026-03-26 17:21:05.919 INFO   : zigbee-adapter: Serial ports that were found:
[Logs]    [2026-03-26T17:21:05.929Z] [webthings-gateway] 2026-03-26 17:21:05.929 INFO   : zigbee-adapter: USB Serial Device 1cf1:0030 Vendor: dresden elektronik ingenieurtechnik GmbH Serial: DE2706277 found @ /dev/ttyACM0
[Logs]    [2026-03-26T17:21:05.929Z] [webthings-gateway] 2026-03-26 17:21:05.929 ERROR  : zigbee-adapter: Failed to start add-on zigbee-adapter: No Zigbee dongle found

Upon further investigation I discovered that despite ostensibly supporting docker-compose version 2.4 files, the balena supervisor does not appear to support device_cgroup_rules and was dropping it and logging a warning:

Mar 26 17:20:38 webthings balena-supervisor[2247]: [warn]    Ignoring unsupported or unknown compose fields: containerName, logging, deviceCgroupRules

Can you confirm that no version of the supervisor currently supports the device_cgroup_rules directive in docker-compose.yml?

If so, can you provide any further guidance on the custom udev rules approach you suggested? I can’t figure out a way to make that work which doesn’t require hard coding vendor/product IDs and/or a static device mapping which causes the container to fail to start if the device is not plugged in on boot.

Thank you

4

Reporting back on this, I have come up with a rather elaborate workaround.

Essentially this PR:

  • Adds a usb-monitor.sh script inside the gateway container which dynamically creates and removes /dev nodes for USB serial devices (ttyACM*, ttyUSB*) by polling /sys/class/tty/
  • Adds a tiny privileged sidecar container that whitelists USB serial device classes (ttyACM major 166, ttyUSB major 188) in the gateway container’s cgroup

This makes it possible to dynamically map serial USB devices into the gateway application’s container as they are plugged and unplugged at runtime, without needing to make the whole gateway application’s container privileged (or needing to give it the SYS_ADMIN capability), which would have largely defeated the object of running the application inside a container in the first place.

This negates the need to hard code device paths in docker-compose.yml and means that:

  1. For limited classes of devices it behaves as if the gateway container supports USB hotplug
  2. Add-on adapters like our Zigbee and Z-Wave adapters can detect USB devices at runtime
  3. The gateway can still start up if one those devices are not connected at boot

This is obviously a bit of an ugly hack, so if there is a cleaner way to achieve this please let me know. And if not, I feel like this is something balenaOS could offer a much cleaner solution to, since USB hotplug seems like a feature that people very commonly need.

3 Likes
6

Hello @benfrancis

That’s quite a neat workaround!