Mount specific USB into each container

Hi Team,

I run multiple SDR dongles, using Multicontainers, I detect that all the devices have the same SerialNumber (I’m talking with the provider to understand why) but in case this can’t be fixed, I’m thinking in mount each USB dongle into 1 container

Right now all the containers see all the USB ports connected

root@raspberrypi3-e78c6d9:/data# rtl_test
Found 4 device(s):
0: Realtek, RTL2838UHIDIR, SN: 00000001
1: Realtek, RTL2838UHIDIR, SN: 00000001
2: Realtek, RTL2838UHIDIR, SN: 00000001
3: NooElec, NESDR Nano 3, SN: 7236991335

Some way to mapping only the device 0 to one container, the device 1 to other, and this way be sure that each container ONLY will see 1 USB devices?

Thanks
Alejandro

Heya ! A quick question - are these devices plugged in permanently and present during the boot time or are they being added during the runtime ?
thanks !

I’m asking because, in short, if the devices appear after the containers are started then I don’t know of any way to do this currently.

If the devices are fairly static though, you could try using custom udev rules on the host, to make sure that the devices always appear under the same paths and then use devices section in the docker compose file. I’m afraid I don’t have a good full example of that setup to share though.

In general, these types of usecases are something we’re looking into implementing in the future in balenaEngine but we’re not there just yet, in terms of full official support and docs.

hope this helps and let us know of the progress :slight_smile:

Hi, this can be static, we setup the Dongles 1 time, and the idea is NOT remove this for the life of the operation.
I made the first try with DEVICES definition into docker-compose, without success. I’ll dig more deeper. But I think is really important this feature, as many times the only interface with external hardware is using USB.

@mediainbox, when you say you did not have success with the ‘devices’ section in docker-compose, is it because the container still sees all the USB ports?
I was looking at the docker-compose reference for the ‘devices’ section and it seems that it works in the same way as the docker create --device option, which is documented as “Add a host device to the container”. So I assume the ‘devices’ section will only add devices, not remove them, but it sounds like you need to remove devices.
Do you have “privileged: true” in docker-compose? If so, try with “privileged: false”, maybe also remove some “cap_add” permissions if you have them, so that your containers won’t see any devices at all… except for the devices listed under ‘devices’. Check these docs pages:

@pdcastro I test with Privileged: false and the container still see ALL the USB connected to the HOST

root@41fa6be36aa0:/data# lsusb
Bus 001 Device 006: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 004: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 007: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Bus 001 Device 009: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T 

I don’t have access from the Application but the devices are mapped, I need avoid that point, the idea is ONLY show 1 device to the Container

root@41fa6be36aa0:/data# rtl_test 
Found 4 device(s):
  0:  , , SN: 
  1:  , , SN: 
  2:  , , SN: 
  3:  , , SN: 

Using device 0: Generic RTL2832U OEM
usb_open error -4
Failed to open rtlsdr device #0.

Some other advice to help me?

@mediainbox, thanks for sharing your results. So privileged: false meant that the application could not access the USB dongles, but could still list them. Taking a step back, how much control to you have over the application? If the application could be influenced through command-line options, environment variables or even source code changes, I think there would be a number of solutions. Some thoughts:

  • Have you tried using a combination of privileged: false and the devices section of docker-compose file? With a single of the SDR dongles listed under the devices section, I would hope that the output of rtl_test would become something like:
$ rtl_test 
Found 4 device(s):
  0:  , , SN: 
  1: Realtek, RTL2838UHIDIR, SN: 00000001
  2:  , , SN: 
  3:  , , SN: 

I have made up the output above (not real), just to illustrate my thinking. In this case, if you could modify your app, it might be possible to try to access each listed dongle in turn, and skip the ones that the app does not have access to. Even though each container would be able to list all dongles, the app in each container would end up accessing only the dongle it has access to.

  • Another thought is to use service environment variables to tell your app, in each container = service, which dongle it should access – avoiding the need of trying each one in turn.

  • Another alternative is to use custom udev rules injected by the containers at runtime and based on service environment variables, so that the correct USB dongle for each container would be listed under a path like /dev/sdr_dongle (i.e. the same path for all containers, but in each container the path would point to a different SDR dongle). But this would require your app to be looking for the dongle at that path, rather trying to list all available USB devices. If this was suitable to your app, we could elaborate on it further. (But presumably if you could change the app to look at /dev/sdr_dongle, it might be easier to change the app so that the apps looks up the service environment variable directly.)

Currently my application support the param -d that allow to pass a Device Index, but for some reason, only works if I run multiple commands into the SAME Container.

When run into multiple containers the -d params don’t works, always all the containers trying to access to the same USB device

currently the Envs show -d 0 into all the cases, but I try with 1, 2, 3 to map to other devices, and try using Serial Numbers too… this approaches all works into single container app, but if I run the same command into individual container all devices trying to access to the same device.

@pdcastro take a look this, very extrange:

using this Docker-compose.yml

version: '2'
services:
  encoder-fm-1:
    privileged: false
    build: ./encoder-node
    restart: never
    devices:
      - "/dev/bus/usb/001/004:/dev/bus/usb/001/004"
    cap_add:
      - ALL
  encoder-fm-2:
    privileged: true
    build: ./encoder-node
    restart: never

When list the devices show this:

root@raspberrypi3-e78c6d9:/data# rtl_test 
Found 4 device(s):
  0:  , , SN: 
  1:  Realtek, RTL2838UHIDIR, SN: 67972493
  2:  Realtek, RTL2838UHIDIR, SN: 67972493
  3:  Realtek, RTL2838UHIDIR, SN: 67972493

Using device 0: Generic RTL2832U OEM
usb_open error -1
Failed to open rtlsdr device #0.

Into the second container that use privileged: true list this:

root@raspberrypi3-e78c6d9:/data# rtl_test 
Found 4 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000001
  1:  Realtek, RTL2838UHIDIR, SN: 67972493
  2:  Realtek, RTL2838UHIDIR, SN: 19151912
  3:  Realtek, RTL2838UHIDIR, SN: 00000001

Some ideas?

@mediainbox, that’s indeed some strange output printed by rtl_test. I have found the source code for rtl_test and I’m thinking that there may be a bug in that code, in that it does not anticipate the possibility of getting empty strings or null pointers when not having permission to access some of the indexed devices. To be fair to the developer, it is unusual to have permission to query only some of the devices!

Given that the source code is available, I suggest changing the code according to your needs. You can print device string values and find out the difference when the container is granted or denied access permissions, and skip devices that don’t have permissions.

Hey @mediainbox,
I just wanted to check whether you had the chance to give this a try and how it worked for you.

Kind regards,
Thodoris

Hi @thgreasi, yes I can figure out how select the correct device with the real tool I’m using.

From Balena Side I’m running as Privileged:True and mapping the devices individually one by container, but all the containers see all the devices.