Custom network and VPN routing

Dear Balena,

thank you for your time and energy to handle questions like mine. It’s very much appreciated. I love your product and we are building great things with it.

We are deploying drones globally running containerized ROS2 nodes. Each docker container running a single ROS2 node respectively.

Currently we have the desire to have drones and base stations communicate with each other in the most ROS-like fashion. This to avoid translation layers and custom interfaces. All this over 4G.

We stumbled upon a solution like this (SOLUTION 3 paragraph): Connecting Remote Robots Using ROS2, Docker & VPN | Husarnet

and especially excited about a deployment like this:

version: '2.3'

services:
    color_controller:
        build:
            context: ../../
            dockerfile: eg6/dev1/Dockerfile
        volumes:
            - ./cyclonedds.xml:/cyclonedds.xml
        command:
            - bash
            - -c
            - |
                export CYCLONEDDS_URI=file:///cyclonedds.xml
                ros2 run my_demo_pkg color_controller
        network_mode: service:husarnet
        
    move_controller:
        build:
            context: ../../
            dockerfile: eg6/dev1/Dockerfile
        volumes:
            - ./cyclonedds.xml:/cyclonedds.xml
        command:
            - bash
            - -c
            - |
                export CYCLONEDDS_URI=file:///cyclonedds.xml
                ros2 run my_demo_pkg move_controller
        network_mode: service:husarnet

    husarnet:
        image: husarnet/husarnet
        restart: unless-stopped
        volumes:
            - /var/lib/husarnet  # This will persist your Husarnet Client keys, thus IP of the container will be stable/the same between (re)boots
        sysctls:
            - net.ipv6.conf.all.disable_ipv6=0  # Husarnet is using IPv6 for the internal connections
        cap_add:
            - NET_ADMIN
        devices:
            - /dev/net/tun
        env_file: 
            - ./.env  # create .env file in the same folder as Dockerfile and specify HOSTNAME and JOINCODE there

However in your balena docker-compose definition, you specifically mention that the

        network_mode: service:husarnet

is incompatible. I was wondering if that is (still) the case or if there is some other workaround available?

They do propose a solution where we compile their VPN/networking code in every container, but that feels less modular and portable. So that’s not the first choice.

Greetings and thank you for your time.