routing rules for multiple networks when NetworkManager doesn't handle wwan0

Product support balenaOS
1

Hi,

I’m trying to sort out options for managing multiple connections. We want to support two network interfaces:

  1. a wired connection which is managed by NetworkManager (eth0)
  2. a ethernet over USB (CDC_ECM) modem initialized in a container. We have the modem working using ip and dhclient to initialize the interface (wwan0).

Here is various logging, showing a device with the two interfaces configured:

## connecting wwan0
May 11 22:11:52 4c5ad7c kernel: cdc_ether 1-1.4.4:1.2 wwan0: register 'cdc_ether' at usb-3f980000.usb-1.4.4, Mobile Broadband Network Device, 42:cd:24:2f:f5:2d
May 11 22:11:52 4c5ad7c kernel: usbcore: registered new interface driver cdc_ether

root@4c5ad7c:~# ip addr show wwan0
8: wwan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 42:cd:24:2f:f5:2d brd ff:ff:ff:ff:ff:ff
root@4c5ad7c:~# ip link set wwan0 up
root@4c5ad7c:~# ip addr show wwan0
8: wwan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel qlen 1000
    link/ether 42:cd:24:2f:f5:2d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::40cd:24ff:fe2f:f52d/64 scope link tentative
       valid_lft forever preferred_lft forever

root@4c5ad7c:/usr/src/app# dhclient -v wwan0 -e IF_METRIC=500
Listening on LPF/wwan0/42:cd:24:2f:f5:2d
Sending on   LPF/wwan0/42:cd:24:2f:f5:2d
Sending on   Socket/fallback
DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 6
DHCPOFFER of 192.168.225.27 from 192.168.225.1
DHCPREQUEST for 192.168.225.27 on wwan0 to 255.255.255.255 port 67
DHCPACK of 192.168.225.27 from 192.168.225.1
bound to 192.168.225.27 -- renewal in 18840 seconds.

root@4c5ad7c:/usr/src/app# ip route
default via 192.168.78.1 dev eth0 proto dhcp metric 100
default via 192.168.225.1 dev wwan0 metric 500

My concern is how this setup will interact with the NetworkManager connectivity check. My understanding is, that if eth0 loses connectivity, it will add 20000 to the metric and switch traffic to wwan0. However, since NetworkManager isn’t managing wwan0, I’m concerned it will never switch back to eth0 if wwan0 loses network connectivity?

I tried to use NetworkManager (instead of ip and dhclient) to initialize dhcp after enabling the modem, but NM doesn’t recognize the wwan0 interface as a device, so there is no way to tell NM to connect.

root@4c5ad7c:~# nmcli dev
DEVICE           TYPE      STATE                   CONNECTION
eth0             ethernet  connected               Wired connection 1
supervisor0      bridge    connected (externally)  supervisor0
balena0          bridge    unmanaged               --
br-8d0bce04f7e9  bridge    unmanaged               --
resin-dns        bridge    unmanaged               --
lo               loopback  unmanaged               --
resin-vpn        tun       unmanaged               --

The options I’m considering:

  1. Figure out why networkmanager won’t see the wwan0 device - any explanations for this?
  2. Use the current setup, and add our own connectivity check on the modem and manipulate the wwan0 metric( or somehow reset the eth0 metric).
  3. Try to get ModemManager working and get rid of the custom setup, but the modem provider says its not supported.

Can you offer any advice on 1 or 2, and/or clarifications on the metric connectivity check rules? Am I correct that NM will only manipulate the metric on interfaces it connects?

Thanks!