Host and docker-compose network

Product support balenaOS
1

I’m running a container with an app that listens to the board ip (192.168.0.48). It works fine when I use ‘balena push’ in local mode without docker-compose.

However, my application is bigger now and I need to run multiple containers that are in the same network. For that I set up a docker-compose like:

version: '2'

services:
    master:
        image: ros:melodic-ros-core
        container_name: master
        environment:
            - ROS_HOSTNAME = master
            - ROS_MASTER_URI=http://master:11311
        command: roscore
        networks:
            - ros_net

    comm_node:
        build:
            context: .
            dockerfile: DockerfileComm
        container_name: comm_node
        environment:
            - ROS_HOSTNAME = comm_node
            - ROS_MASTER_URI=http://master:11311
        network_mode: host
        networks:
            - ros_net
        labels:
            io.balena.features.balena-socket: '1'
            io.balena.features.dbus: '1'
        ports:
            - "1024:1024"
        depends_on:
            - master

networks:
    ros_net:

But now I can’t listen to the board ip(192.168.0.48) anymore. How can I solve this? Can I access the host network and a custom network at the same time?

2

Hi,

What is running on the board IP that isn’t inside your compose stack?

4

Hello,

I have two raspbery exchanging messages. Rasp A sends to rasp B ip and listen to its own ip, and vice-versa.

Something like this:

import time
from socket import *

def send(msg, ip, port):
    s = socket(AF_INET, SOCK_DGRAM)
    s.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
    for m in msg:
        data = "{0}: Device: {1} Message:{2}".format(time.ctime(), gethostname(), m)
        s.sendto(data, (ip, port))
        print("Sending " + data)
        time.sleep(1)
    s.close()


def recieve(ip, port):
    s = socket(AF_INET, SOCK_DGRAM)
    s.bind((ip, port))
    m = s.recvfrom(1024)
    s.close()
    return m

Listener:

import simple_comms as comms
.
.
.
m=comms.recieve(IP_LISTEN, PORT)

Sender:

import simple_comms as comms
.
.
.
comms.send(DATA, IP_SEND, PORT)
5

hey @rezenders , I am not 100% sure, but it seems like it might be related to this issue https://github.com/balena-io/balena-supervisor/issues/994 , where services can’t resolve other service names because they are running host networking as in the case of your comm_node service.

7

Following the recommendations that I found in the link you provided a workaround would be to add the $HOSTNAME to /etc/hosts, like:

echo "Setting hostname..."
echo "127.0.0.1 $HOSTNAME" >> /etc/hosts

Even when doing this I can’t listen to the host ip(192.168.0.48). I keep getting this error:

[Logs]    [7/16/2019, 7:01:38 PM] [comm_node] Traceback (most recent call last):
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]   File "comm.py", line 53, in <module>
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]     main()
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]   File "comm.py", line 40, in main
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]     m=comms.recieve(settings.IP_LISTEN, settings.PORT)
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]   File "/testes/simple_comms.py", line 17, in recieve
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]     s.bind((ip, port))
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]   File "/usr/lib/python2.7/socket.py", line 228, in meth
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node]     return getattr(self._sock,name)(*args)
[Logs]    [7/16/2019, 7:01:38 PM] [comm_node] socket.error: [Errno 99] Cannot assign requested address

I added this service to check if I could ping the host, and it works just fine.

ping:
        image: busybox
        container_name: busybox
        environment:
            - ROS_HOSTNAME = busybox
            - ROS_MASTER_URI=http://master:11311
        # network_mode: "host"
        networks:
            - ros_net
        labels:
            io.balena.features.balena-socket: '1'
            io.balena.features.dbus: '1'
        ports:
            - "1024:1024"
        depends_on:
            - master
        command: ["ping", "192.168.0.48"]

Inspecting /etc/hosts inside the container I get the following:

When using single container:

127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.114.101.2 6e1fdb43c25d

When using docker-compose:

127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.19.0.2 905706cfdc61
9

I think that’s a fundamental question that needs answering, and I think the answer is no. I was running some tests with the docker-compose file you shared, having the master and comm_node services use the ubuntu:bionic base image. Running with standard docker / docker-compose on a Mac laptop, it prints an error message:

$ docker-compose up
ERROR: 'network_mode' and 'networks' cannot be combined

Running it with “balena push ipAddress”, giving it the IP address of an Intel NUC device with balenaOS 2.32.0, resulted for me in the comm_node container crashing on a restart loop. I also found these posts that seem to confirm that host networking and custom networks don’t go together:

Removing the network_mode: host line from the docker-compose file avoids those errors I came across, but it also means that the ‘eth0’ virtual network interface uses a private IPv4 network for the ros-net “custom network” like, for example, 172.18.0.3/16:

root@ab56ba8961f3:/# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
      inet 172.18.0.3  netmask 255.255.0.0  broadcast 172.18.255.255
      ...

In such an environment, it would be expected that binding a server socket to the host OS IP address (the 192.168.0.48 board IP in your example) would fail just like you described, because the IP address belongs to a different network.

The fact that ping succeeds for the host OS IP address (192.168.0.48) does not mean much: we can also ping www.google.com, but this does not mean we can bind a server socket to one of Google’s web server IP addresses. :slight_smile:

Also, unless I am missing something, I think that the /etc/hosts issue is unrelated to this problem.

I am thinking: is it really necessary for the server to bind to the board IP (192.168.0.48)? Is it not enough for the comm_node container to use the “expose” and/or “ports” instructions like in the following sample project? -

https://github.com/balena-io-projects/multicontainer-getting-started/blob/master/docker-compose.yml

I gather that the sample project linked above runs servers in the app containers, and allow those servers to be contacted from the outside world via the expose / ports instructions, without explicitly binding to the host OS IP address.

11

Hello,

Thank you for your answer. You are absolutely right, I just had to bind the right ports between the host and container, and then listen to the container’s ip which in my case is 172.22.0.3.

I will leave the docker-compose file that worked for me, in case anyone face the same questions:

version: '2'

services:
    master:
        image: ros:melodic-ros-core
        container_name: master
        environment:
            - ROS_HOSTNAME = master
            - ROS_MASTER_URI=http://master:11311
        command: roscore
        networks:
            - ros_net

    comm_node:
        build:
            context: .
            dockerfile: DockerfileComm
        container_name: comm_node
        environment:
            - ROS_HOSTNAME = comm_node
            - ROS_MASTER_URI=http://master:11311
        networks:
            - ros_net
        labels:
            io.balena.features.balena-socket: '1'
            io.balena.features.dbus: '1'
        ports:
            - "1024:1024/udp"
        depends_on:
            - master

networks:
    ros_net:
13

Hello again,

I have another doubt now.
By following your answer I was able to send and receive packages. In the receiver I bound the socket with the container IP (which is bound to the host port) and in the sender I send the packages to the other board IP.

In order to find what is the container IP I’m using its name in the network.

Now, I want the sender to broadcast instead of sending to a specific IP. In order to accomplish that I’m sending the packages to the network broadcast IP 192.168.0.255 and I’m still listening to the container’s IP.

With this, the board that broadcast the packages receives the package but the other boards in the network doesn’t.

I found this post in stack overflow: https://stackoverflow.com/questions/22878625/receiving-broadcast-packets-in-python`

One answer says to do something like this:

socket.bind(('',1024))

I tried doing this but it didn’t work.

How do I fix this? Any thoughts?

Thank you

19

Hi @rezenders

From the last docker-compose file you posted, it looks like both comm_node and master containers belong to a single network created by the engine on your machine.
This network is different from the network you host belongs to. Hence, the broadcast UDP packet you send from a container should not actually leave the machine that container runs on.

To my understanding, if you want to send broadcasts to other machines, you have to do to it from a process that has access to the host network…