Balena application iptables no longer propagating to host BalenaOS

Product support balenaOS
1

Hi all,

Hoping I can get both some suggestion and insight into a mechanism that was previously working (and has been suggested elsewhere on this board) and now seems to not be functioning.

We have a single application running in a privileged, host network mode which needs to act as a router for traffic that it encounters. Previously, we solved this be setting (for now) very wide open iptables rules during the initialization of the container (i.e. just before it launches the Python-based system that runs the application). These requests are in a bash script as follows:

iptables -P FORWARD ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

I then list out the iptables just to verify that the settings have stuck, and as seems to be expected, these settings propagate from the container into the host OS. All of a sudden however, this is no longer true, and the same settings fail to alter the iptables in BalenaOS.

To verify, our docker-compose file includes the following network settings that seem to encompass all of the possible rights we’d need to be able to set this (to my knowledge):

network_mode: host

cap_add:
    - NET_ADMIN
    - NET_RAW
privileged: true

Further - I added a subprocess call to the Python code to again request iptables -P FORWARD ACCEPT. This command also runs successfully (I can also do it interactively in ssh into the container without any errors), but the changes still don’t get back to the host OS, as when I verify ssh’d into Balena OS, I invariably see:

Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

and all of my traffic is dropping. Can anyone shed some light on what else we can try / what has changed?

Thanks in advance,

~Michael

3

Hello Michael,

Can you please provide the version of balenaOS (and supervisor version if possible - available on the dashboard) where you are seeing this behavior with the iptables? When it was previously working, was that on a different version of the OS?

5

Hi Alan,

Sure - current versions are:

balenaOS: balenaOS 2.58.3+rev1 (development)
Supervisor: [
11.14.0

it looks as if this version is already slightly outdated, so I’m assuming the OS didn’t have an automatic push to update or anything to have broken this - I can try updating the OS to see if this improves, but it would be good if I knew where this change in behavior originated so we can be careful in future releases.

Thanks!

7

The problem is with the base image actually. Once you add RUN update-alternatives --set iptables /usr/sbin/iptables-legacy in your Dockerfile.template file, the correct iptables version should be picked up.

For more information: https://wiki.debian.org/iptables

9

Hi again -

Thanks, this fixed the issue - so this wasn’t due to a change in balenaOS but the base for the Docker image.

Thanks again, a major relief!

~Michael