Enable Public Device URL but not IP Address Access


#1

Currently utilizing noVNC and x11vnc on a raspberry pi to provide remote support for device through the browser. Is it possible to specify somewhere in the docker-compose.yml or in the network settings of the device, to ONLY allow connections/traffic through the Public Device URL and not the IP address of the device?

In other words, I am looking for a way to only gain access to the device when using the public URL - I dont want anything exposed when the IP address of the device is entered into a browser on another device.


#4

I was able to specify for noVNC to bind to the resin-vpn ip address. However, it appears as though this address is specified by DHCP. Is there any way to fix this IP address to be static?


#17

@wwalker
getting a constant IP on the vpn is unlikely to work so the way to go, will be using iptables firewall rules inside the container to prevent access from interfaces other than the VPN.
The Balena superisor does the same sort of thing by setting rules as follows:
INPUT -p tcp --dport {port} -i {iface} -j ACCEPT
INPUT -p tcp --dport ${port} -j REJECT

Don’t hesitate to reach out to us if you need further support.


#20

Hey @samothx
I am trying to restrict access to my port 80. I tried the following by ssh’ing into the host (with balena ssh -s option) :

iptables -A INPUT -i resin-vpn -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

and

ip6tables -A INPUT -i resin-vpn -p tcp -m tcp --dport 80 -j ACCEPT
ip6tables -A INPUT -p tcp -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

I still can’t seem to get it to work. :confused: Can you help?

output from netstat -ant

tcp        0      0 127.0.0.2:53            0.0.0.0:*               LISTEN      
tcp        0      0 10.114.102.1:53         0.0.0.0:*               LISTEN      
tcp        0    222 192.168.1.161:52802     34.237.229.125:443      ESTABLISHED 
tcp        0      0 :::48484                :::*                    LISTEN      
tcp        0      0 :::22222                :::*                    LISTEN      
tcp        0      0 :::80                   :::*                    LISTEN      
tcp        0    752 ::ffff:10.240.44.68:22222 ::ffff:52.4.252.97:38174 ESTABLISHED 

snipp from iptables -vL

Chain INPUT (policy ACCEPT 1338 packets, 152K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  supervisor0 any     anywhere             anywhere             tcp dpt:48484
   18  1224 ACCEPT     tcp  --  lo     any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  docker0 any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  tun0   any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  resin-vpn any     anywhere             anywhere             tcp dpt:48484
    0     0 REJECT     tcp  --  any    any     anywhere             anywhere             tcp dpt:48484 reject-with icmp-port-unreachable
    0     0 ACCEPT     tcp  --  resin-vpn any     anywhere             anywhere             tcp dpt:http
    0     0 REJECT     tcp  --  any    any     anywhere             anywhere             tcp dpt:http reject-with icmp-port-unreachable

snipp from ip6tables -vL

Chain INPUT (policy ACCEPT 27 packets, 3798 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp      resin-vpn any     anywhere             anywhere             tcp dpt:http
    0     0 REJECT     tcp      any    any     anywhere             anywhere             tcp dpt:http reject-with icmp6-port-unreachable

#21

Hi @xandfury, I had a little play around with this on my RPI3 (OS 2.29.2) and used the following rules:

iptables -A INPUT -j ACCEPT -p tcp --destination-port 80 -i resin-vpn
iptables -A INPUT -j DROP -p tcp --destination-port 80

which seems to work for my test (a simple node.js server on port 80) and gives the following on my device.

root@9144766:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION
-N DOCKER-USER
-A INPUT -i supervisor0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i lo -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i docker0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i tun0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i resin-vpn -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 48484 -j REJECT --reject-with icmp-port-unreachable
-A INPUT -i wlan0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i resin-vpn -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j DROP
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o br-33ddcd4cfdea -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-33ddcd4cfdea -j DOCKER
-A FORWARD -i br-33ddcd4cfdea ! -o br-33ddcd4cfdea -j ACCEPT
-A FORWARD -i br-33ddcd4cfdea -o br-33ddcd4cfdea -j ACCEPT
-A FORWARD -o supervisor0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o supervisor0 -j DOCKER
-A FORWARD -i supervisor0 ! -o supervisor0 -j ACCEPT
-A FORWARD -i supervisor0 -o supervisor0 -j ACCEPT
-A FORWARD -o balena0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o balena0 -j DOCKER
-A FORWARD -i balena0 ! -o balena0 -j ACCEPT
-A FORWARD -i balena0 -o balena0 -j ACCEPT
-A DOCKER-ISOLATION -i supervisor0 -o br-33ddcd4cfdea -j DROP
-A DOCKER-ISOLATION -i br-33ddcd4cfdea -o supervisor0 -j DROP
-A DOCKER-ISOLATION -i balena0 -o br-33ddcd4cfdea -j DROP
-A DOCKER-ISOLATION -i br-33ddcd4cfdea -o balena0 -j DROP
-A DOCKER-ISOLATION -i balena0 -o supervisor0 -j DROP
-A DOCKER-ISOLATION -i supervisor0 -o balena0 -j DROP
-A DOCKER-ISOLATION -j RETURN
-A DOCKER-USER -j RETURN

#22

Thanks for the reply @shaunmulligan . I deleted some of my earlier rules to match those mention by you. For some reason this does not work with my device :confused:

root@c4773ae:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION
-N DOCKER-USER
-A INPUT -i supervisor0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i lo -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i docker0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i tun0 -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -i resin-vpn -p tcp -m tcp --dport 48484 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 48484 -j REJECT --reject-with icmp-port-unreachable
-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i wlan0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i resin-vpn -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j DROP
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o balena0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o balena0 -j DOCKER
-A FORWARD -i balena0 ! -o balena0 -j ACCEPT
-A FORWARD -i balena0 -o balena0 -j ACCEPT
-A FORWARD -o supervisor0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o supervisor0 -j DOCKER
-A FORWARD -i supervisor0 ! -o supervisor0 -j ACCEPT
-A FORWARD -i supervisor0 -o supervisor0 -j ACCEPT
-A FORWARD -o br-dfd0deb2f977 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-dfd0deb2f977 -j DOCKER
-A FORWARD -i br-dfd0deb2f977 ! -o br-dfd0deb2f977 -j ACCEPT
-A FORWARD -i br-dfd0deb2f977 -o br-dfd0deb2f977 -j ACCEPT
-A DOCKER -d 172.18.0.2/32 ! -i br-dfd0deb2f977 -o br-dfd0deb2f977 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION -i supervisor0 -o balena0 -j DROP
-A DOCKER-ISOLATION -i balena0 -o supervisor0 -j DROP
-A DOCKER-ISOLATION -i br-dfd0deb2f977 -o balena0 -j DROP
-A DOCKER-ISOLATION -i balena0 -o br-dfd0deb2f977 -j DROP
-A DOCKER-ISOLATION -i br-dfd0deb2f977 -o supervisor0 -j DROP
-A DOCKER-ISOLATION -i supervisor0 -o br-dfd0deb2f977 -j DROP
-A DOCKER-ISOLATION -j RETURN
-A DOCKER-USER -j RETURN

Here are my active rules:

root@c4773ae:~# iptables -vL
Chain INPUT (policy ACCEPT 471 packets, 51948 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  supervisor0 any     anywhere             anywhere             tcp dpt:48484
  127  8716 ACCEPT     tcp  --  lo     any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  docker0 any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  tun0   any     anywhere             anywhere             tcp dpt:48484
    0     0 ACCEPT     tcp  --  resin-vpn any     anywhere             anywhere             tcp dpt:48484
    0     0 REJECT     tcp  --  any    any     anywhere             anywhere             tcp dpt:48484 reject-with icmp-port-unreachable
    0     0 DROP       tcp  --  eth0   any     anywhere             anywhere             tcp dpt:http
    0     0 DROP       tcp  --  wlan0  any     anywhere             anywhere             tcp dpt:http
    0     0 ACCEPT     tcp  --  resin-vpn any     anywhere             anywhere             tcp dpt:http
    0     0 DROP       tcp  --  any    any     anywhere             anywhere             tcp dpt:http

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 7446 7999K DOCKER-USER  all  --  any    any     anywhere             anywhere            
 7446 7999K DOCKER-ISOLATION  all  --  any    any     anywhere             anywhere            
    0     0 ACCEPT     all  --  any    balena0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    balena0  anywhere             anywhere            
    0     0 ACCEPT     all  --  balena0 !balena0  anywhere             anywhere            
    0     0 ACCEPT     all  --  balena0 balena0  anywhere             anywhere            
    0     0 ACCEPT     all  --  any    supervisor0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    supervisor0  anywhere             anywhere            
    0     0 ACCEPT     all  --  supervisor0 !supervisor0  anywhere             anywhere            
    0     0 ACCEPT     all  --  supervisor0 supervisor0  anywhere             anywhere            
 3808  325K ACCEPT     all  --  any    br-dfd0deb2f977  anywhere             anywhere             ctstate RELATED,ESTABLISHED
   36  2160 DOCKER     all  --  any    br-dfd0deb2f977  anywhere             anywhere            
 3602 7672K ACCEPT     all  --  br-dfd0deb2f977 !br-dfd0deb2f977  anywhere             anywhere            
    0     0 ACCEPT     all  --  br-dfd0deb2f977 br-dfd0deb2f977  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 518 packets, 59818 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (3 references)
 pkts bytes target     prot opt in     out     source               destination         
   36  2160 ACCEPT     tcp  --  !br-dfd0deb2f977 br-dfd0deb2f977  anywhere             172.18.0.2           tcp dpt:http

Chain DOCKER-ISOLATION (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  supervisor0 balena0  anywhere             anywhere            
    0     0 DROP       all  --  balena0 supervisor0  anywhere             anywhere            
    0     0 DROP       all  --  br-dfd0deb2f977 balena0  anywhere             anywhere            
    0     0 DROP       all  --  balena0 br-dfd0deb2f977  anywhere             anywhere            
    0     0 DROP       all  --  br-dfd0deb2f977 supervisor0  anywhere             anywhere            
    0     0 DROP       all  --  supervisor0 br-dfd0deb2f977  anywhere             anywhere            
 7446 7999K RETURN     all  --  any    any     anywhere             anywhere            

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 7446 7999K RETURN     all  --  any    any     anywhere             anywhere        

IP address of my device running balena

root@c4773ae:~# ip address show eth0 | grep "inet 192"
    inet 192.168.1.161/24 brd 192.168.1.255 scope global dynamic eth0
root@c4773ae:~# 

I can still access Node-RED from my laptop (192.168.1.110) as shown in the following screen shot:

image

I used incognito mode to ensure this was not a cached page.


#24

I am using Raspberry Pi 3 Model B + and OS version is balenaOS 2.29.2+rev1 - production


#25

Thanks @xandfury , could you also let us know what device-type and OS version you are running on. Also if you have a spare device lying around, perhaps you could try do the setup on a freshly provisioned device and see if that affects things (just to rule out some older changes in networking)


#29

@shaunmulligan I mention device-type and OS version in a separate comment above :slight_smile: Unfortunately I do not have a spare device with me. So can’t test this with a fresh install.

On a side-note, my iptable rules do not persist over reboot as well.


#31

@xandfury if you can send us the device UUID and grant support access to it, we can dig a bit more deeply into it.


#107

The confusion here is due to different networking setups depending on if you are using a multicontainer setup or not.

In a single container setup, the container operates in host networking mode so you can filter it using the INPUT chain:

iptables -A INPUT -i resin-vpn -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

When using multicontainer, Docker will create DNAT rules in the nat tables and filtering should be performed on the DOCKER-USER chain:

iptables -I DOCKER-USER -p tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
iptables -I DOCKER-USER -i resin-vpn -p tcp --dport 80 -j RETURN

This will leave you with a DOCKER-USER chain looking something like this:

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     tcp  --  resin-vpn *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80
    0     0 REJECT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 reject-with icmp-port-unreachable
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

#111

@wrboyce This worked! Thanks a lot :star_struck:

I currently have a single container setup. Not so long ago on the same device I had multiple containers. Is there any chance I can do this without actually ssh`ing onto the hostOS?

Also is it safe to assume that these rules would persist across reboot?


#123

Hi @xandfury,

We currently don’t have a way to let the user apply iptables rules without ssh’ing and they will not be persistent after reboot.

What you can do in the context of a single container application is to add in your application the instructions to modify the rules. This way every time your application starts it will modify the rules.

Regards!


#124

Thanks @spanceac @wrboyce @shaunmulligan!
I did some digging and found a work-around for this. I am using NetworkManger to populate my iptable rules when the device is rebooted. Here is what I did:

  1. Create a file /etc/NetworkManager/dispatcher.d/pre-up.d/iptables.sh with the following:
#!/bin/sh
 
LOGFILE=/var/log/iptables.log
IPTABLES_RESTORE=$(which iptables-restore)
 
if [ "$1" = lo ]; then
    echo "$0: ignoring $1 for \`$2'" >> $LOGFILE
    exit 0
fi
 
case "$2" in
    pre-up)
        echo "$0: restoring iptables rules for $1" >> $LOGFILE
        $IPTABLES_RESTORE /etc/network/iptables.up.rules >> $LOGFILE 2>&1
        ;;
    *)
        echo "$0: nothing to do with $1 for \`$2'" >> $LOGFILE
        ;;
esac
 
exit 0
  1. Give the permissions to the script with chmod a+x /etc/NetworkManager/dispatcher.d/pre-up.d/iptables.sh
  2. Apply your iptable rules and verify them
root@test:~# iptables -I DOCKER-USER -p tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
root@test:~# iptables -I DOCKER-USER -i resin-vpn -p tcp --dport 80 -j RETURN
root@test:~# iptables -A INPUT -i resin-vpn -p tcp --dport 80 -j ACCEPT
root@test:~# iptables -A INPUT -p tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
  1. Create a file /etc/network/iptables.up.rules and add all my rules to it
iptables-save > /etc/network/iptables.up.rules
  1. Re-boot the device and check the contents of /var/log/iptables.log. If everything went well, you should see the rules applied :slightly_smiling_face:

Can anyone of you please comment on this? Will this be safe in a commercial/production environment?

`


#130

@xandfury it looks like you are modifying the NetworkManager dispatcher configuration on the host OS through SSH, right? This is not safe for production environment as any host OS update will wipe out that modification. Is there any issue with modifying the iptable rules at the very beginning of the start shell script of your container? This way whenever your application starts afterwards those will be already applied.


#131

@majorz

it looks like you are modifying the NetworkManager dispatcher configuration on the host OS through SSH, right?

Correct!

Is there any issue with modifying the iptable rules at the very beginning of the start shell script of your container? This way whenever your application starts afterwards those will be already applied.

That honestly would’ve been the best solution. The problem with that is resin-vpn interface is not available on my container


#132

@xandfury you mentioned you are using a single-container application, so the resin-vpn interface should be available, at least I can see it on a device I have here. If you are using a multi-container setup, then you will need to enable host networking and privileged mode for the particular container. Please let me know if what I say does not work for any reason.


#133

@majorz

Ah… It was multi-container initially. I removed all services except one - so now single container.
Just made changes to the docker-compose. I see the interfaces now :slightly_smiling_face:

Tbh making changes to NetworkManager dispatcher config felt a bit too hackish. :sweat_smile: This should definitely solve my issue.

Thanks for the help!


#134

I found that the following worked well for me

iptables -A INPUT -i resin-vpn -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -i resin-vpn -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j REJECT