This feels very similar to Redsocks proxy not working - #27 by Nodens … but I didn’t want to revive someone else’s 3-year old thread…
balenaOS 5.1.0 (generic-amd64) … also tried a few builds of balenaOS 4.x, same result.
(balenaEngine v20.10.41 on the 5.1.0 os)
balenaOS is running inside a VM (on VMWare Workstation … but also tried a physical box, it is just faster to iterate with the VM). The VM’s network adapter is set to “Host-only: A private network shared with the host”.
The host is running squid, all properly configured to be an http-connect proxy (no auth or anything fancy).
Inside the balenaOS image, running curl https://www.google.com
fails properly. Running curl -x http://192.168.238.1:8080 --proxytunnel -k https://www.google.com
works great, and if I pass -v
to curl I see all the proxy CONNECT requests … similarly, the hosts Squid proxy access.log
shows all the requests I do via curl.
Ok, /mnt/boot/redsocks.conf
has:
base {
log_debug = off;
log_info = on;
log = stderr;
daemon = off;
redirector = iptables;
}
redsocks {
type = http-connect;
ip = 192.168.231.1;
port = 8080;
local_ip = 127.0.0.1;
local_port = 12345;
}
(I tried changing local_ip
to 0.0.0.0
and a few other values, all with no changes).
The output of journalctl -u balena-proxy-config
has:
Dec 23 21:36:36 0011223 sh[1599]: balena-proxy-config: Found config.json in /mnt/boot/config.json .
Dec 23 21:36:36 0011223 sh[1602]: iptables: No chain/target/match by that name.
Dec 23 21:36:36 0011223 sh[1613]: iptables: No chain/target/match by that name.
Dec 23 21:36:36 0011223 sh[1614]: iptables v1.8.7 (legacy): Couldn't load target `REDSOCKS':No such file or directory
Dec 23 21:36:36 0011223 sh[1614]: Try `iptables -h' or 'iptables --help' for more information.
Dec 23 21:36:36 0011223 sh[1615]: iptables v1.8.7 (legacy): Couldn't load target `REDSOCKS':No such file or directory
Dec 23 21:36:36 0011223 sh[1615]: Try `iptables -h' or 'iptables --help' for more information.
Dec 23 21:36:36 0011223 sh[1616]: iptables v1.8.7 (legacy): Couldn't load target `REDSOCKS':No such file or directory
Dec 23 21:36:36 0011223 sh[1616]: Try `iptables -h' or 'iptables --help' for more information.
Dec 23 21:36:36 0011223 sh[1617]: iptables: No chain/target/match by that name.
Dec 23 21:36:36 0011223 sh[1618]: iptables: No chain/target/match by that name.
The output of iptables -L
has:
Chain INPUT (policy ACCEPT)
target prot opt source destination
BALENA-FIREWALL all -- anywhere anywhere
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain BALENA-FIREWALL (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere ADDRTYPE match src-type LOCAL
ACCEPT tcp -- anywhere anywhere tcp dpt:48484
ACCEPT tcp -- anywhere anywhere tcp dpt:48484
ACCEPT tcp -- anywhere anywhere tcp dpt:48484
ACCEPT tcp -- anywhere anywhere tcp dpt:48484
ACCEPT tcp -- anywhere anywhere tcp dpt:48484
REJECT tcp -- anywhere anywhere tcp dpt:48484 reject-with icmp-port-unreachable
ACCEPT tcp -- anywhere anywhere tcp dpt:22222
ACCEPT tcp -- anywhere anywhere tcp dpt:2375
ACCEPT all -- anywhere anywhere ADDRTYPE match dst-type MULTICAST
ACCEPT icmp -- anywhere anywhere
ACCEPT udp -- anywhere anywhere udp dpt:domain
RETURN all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain DOCKER (2 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Ok, so the Couldn't load target 'REDSOCKS':No such file or directory
line made me think it was just balena-proxy-config
not creating the REDSOCKS
custom-chain in iptables, so I ran iptables -N REDSOCKS
, then systemctl restart balena-proxyconfig
. That got rid of the REDSOCKS not found
error, and now journalctl -u balena-proxy-config
just has:
Dec 23 21:39:36 0011223 sh[2245]: balena-proxy-config: Found config.json in /mnt/boot/config.json .
Dec 23 21:39:36 0011223 sh[2246]: iptables: No chain/target/match by that name.
Dec 23 21:39:36 0011223 sh[2247]: iptables: No chain/target/match by that name.
but nothing was populated in iptables for the proxy (new lines at the end of iptables -L
):
Chain REDSOCKS (0 references)
target prot opt source destination
The logs from balena-supervisor don’t seem of any use, as it is just failing to communicate due to the lack of proxy config:
Dec 23 22:02:35 0011223 balena-supervisor[1878]: Error response from daemon: No such container: resin_supervisor
Dec 23 22:02:35 0011223 balena-supervisor[1885]: balena_supervisor
Dec 23 22:02:35 0011223 balena-supervisor[1891]: active
Dec 23 22:02:35 0011223 balena-supervisor[1892]: [gen-conf-unit][INFO] Generating configuration unit for balena-supervisor in /var/volatile
Dec 23 22:02:35 0011223 gen-conf-unit[1895]: Generating configuration unit for balena-supervisor in /var/volatile
Dec 23 22:02:35 0011223 balena-supervisor[1892]: [gen-conf-unit][INFO] New balena-supervisor configuration:
Dec 23 22:02:35 0011223 gen-conf-unit[1907]: New balena-supervisor configuration:
Dec 23 22:02:35 0011223 balena-supervisor[1908]: {
Dec 23 22:02:35 0011223 balena-supervisor[1908]: "listenPort": 48484
Dec 23 22:02:35 0011223 balena-supervisor[1908]: }
Dec 23 22:02:35 0011223 balena-supervisor[1919]: balena_supervisor
Dec 23 22:02:35 0011223 balena-supervisor[1955]: Error response from daemon: No such container: resin_supervisor
Dec 23 22:02:35 0011223 balena-supervisor[1962]: balena_supervisor
Dec 23 22:02:35 0011223 balena-supervisor[1968]: active
Dec 23 22:02:35 0011223 balena-supervisor[1971]: Container config has not changed
Dec 23 22:02:36 0011223 balena-supervisor[2024]: INFO: Found device /dev/sda1 on current boot device sda, using as mount for '(resin|balena)-boot'.
Dec 23 22:02:36 0011223 balena-supervisor[2024]: INFO: Found device /dev/sda4 on current boot device sda, using as mount for '(resin|balena)-state'.
Dec 23 22:02:36 0011223 balena-supervisor[2024]: INFO: Found device /dev/sda5 on current boot device sda, using as mount for '(resin|balena)-data'.
Dec 23 22:02:36 0011223 balena-supervisor[2024]: find: /mnt/root/tmp/balena-supervisor/services: No such file or directory
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [info] Supervisor v15.0.4 starting up...
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [info] Setting host to discoverable
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Starting systemd unit: avahi-daemon.service
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Starting systemd unit: avahi-daemon.socket
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Starting logging infrastructure
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [info] Starting firewall
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [warn] Invalid firewall mode: . Reverting to state: off
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [info] Applying firewall mode: off
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [success] Firewall mode applied
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Starting api binder
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Performing database cleanup for container log timestamps
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [info] Previous engine snapshot was not stored. Skipping cleanup.
Dec 23 22:02:37 0011223 balena-supervisor[2024]: [debug] Handling of local mode switch is completed
Dec 23 22:02:37 0011223 balena-supervisor[2024]: (node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Dec 23 22:02:37 0011223 balena-supervisor[2024]: (Use `node --trace-deprecation ...` to show where the warning was created)
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] API Binder bound to: https://api.my-open-balena.com/v6/
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [event] Event: Supervisor start {}
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Starting API server
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Supervisor API successfully started on port 48484
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [debug] Ensuring device is provisioned
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [debug] Connectivity check enabled: true
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [debug] Starting periodic check for IP addresses
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Reporting initial state, supervisor version and API info
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Waiting for connectivity...
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] VPN connection is not active.
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [debug] Skipping preloading
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Applying target state
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [info] Reporting initial configuration
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [debug] Finished applying target state
Dec 23 22:02:38 0011223 balena-supervisor[2024]: [success] Device state apply success
Dec 23 22:02:53 0011223 balena-supervisor[2024]: [error] LogBackend: unexpected error: Error: getaddrinfo EAI_AGAIN api.my-open-balena.com
Dec 23 22:02:53 0011223 balena-supervisor[2024]: [error] at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:119:26)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] Error reporting initial configuration, will retry RequestError: getaddrinfo EAI_AGAIN api.my-open-balena.com
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at ClientRequest.<anonymous> (/usr/src/app/dist/87.app.js:1:158220)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at Object.onceWrapper (node:events:632:26)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at ClientRequest.emit (node:events:529:35)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at _destroy (node:_http_client:882:13)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at onSocketNT (node:_http_client:902:5)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
Dec 23 22:03:26 0011223 balena-supervisor[2024]: [error] at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:119:26)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] Error reporting initial configuration, will retry RequestError: getaddrinfo EAI_AGAIN api.my-open-balena.com
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at ClientRequest.<anonymous> (/usr/src/app/dist/87.app.js:1:158220)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at Object.onceWrapper (node:events:632:26)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at ClientRequest.emit (node:events:529:35)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at _destroy (node:_http_client:882:13)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at onSocketNT (node:_http_client:902:5)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
Dec 23 22:04:44 0011223 balena-supervisor[2024]: [error] at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:119:26)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] Error reporting initial configuration, will retry RequestError: getaddrinfo EAI_AGAIN api.my-open-balena.com
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at ClientRequest.<anonymous> (/usr/src/app/dist/87.app.js:1:158220)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at Object.onceWrapper (node:events:632:26)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at ClientRequest.emit (node:events:529:35)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at _destroy (node:_http_client:882:13)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at onSocketNT (node:_http_client:902:5)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
Dec 23 22:06:02 0011223 balena-supervisor[2024]: [error] at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:119:26)