Any exceptions when routing through a proxy

I am looking to route all the balenaOS traffic, deployed from OpenBalena through a proxy. This guide is helpful: https://www.balena.io/docs/reference/OS/network/2.x/#connecting-behind-a-proxy

I’m curious as to the different interfaces BalenaOS has and uses, and what would be the exception, if any, to this proxy change? Containers will circumvent this rule and connect directly I presume? Other than that, will all traffic from the device to the internet be forced through this proxy?

Hi @Maggie,

So as you’ll have seen, the networking requirements show all the main ports that balena requires, as well as some of the external host details. The OS will route all TCP traffic via redsocks should it be configured, including all container traffic. UDP is slightly different, redudp only deals with DNS traffic, and any other UDP traffic will still attempt to use the default interfaces.

Please let us know if this helps, or if you need any more detail!

Best regards,

Heds

Most helpful, thanks.

Is there any recommendations on how best to route UDP traffic through a proxy too? Or perhaps a better way to put it, a strategy to route all traffic through a proxy?

At the moment I am thinking the best strategy would be to have tiny proxy (https://github.com/tinyproxy/tinyproxy) running in a container, and I could route all the traffic from the device through that container. The front facing interface of that container to the internet could then consist of any setup, VPNs, web proxies etc. So the question in this scenario, would be how to route all BalenaOS traffic to a proxy (in this scenario, a local proxy running in a container, although of course the process for routing to a proxy whether it is local or remote I imagine will not differ)?

Here is some further details of what is trying to be achieved. Balena is being deployed to devices in humanitarian contexts, working out of schools. While the content and purpose is completely legitimate, locations of schools can be information you may not want to be revealed, as they have often become victims of attacks. At the moment, routing the TCP traffic is a start. The X factor here seems to be ‘0.resinio.pool.ntp.org’ NTP servers which are quite unique identifiers of the type of devices being used. Potentially also the DNS resolution, especially if pulling from the local router the DNS server addresses if DNS isn’t routed through the redsocks proxy?

Not easy questions, or scenarios I realise.

To note, I am aware of https://www.balena.io/docs/reference/OS/time/ and https://github.com/balena-os/meta-balena#dnsservers. Adjusting these may tackle immediate issues, but doesn’t seem as efficient as routing all traffic.

Would also be helpful just to clarify your earlier comment:

UDP is slightly different, redudp only deals with DNS traffic, and any other UDP traffic will still attempt to use the default interfaces.

You are saying here that DNS is routed through the proxy, it is only the other traffic that isn’t?

Hi again @Maggie,

That’s a really useful and detailed scenario, thank you for the explanation!

Whilst I think you’re right that you could use tinyproxy, due to the way the routing works out of the box you’d obviously need to use this in host network mode and rewrite some the netfilter rules to change the default route. There’s potentially a problem here. In the case that something happens to the balenaEngine or your application and the proxy isn’t present, it would potentially become impossible to access the device to recover it. Currently traffic is automatically routed via whichever NIC is currently Internet connected. This means we can usually guarantee access when a network is present. I’d be worried in this situation that in an instance where, for some reason, the application service dies (or as I say balenaEngine), you’d end in a situation where we could not remotely recover and reprovision the device (without a reboot, which would obviously reset the rules).

I don’t believe that we send any UDP traffic apart from DNS and NTP and as you’ve seen, we do allow the configuration of hosts to use for these (and if they were local to the network then of course no location information would leak).

On clarifying my comment, yes, that’s exactly right. DNS traffic will be routed through redup but no other UDP traffic (so you’d still have an issue with NTP for example). I believe there is an ability to allow UDP traffic to also get routed via a proxy, but it involves a kernel option that we don’t configure in the kernel for OS releases.

What I’m going to do is make the product manager and another couple of colleagues aware of this thread so we can throw around a few ideas. We’ll obviously let you know, but I think for now the best option to ensure recoverability of a device is to use redsocks and configure the DNS/NTP hosts to use locally.

Best regards,

Heds

I have recently came accross shadowsocks which seems to support transparent udp/tcp proxying.

It would be great if shadowsocks could be integrated to balena-os as an alternative to redsocks.

1 Like

Hi,

Thank you for your suggestion, we will discuss it internally and update this tread once we know more.