Recently we got a few questions regarding the resin.io bandwidth usage, whether or not the numbers mentioned in our Device Bandwidth/Data usage blogpost and documentation are still accurate or not.
We’ve decided to revisit it, and take some more notes on how did we test, so it’s easy to replicate in the future. 
Setup
The setup we had was the following:
We had two Raspberry Pi 3 running the latest 2.3.0+rev1 resinOS, freshly provisioned. Choose two to run two tests in parallel: the default/not-adjusted resinOS behaviour for a baseline, and a bandwidth minimized behaviour.
Both devices were connected over Ethernet to measurement laptops, that shared their wifi to the Ethernet. That made it really easy to separate the test devices’ traffic from elsewhere.
Run Wireshark on the laptop to record all the traffic on the Ethernet interface to the test devices.
The devices had the following application deployed:
# Dockerfile.template
FROM resin/%%RESIN_MACHINE_NAME%%-alpine
CMD while : ; do if [ ! -z "${DEBUG}" ]; then echo "Hello..."; fi ; sleep ${INTERVAL=10}; done
By default this application just continuously sleeps, but could enable log output by setting the DEBUG environment to anything at all, and can adjust the frequency of log entries by setting the INTERVAL env var. In these tests we’ve kept everything the default (that is no log output), so that we can benchmark the bandwidth usage of resinOS alone.
The “baseline” device was not modified in any way, while the “minimal traffic” device had these variables set (as mentioned in the docs):
-
RESIN_SUPERVISOR_VPN_CONTROLtofalse -
RESIN_SUPERVISOR_CONNECTIVITY_CHECKtofalse -
RESIN_SUPERVISOR_POLL_INTERVALto86400000 -
RESIN_SUPERVISOR_LOG_CONTROLtofalse
This would minimize the amount data the device transfers, and still would enable to control the device (as it checks in once every 24h=86400000ms).
After these settings the devices were rebooted, and the recording started 30 minutes after restart (to keep it similar to the blogpost).
Wireshark recorded the traffic for about 134 hours (5.5 days) or so, to have some decent averaging.
Results
Taking the Wireshark records, we can filter out some traffic, that is due to fact that the devices are on a LAN, and it’s not due to resinOS, but because of the network environment (and e.g. would not be present in a 3G modem setting). The filter we’ve used for the analysis removes mdns (Avahi) traffic, DHCP updates (the “type==53” filter below), and address resolution protocol (ARP) packages:
not mdns && not (bootp.option.type == 53) && not arp
What’s left is the actual communication tcp/ssl/tls traffic, the ntp network time protocol traffic, dns DNS resolution traffic, and a couple of icmp packages.
Baseline
For the baseline device, the bandwidth usage results come in ~around 145 B/s ~= 11.95 MB/day ~= 358 MB/month (30 days) (see the “Displayed” column in the results below:
The details are:
- TLS traffic: averages around 127 B/s, of which is about 89 B/s to our API and 38 B/s to our VPN (filtered in the results with
ip.addr == a.b.c.dwhere the IPs ofapi.resin.ioandvpn.resin.iowere filled in - DNS/ICMP traffic: averages around 17 B/s
- NTP: 2*90 bytes messages every ~2000s (as
systemd-timesyncdsynchronizes time), or ~0.09 B/s
So the breakdown would be something like
- API traffic: 61%
- VPN traffic: 26%
- DNS traffic: 12%
- NTP traffic: << 1.%
This result is better than the previous test’s when about 527MB/month was extrapolated (see the linked blogpost).
Minimal
The bandwidth minimization resulted in an average of 0.283 B/s ~= 23.9 kB/day ~= 716.3 kB/month (which is ~0.2% of the baseline traffic): (See the “Displayed” column below, when the appropriate filters were applied):
- TCP traffic: 0.172 B/s, all of it to the API, there was zero VPN traffic found, as it is correct due to the settings; the API calls show up ~every 24h in the logs also just as it was set
- DNS/ICMP traffic: 0.021 B/s DNS traffic was averaged, since much less traffic needed DNS resolution
- NTP traffic: got ~0.09 B/s, which is the same as in the baseline case, as the settings did not affect NTP in any way.
So the breakdown would be something like:
- API traffic: 61%
- DNS traffic: 7%
- NTP traffic: 32%
This is again better than before, about 50% of the previous minimal bandwidth usage estimate.
Future tests
This should be a good cross-check to show that previous bandwidth estimates are still in the right ballpark, and if anything those are too conservative. It was a bit contrived setup (filtering out local network traffic, etc, even if justified). Most of our users who want minimal bandwidth usage are on metered 3G connections. For more realistic setup we’ll be setting up some devices on the 3G network, and use the network provider’s traffic estimate to confirm these numbers above.
Would be happy to hear any feedback or comments on these tests or your network traffic minimization setup! 
