Massive data usage from devices

I installed the first couple of my Balena devices on Thursday, and have just had an email from our data provider asking if our data usage was expected. These two devices have used 3.5GB of data over the weekend. Other devices running identical code have used 250MB since the start of October. The only difference between the devices is the Balena.

Can anyone explain this? We can’t afford to keep running the devices with such high data usage.

One thing I’ve noticed is that the azure iot hub nodes I am using in node red are sending the entire data packet to balena as well in the device logs. Could this be part of the issue? I wouldn’t have though it would have caused such a high increase in data usage though.

I’ve disabled the device logs, and it has reduced the data usage by 10x. The usage is still about 3 times as much as my other devices. What else can I do to reduce this?

Hey @GregorR1, the data usage is mainly comprised of:

  • device logs (as you discovered): this you either disable logs, or control what your application is logging
  • container updates: when you update your application it will pull the updated images. This you can reduce with delta updates (see our delta updates docs)
  • very small numbers from other sources, which should not be significant, but worth mentioning: web terminal if you are connecting to the device, VPN keepalives, NTP time updatesetc. There is another post where were we measured and gave some advice minimizing these last group, just the sake of completeness: Updated bandwidth usage numbers

These are regular cases. Irregular case would be if there’s a device that is not working properly (for example repeatedly downloading images due to some issues, or other cases that we cannot guess from this info so far). Thus did you see anything strange in the logs before you disabled them?

Or, if you would like our support agents to take a look, you can enable support access in the device’s dropdown actions menu, and send us the device’s UUID, and we can take a look, if there’s anything specific.

Hi @imrehg,

Now that I’ve turned off the logs, it appears to have settled down to a more reasonable amount so I believe this was the issue. I’m going to look into using the delta updates as we need to try and keep any updates to as little data as possible - we didn’t push any updates over the weekend so this wasn’t the cause of the massive data usage, but we will be pushing updates at some point in the future.

Thanks for your reply.

@GregorR1 thanks for the update! Yeah, logs can be tricky, depending on what’s happening on the device. Thus our suggestion is not necessarily to disable the logs on devices, but adjust your application to log the things that are more important. That way you can still see what’s happening, but don’t use up too much data. But disabling logs is indeed a quick way to cut back while checking the cause.

Hope all will work out soon for your use case! :slight_smile:

I believe that it is the azure IoT nodes in node-red that are the main cause of the issue as every time a message is sent to the IoTHub there is a log sent with the entire data packet and then a confirmation message to say that the data has been sent successfully.

My program was sending data every 5 seconds, so it was a lot of messages being sent.

Indeed, that would explain it. I guess there might be an Azure IoT node-red module setting to reduce the log level, but not totally sure. Indeed that sounds like a lot of logs. I also took some notes from your question for possible future features, that would help debugging data usage easier. Thanks a lot for letting us know of this case, and glad that it seems clearer!

As far as I can see, there is no settings currently available for this specific node-red nodes. I’ll add a feature request to the node’s git repo.

My main issue was that I couldn’t accurately see how much data usage I was using for each feature during the development of the device - and as we are testing in our offices it was through wifi for the majority of the time so data usage wasn’t such a high concern.

The ability to monitor device data usage on development images would be amazing - but I don’t know how difficult this would be to implement.

Thanks for the feedback, @GregorR1, yeah the “provide more more monitoring so users can make better decisions” is definitely an approach we are taking. It’s work in progress, so not really sure what any timeline would be, but we have more useful context and ideas of how it would look, from feedback like yours.

Indeed product development requires a lot of information, and if you hit any other issues, or get confused by any other behaviour while developing, please let us know as well! Thanks a lot for raising the issue for the upstream project too, that will definitely benefit more users as well :slight_smile: