Deployed devices losing Balena connection after exactly 12hrs

Product support balenaOS
1

I have a number of devices which are deployed using Balena Cloud onto Raspberry Pi 4s. The devices obtain their connectivity via an LTE Hat. After exactly 12 hours, the devices lose connection with Balena console and therefore show “offline” and I am unable to interact with them remotely. I have performed a number of tests and exactly 12 hours is consistent. A power reboot brings them back online again, but for only 12hrs.

I am running the latest Balena OS 6.0.10+rev 2 and Supervisor 16.6.1. I’m thinking it is something to do with a DHCP lease or similar? Any one experience similar issues and able to assist in resolving this pressing issue please?

Thank you in advance.

2

Hello @Tissy

Could you please share more details of the connectivity?

What is the LTE company that you are using? Could you please confirm that there is any issue on the connectivity through their portal?

3

Hi @mpous, thank you for your reply.

The connectivity is provided using a WaveShare LTE Hat which the rest of the time works flawlessly ( Waveshare - EM060K-GL LTE Cat-6 HAT).

The LTE Company is one which specialises in IoT devices and connection to their network is via a single APN, however they will connect to which ever service is available and strongest in the area of operation. The SIM is provided by https://infisim.com/.

I have experienced no issues on connectivity apart from the 12hrs drop out. I have even tried a SIM which was not as provided by InfiSIM, so a straight forward Vodafone SIM and experienced the same issue.

Thank you for your help :slight_smile:

4

That’s weird! Are all the devices together at the same location during the 12h drop out?

Do you think you can enable support access to your devices via direct message, so we can check your devices?

Thanks

5

Thank you for your help @mpous. The devices are at different locations when they drop out.

I have enabled the support access, what do I need to send to you via DM please? Is it just the fleet name?

I cycled the power at 2336hrs GMT, so I will expect a drop out at 1136hrs GMT in the morning.

Thank you,

Steve

6

@Tissy send me the complete UUIDs by DM please! Thanks!

7

Hello @Tissy i see that your device is connected using Ethernet. Is that correct?

Is the device behind a firewall? it sounds like a DHCP lease renewal issue or similar.

Finally could you please confirm the LTE hat that you are using?

8

Hi Mark,

The device is not connected via ethernet, however it does have an ethernet IP address assigned as there is an SDR connected to the Raspberry Pi via the internal ethernet port. Therefore it is not behind a Firewall.

Connectivity to the “outside world” is via a 4G HAT which is this make and model - EM060K-GL LTE Cat-6 HAT - Waveshare Wiki

Hope that helps :slight_smile:

9

@Tissy could you please confirm if you have tested these devices running vanilla Linux? Do you observe the same behaviour? We would like to discard if this is an issue of the LTE modem or not.

Thanks

10

Hi Mark, Its a good point and no I haven’t.

I shall do that test over the weekend. A Vanilla version of Linux may not have the same logging capabilities as Balena, so I am still interested if anything is picked up. It’s 12hr drop out has just occurred. At what stage should I be able to access the persistent logs please and where are they located?

I also tried manually resetting the 4G HAT whilst the Pi is running which made no difference, device is still offline

Whilst researching, I also came across this thread, does that have any similarities to my issue perhaps?

IoT2040 running Balena OS 2.12.7+rev1 stop working after 2 days - Product support / balenaOS - balenaForums

Many thanks.