Hi there, I’m getting an issue with one of our device running Screenly OSE deployed with balena, I get “Request error: tunneling socket could not be established, statusCode=500” and I have “Heartbeat not received” with Online (VPN only). I have try to restart services and also to reboot the device but I can’t do it remotely, ATM I have activate “Grant support access”, what should I do to get support?
I’m new to balena’s world, sorry
Thanks for the great project.
Best regards,
Alessandro
Hi, thanks for contacting Balena support and welcome to the community. The first thing I would suggest is to check that your network complies with Balena requirements as stated in Network Setup on balenaOS 2.x - Balena Documentation.
If that’s the case, you will need to share the UUID of the device after enabling support access so we can take a look. I would also recommend that you set a long support access in case our support team does not get to it straight away.
Hello alexgg and thankyou for your reply!
On the same network I have other 3 devices running the same configuration and they are working great, also this one worked great for a couple of day (I have deployed them the last friday), but today it stopped to work.
Device UUID is 73e60b84ee1671078f6a4c3aef689da6 and I have enabled long time support (the longest I could enable).
Should you give me help?
Thanks in advance,
Alessandro
Hi Alessandro, I tried accessing the device but it does not seem to be accessible. Could you maybe give us support access to one of the working devices on the same network? We can use it as a gateway to try an access the device from the local network.
Sure! Here UUID: 9dc7c8977c0e67746a147e1623e5c788
Hi. Unfortunately I am unable to access the problematic device at all. Do you have physical access to the device 73e60b84ee1671078f6a4c3aef689da6
?
No Ereski, unfortunately no, not now for sure, maybe in the next days if I have the possibility to go there, sorry
[EDIT]: maybe my fault, but should I enable also public IP to let you connect to the devices? In this case I’ll enable it but ATM the one with the issue is offline so I don’t think it will be reachable anyway [/EDIT]
Hi Alessandro, the Public Device URL feature is not applicable in this case, so no worries, and there is no need to enable that. I am glad to hear that th three are devices are functional and working, but for the offline device it sounds like it will need to be power cycled, and definitely double check the network settings at the location that one is installed. Thanks!
Hello dtischler, now another Pi on the same project have issue, here the UID 9dc7c8977c0e67746a147e1623e5c788 I have granted support access for 3 weeks, can you please chack it why it goes down?
Thanks,
Alessandro
Hi @Niubbo75, this device is also experiencing some networking challenges, and is currently “Heartbeat Only”, which means that it cannot establish a link to the VPN.
You’ll need to double check that your network complies with Balena requirements as stated in Network Setup on balenaOS 2.x - Balena Documentation, but if I had to take a guess the network provider has some quality of service or traffic shaping that is killing your connectivity. You could try simply rebooting the device to see if that clears it up temporarily, but long term you have to make sure the network connectivity meets our guidelines.
Hi again dtischler, today also the other last two devices are “Online (VPN only)” and I can’t do anything, if I try to remote reboot I got “Request error: tunneling socket could not be established, statusCode=500”. Next week I hope to could go there and try to reboot manually all the devices, last time I did that, I was in need to rebuild all the devices because they do not reboot (srtill on black screen or on balena logo). Network is stable (fiber connection on new LAN cable, it’s a new build), PC’s aren’t affected from any issue at all.
Best regards,
Alessandro
Hi @dtischler next Thursday I’ll go there and I try to reboot all the devices hoping they will come back online again, last time I was in need to rebuild all the devices with new images because they do not boot at all.
I have noticed that before go down, all device have a CPU consuption exagerated (about 70 to 80%) also if they are not displaying anything complex (just a single picture), how could I debug this situation?
Thanks in advance, best regards,
Alessandro
It is probably worth turning on Persistent Logging on the devices so that the logs can be examined after the devices are rebooted / recovered to see what that last thing that happened was. You can enable it by going to the “Configuration” menu on the left in balenaCloud Dashboard, then clicking on “Activate” next to “Enable persistent logging. Only supported by supervisor versions >= v7.15.0.”, and finally sliding the toggle switch over to turn it on.
Hi @dtischler yesterday I had reboot all the devices, 3 on 4 come back online, the first that had have issue still trying to reboot but with no luck, today I’m going there to flash again the USB Stick.
I have enabled persistent logging, well this morning another device is went Heartbeat Only, where I have to look to get the infos you need to have?
If I go under “Diagnostic” the only thing I can read is in Target supervisor state (here I have paste it: Balena Screenly OSE Issue - Pastebin.com ) since I need to have stability and I could not admit to need to reboot/re flash devices every time, what I need to check? Today to be sure I’ll redo all the cable patches to be sure there aren’t any issue with that.
Thanks in advance, best regards,
Alessandro
Hi @Niubbo75 – based on the symptoms in this situation, the best thing I can recommend is to double-check with the network provider at the site, that these requirements are being met: Network Setup on balenaOS 2.x - Balena Documentation
Many times what I see is that network providers, ISP’s, and cellular connection providers do “Network Shaping” or other active management and prioritization of traffic and packets, and even if a network does meet those requirements, the providers are interfering as there is demand on their networks.
So, that is the first step I recommend based on the description of problems you are having. Hope that helps!
Hi @dtischler I think you are right but the issue seems not to be related to ISP but due to poor quality cable
When I have redo plugin Rj45 I have noticed that the small cable of the couple esaly broke, ATM I have another device that is Heatbeat and is the only one I haven’t redo the plug due to lack of cable.
The single device that didn’t come back online had a broken USB Stick (I have use USB stick as boot device & storage).
So I think I could close the issue and start to change network cable.
Thanks for all the support, best regards,
Alessandro
@dtischler I still have this issue also after redoing cables, I can now see only a single device online on dashb oard but if I connect to this device I can ping the other but I can’t ssh into them (I have try both ssh device-ip and ssh root@device-ip) what can I do to try at least a reboot of the other devices?
Thanks, best regards,
Alessandro
You can only SSH to them if they have the Development version of the OS installed. If you did flash the Development version, then you can SSH by using either the balena CLI utility or a standard ssh application following these instructions: SSH access - Balena Documentation
Hi @dtischler thanks for your reply!
I have flashed production OSes so I can’t SSH into them.
Best regards,
Alessandro
Sorry for the very late reply to this thread Alessandro, I definitely missed the notification on this one. Were you able to get these issues resolved?