# Devices appear offline

**URL:** <https://forums.balena.io/t/devices-appear-offline/373541>\
**Category:** openBalena\
**Created:** [February 5, 2025, 3:55pm UTC](https://forums.balena.io/t/devices-appear-offline/373541 "2025-02-05T15:55:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![benjboy](https://avatars.discourse-cdn.com/v4/letter/b/e9a140/32.png) [@benjboy](https://forums.balena.io/u/benjboy)\
**Post date:** [February 5, 2025, 3:55pm UTC](https://forums.balena.io/t/devices-appear-offline/373541/1 "2025-02-05T15:55:11Z")

</div>

The devices appear offline but the logs can still be accesed and the are current

found that vpn service is throwing errors:

```auto
Feb 05 15:48:29 3fe242534b91 haproxy[1224]: Routing 77.38.64.241:57874@tcp-443 to vpn-workers/vpn1:55798 [C:1/1 Q:0/0 T:1/0]
Feb 05 15:48:29 3fe242534b91 vpn[1362]: notice: [vpn-20.1] TCP connection established with [AF_INET]127.0.0.1:55798
Feb 05 15:48:29 3fe242534b91 vpn[1362]: notice: [vpn-20.1] Socket flags: TCP_NODELAY=1 succeeded
Feb 05 15:48:29 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55798 TLS: Initial packet from [AF_INET]127.0.0.1:55798, sid=954b442f 65b7bfd2
Feb 05 15:48:29 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55798 Connection reset, restarting [0]
Feb 05 15:48:29 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55798 SIGUSR1[soft,connection-reset] received, client-instance restarting
Feb 05 15:48:31 3fe242534b91 haproxy[1224]: Routing 91.224.238.6:58025@tcp-443 to vpn-workers/vpn1:55814 [C:1/1 Q:0/0 T:1/0]
Feb 05 15:48:31 3fe242534b91 vpn[1362]: notice: [vpn-20.1] TCP connection established with [AF_INET]127.0.0.1:55814
Feb 05 15:48:31 3fe242534b91 vpn[1362]: notice: [vpn-20.1] Socket flags: TCP_NODELAY=1 succeeded
Feb 05 15:48:31 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55814 TLS: Initial packet from [AF_INET]127.0.0.1:55814, sid=a65413c5 ec707315
Feb 05 15:48:31 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55814 Connection reset, restarting [0]
Feb 05 15:48:31 3fe242534b91 vpn[1362]: notice: [vpn-20.1] 127.0.0.1:55814 SIGUSR1[soft,connection-reset] received, client-instance restarting
Feb 05 15:48:38 3fe242534b91 haproxy[1224]: Routing 91.224.238.6:58026@tcp-443 to vpn-workers/vpn1:42920 [C:1/1 Q:0/0 T:1/0]

```

did anyone come across similar issue?

---

<div class="post-metadata">

**Author:** ![Stewie](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/stewie/32/10249_2.png) [@Stewie](https://forums.balena.io/u/Stewie)\
**Post date:** [February 18, 2025, 8:57pm UTC](https://forums.balena.io/t/devices-appear-offline/373541/3 "2025-02-18T20:57:30Z")

</div>

I have a similar issue. The `is_online` flag is not updating. New devices can join the fleet but the state is always **offline** (or **online** if the device was online prior to the upgrade mentioned below).

The device logs feed is working fine, and I can open tunnels, but can’t communicate with the devices.

**Backstory:**  
I upgraded from `4.1.251` to `4.1.349` and created a new ssl certificate using certbot. When I do `make up`, `docker ps` often show that the `vpn` service as **unhealthy**.

The only time I can manage to get the `vpn` service to become **healthy** is when the `api` fail to run `migration "0074` and I run the following:

```auto
docker exec -it open-balena-api-1 /bin/bash
rm src/migrations/0074-normalize-release-contract.async.ts 
exit
docker restart open-balena-api-1
docker restart open-balena-vpn-1

```

(cred to @pschmo in [openBalena 2024 - #7 by pschmo](https://forums.balena.io/t/openbalena-2024/370459/7))

And that’s when I encounter the issue of device state not being updated, e.g. `is_online` never updates. When I upgraded to `4.1.251`, I only had to remove the `0074` migration file and restart the services to get up and running. I wonder if this is even related to my issue? 🤔

Maybe I need to dig into what’s happening in the `vpn` container?

@benjboy did you manage to figure out what caused the issue? 😬

​​

​​

---

<div class="post-metadata">

**Author:** ![benjboy](https://avatars.discourse-cdn.com/v4/letter/b/e9a140/32.png) [@benjboy](https://forums.balena.io/u/benjboy)\
**Post date:** [February 19, 2025, 12:15pm UTC](https://forums.balena.io/t/devices-appear-offline/373541/4 "2025-02-19T12:15:06Z")

</div>

my issue was that the certificate of the vpn service was expired updated it using @wolf_karl comment [https://forums.balena.io/t/vpn-certs-seems-to-be-expired/262191/10?u=benjboy](https://forums.balena.io/t/vpn-certs-seems-to-be-expired/262191/10) .  
But i’m using open balena version 3.8.2 so we are probably having different issues.

---

<div class="post-metadata">

**Author:** ![Stewie](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/stewie/32/10249_2.png) [@Stewie](https://forums.balena.io/u/Stewie)\
**Post date:** [February 19, 2025, 3:13pm UTC](https://forums.balena.io/t/devices-appear-offline/373541/5 "2025-02-19T15:13:13Z")

</div>

Thanks for the reply!  
Yeah, I think so too! Created my own thread.

---

<div class="post-metadata">

**Author:** ![Abdullah](https://avatars.discourse-cdn.com/v4/letter/a/aeb1de/32.png) [@Abdullah](https://forums.balena.io/u/Abdullah)\
**Post date:** [February 26, 2025, 2:57pm UTC](https://forums.balena.io/t/devices-appear-offline/373541/6 "2025-02-26T14:57:06Z")

</div>

> [@benjboy](#):
>
> The devices appear offline but the logs can still be accesed and the are current
> 
> found that vpn service is throwing errors:

This scenario, where devices appear offline but logs are current and accessible, coupled with VPN service errors, strongly indicates a networking or VPN-specific connectivity issue, rather than a complete device failure. Here’s a breakdown of potential causes and troubleshooting steps:

**Understanding the Symptoms:**

- **Devices “Offline”:**
  - This suggests a lack of real-time communication. Devices might not be responding to ping requests or other network probes.
  - This could mean a break in the live network connection.

- **Current, Accessible Logs:**
  - This indicates that devices are still able to record and transmit data, even if live communication is impaired.
  - This suggests the underlying systems are functioning, but a specific communication pathway is blocked.

- **VPN Service Errors:**
  - This is a crucial clue. VPNs are complex, and errors can disrupt network traffic in various ways.

**Possible Causes:**

- **VPN Tunnel Issues:**
  - **Tunnel Failure:** The VPN tunnel itself might be unstable or failing to establish correctly. This can be due to configuration errors, network congestion, or firewall interference.
  - **MTU Issues:** Maximum Transmission Unit (MTU) mismatches can cause packet fragmentation, leading to connectivity problems. VPN encryption adds to packet size, exacerbating this issue.
  - **Port Blocking:** Firewalls or ISPs might be blocking the ports required for the VPN to function (e.g., UDP 500 and 4500 for IKE).
  - **Certificate Problems:** Expired or invalid VPN certificates can prevent successful authentication.

- **Routing Problems:**
  - Incorrect routing tables can prevent devices from reaching each other, even if they are technically online.
  - Double NAT (Network Address Translation) can also cause routing conflicts.

- **Firewall Interference:**
  - Firewall rules, either on the devices themselves or on network firewalls, might be blocking VPN traffic.

- **DNS Issues:**
  - Although logs are still working, DNS problems can prevent devices from resolving hostnames, making them appear offline.

- **Network Congestion:**
  - Heavy network traffic can overwhelm the VPN or network infrastructure, leading to dropped connections.

**Troubleshooting Steps:**

1. **VPN Log Analysis:**
  - Carefully examine the VPN error logs. Look for specific error codes or messages that indicate the nature of the problem.

2. **Network Connectivity Tests:**
  - Test connectivity from different points in the network, including within and outside the VPN.
  - Use tools like `ping`, `traceroute`, and `telnet` to identify where the connection is breaking down.

3. **VPN Configuration Review:**
  - Verify the VPN configuration settings, including tunnel type, encryption protocols, and authentication methods.
  - Ensure that all devices are using the same configuration.

4. **Firewall Checks:**
  - Review firewall rules on all devices and network firewalls to ensure that VPN traffic is allowed.
  - Temporarily disable firewalls to isolate potential conflicts.

5. **MTU Adjustment:**
  - If MTU issues are suspected, try reducing the MTU size on the VPN interface.

6. **Port Verification:**
  - Ensure that the necessary VPN ports are open on all firewalls and routers.

7. **Certificate Validation:**
  - Verify that all VPN certificates are valid and up to date.

8. **Routing Table Inspection:**
  - Inspect routing tables on all devices and routers to identify any routing errors.

9. **DNS Troubleshooting:**
  - Test DNS resolution from the devices.

10. **ISP Involvement:**
  - If problems persist, contact your ISP to rule out network issues on their end.

**Key Considerations:**

- Security: Be cautious when disabling firewalls or making changes to VPN configurations.
- Documentation: Keep detailed records of all troubleshooting steps and findings.

By systematically working through these steps, you should be able to identify and resolve the connectivity issues.  
[For More Details](https://www.fashionholic.pk/product/transparent-nude-bridal-shoes/)
