IOT2000 doesn't go online

Hi!
Since yesterday we are experiencing some problems with the connection of our Siemens IOT2000s.

Yesterday one of the two device (it is in our office) seemed to be online but it was not possible to interact with it from the resin.io dashboard. I have manually rebooted it but did not work; then I have newly flashed the sd card with a fresh image downloaded from the resin.io website, but once booted the device does not appear in the dashboard.

A second IOT2000(68418a1b004a1eed483036f2c8df46d0) that yesterday evening was correctly working, this morning is shown as offline.
Thank you for your support!

Cheers,
Alberto

Hi, that’s quite strange, what version of resinOS are you running on it?

Did anything change on your network yesterday? It looks like some of the communication on the network is blocked (likely to the resin VPN). Do you have any other resin devices on your network (or try with a Qemu virtual device, for example), that would check if it works?

Also, on the local network, the second device if it is turned on, you can try to ping over the default hostname, if you haven’t changed it yet (that would be 68418a1.local in this device’s case, unless it was changed by your application or settings). I would expect the device to be reachable on the local network. Since it’s a dev device, you should also be able to connect to it over the local network if it is’ on, with something like ssh -P 22222 root@68418a1.local, and then can check the local logs (just looking into journalctl -f -a, or more targeted way looking at the OpenVPN logs journalctl -a -u openvpn-resin).

This might help with the initial investigation.

Hi!

The network is not under our control and we don’t know if something changed yesterday, but we have also a raspberry Pi 3 with resin under the same network and is working fine.
The IOT2000s are running the last available version of resinOS (2.7.6+rev1).
Rebooting the second IOT2000 it returned alive and running journalctl -a -u openvpn-resin this is the output:

Apr 05 12:49:22 68418a1 systemd[1]: Starting OpenVPN Application On Resin…
Apr 05 12:49:23 68418a1 systemd[1]: openvpn-resin.service: PID file /var/run/openvpn/resin.pid not readable (yet?) after start: No such file or directory
Apr 05 12:49:23 68418a1 systemd[1]: Started OpenVPN Application On Resin.
Apr 05 12:49:23 68418a1 openvpn[650]: [[0;1;39mNOTE: the current --script-security setting may allow this configuration to call user-defined scripts[[0m
Apr 05 12:49:23 68418a1 openvpn[650]: [[0;1;39mSocket Buffers: R=[87380->87380] S=[16384->16384][[0m
Apr 05 12:49:23 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:23 68418a1 openvpn[650]: [[0;1;39mNOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay[[0m
Apr 05 12:49:23 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:28 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:33 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:39 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:44 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:49 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:54 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:49:59 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:04 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:09 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:14 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:19 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:24 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:29 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:34 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:39 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:44 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:49 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:54 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:50:59 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:04 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:09 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:14 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:19 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:24 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:29 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:34 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:39 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:44 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m
Apr 05 12:51:50 68418a1 openvpn[650]: [[0;1;31mRESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname[[0m

The first device (the one with the newly flashed OS) is still offline; I have tried to set a static IP but I am not able to ping it.

The device that is now online must have resolved vpn.resin.io in the end. It does look like the network communication issue as mentioned earlier. Later on in the logs (if you check the latest ones with journalctl -f -a -u openvpn-resin) it must be a connection success in the end.

Sorry, assumed that since it’s “our office”, you have some control or visibility into the networking at least.

What version of resinOS is the Raspberry Pi 3 running?

Yes, the output of journalctl -f -a -u openvpn-resin is:

root@68418a1:~# journalctl -f -a -u openvpn-resin
– Logs begin at Thu 2018-04-05 12:48:38 UTC. –
Apr 05 12:52:30 68418a1 openvpn[650]: TUN/TAP TX queue length set to 100
Apr 05 12:52:30 68418a1 openvpn[650]: do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Apr 05 12:52:30 68418a1 openvpn[650]: /sbin/ip link set dev resin-vpn up mtu 1500
Apr 05 12:52:30 68418a1 openvpn[650]: /sbin/ip addr add dev resin-vpn local 10.240.38.71 peer 52.4.252.97
Apr 05 12:52:30 68418a1 openvpn[650]: /etc/openvpn/upscript.sh resin-vpn 1500 1547 10.240.38.71 52.4.252.97 init
Apr 05 12:52:30 68418a1 openvpn[650]: /sbin/ip route add 52.4.252.97/32 via 52.4.252.97
Apr 05 12:52:31 68418a1 openvpn[650]: ERROR: Linux route add command failed: external program exited with error status: 2
Apr 05 12:52:31 68418a1 openvpn[650]: GID set to openvpn
Apr 05 12:52:31 68418a1 openvpn[650]: UID set to openvpn
Apr 05 12:52:31 68418a1 openvpn[650]: Initialization Sequence Completed

The RaspberryPi 3 is running resinOS 2.12.0+rev1 and has ID 0b1f2556bdf5d886f062d917f58674ec

I don’t know how but now both the devices are online, the first IOT has now ID: c2c104cc508dc1a65f420856621a321a

Yes, since you burned a new card, it reprovisioned on start, as it should be.

If it happens next time, there are a couple of options:

  • since it’s a dev device, you can try to ssh into the device using the <shortuuid>.local address, or resin local ssh --host using resin-cli, and check the logs then, or
  • if there’s one resin device on the network that is still connected properly, then our support agents can “hop” over that to try to connect to the device which doesn’t connect to our VPN on the local network…

Let us know if this pops up again…