Device not downloading latest release

hi, can you private message me with he dashboard link to this device please?

Just did. Let me know if you need something else.

Thanks!

has this device been renamed since it’s been provisioned?

@floion
It was only renamed once (from the default name that resin generates) to the current name.
That happened seconds after the device popped up on the dashboard for the first time.
No renames after that.

can we try to rename it to something without hyphens and then reboot the device?

Sure. You can keep the name, just drop the hyphens.
On the other hand, all of our devices have hyphens on their names and we never had a problem before. But you may go ahead if you think that will make a difference.

FYI: This specific device had a problem in the past (not occurring anymore), which is referenced here:

Hi @floion
Should we rename it, or will you do it?
Did you found anything else wrong? Device is still parked at the old commit.
Thanks!

Hi, we have found the problem and Cameron is working on it. You can see the progress here: https://github.com/resin-io/resin-supervisor/pull/665

Excellent, thanks.
Is there any chance we can apply a workaround in order for the device to update, even without the patch?
Or should we wait so the fix can be tested?

I think it would be best to wait for a new supervisor release and then you can update the supervisor on your board.

Hi,
We updated the supervisor on the aforementioned device and your device is now running the latest release of your application.
As a side note, could you remove the newline character from the device name?

Kind regards,
Thodoris

Hi @thgreasi
Actually, the device is down since yesterday. Did it came back up after the update?

Hi @ymaia ,
Yes the device did come up online and was sending logs properly. I monitored the device for about 15 minutes after the update and everything seemed fine.

The last connectivity event from the device was approximately 4 hours and 40 minutes after my previous response.
Any chance this is related with a network or power issue? Can someone check the device?

I experience a similar issue. Running development on balenaOS 2.31.5+rev1 and 9.11.3. But I do not understand why, since I have several devices running this configuration and they all seems to update seamlessly.
I tried rebooting the device and enable the lock override.
Moving the device to another application and back, but it seems that it keeps it’s release version no matter what application it is running on.
Usually I would just have flashed a new SD card but the device has been shipped to another city.

Extra: It seems that the ip is not the eth0 but something else.

I have access to the host OS, is there anything I can do to force the device to pull the newest release?

Best
th3is

Normally the device should just pick up the changes without any further actions. Is it possible that the device is pinned to a specific release? You can see how to pin a device to a release here https://github.com/balena-io-projects/staged-releases, just check the scripts in the repo. You can just set a device to a specific release and see if that solves the problem. Let us know if that works.

@th3is, in case release pinning (as suggested by sradevski) is not the explanation:

  • The web dashboard now has device diagnostics page and you may find interesting to check the Supervisor State tab: compare the “current state” with the “target state”, and also compare working devices with the problematic device.

  • On a host OS terminal, you can check some logs for any error messages, in order of likelihood:

    • balena supervisor: journalctl -au resin-supervisor
    • balenaEngine: journalctl -au balena
    • overall system: journalctl -ab

As the log messages can be confusing, I suggest comparing the working devices with the problematic devices to spot the difference. There several warning/info messages that may sound like a problem but are actually advisory. In case you do find specific errors, consider searching the forums and whether it makes sense to start a new thread.

Hi and thank you guys. I should never have touched the pinnig, so I jumped directly to the the diagnostics. When trying to access the supervisor state I continously get errors:

Running "journalctl -au resin-supervisor" I can see an error continously popping up:
    May 02 06:41:01 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:01.467Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at IncomingMessage.emit (events.js:185:7)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at endReadableNT (_stream_readable.js:974:12)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at _combinedTickCallback (internal/process/next_tick.js:80:11)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at process._tickCallback (internal/process/next_tick.js:104:9)
    May 02 06:41:02 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:02.091Z] Creating supervisor0 network
    May 02 06:41:16 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:16.265Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at IncomingMessage.emit (events.js:185:7)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at endReadableNT (_stream_readable.js:974:12)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at _combinedTickCallback (internal/process/next_tick.js:80:11)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at process._tickCallback (internal/process/next_tick.js:104:9)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:16.795Z] Creating supervisor0 network
    May 02 06:41:30 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:30.870Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)

For journalctl -au balena, it seems to be similar:
May 02 06:41:45 5b667b7 balenad[787]: [2019-05-02T06:41:45.654Z] Unhandled rejection Error: (HTTP code>
May 02 06:41:45 5b667b7 balenad[787]: at /usr/src/app/dist/app.js:576:112035
May 02 06:41:45 5b667b7 balenad[787]: at /usr/src/app/dist/app.js:576:111998
May 02 06:41:45 5b667b7 balenad[787]: at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
May 02 06:41:45 5b667b7 balenad[787]: at IncomingMessage. (/usr/src/app/dist/app.js:576>

It seems like I must have messed up the network configuration, but I do not remember touching it at all for this particular device. Or do you have a better interpretation.

The raspberry is connected to a modem which is running dhcp and should receive an adrress in the range of 192.168.1.100 - 192.168.1.150. And it seems that this corresponds directly with the eth0 but not the br-245c9ed5116a which’s address is shown in the dashboard:
root@5b667b7:~# ifconfig
balena0 Link encap:Ethernet HWaddr 02:42:71:48:6E:7C
inet addr:10.114.101.1 Bcast:10.114.101.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

br-245c9ed5116a Link encap:Ethernet  HWaddr 02:42:F7:B9:C3:3F  
          inet addr:172.19.0.1  Bcast:172.19.255.255  Mask:255.255.0.0
          inet6 addr: fe80::42:f7ff:feb9:c33f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21779195 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17496919 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2374493744 (2.2 GiB)  TX bytes:2030715348 (1.8 GiB)

eth0      Link encap:Ethernet  HWaddr B8:27:EB:75:F5:B1  
          inet addr:192.168.1.133  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::7873:13b5:70fe:a02/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18767055 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21895219 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1909061511 (1.7 GiB)  TX bytes:2938177981 (2.7 GiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:742 errors:0 dropped:0 overruns:0 frame:0
          TX packets:742 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:68192 (66.5 KiB)  TX bytes:68192 (66.5 KiB)

resin-dns Link encap:Ethernet  HWaddr 9E:86:D7:09:C8:36  
          inet addr:10.114.102.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::9c86:d7ff:fe09:c836/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:197 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:16603 (16.2 KiB)

resin-vpn Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.240.54.165  P-t-P:52.4.252.97  Mask:255.255.255.255
          inet6 addr: fe80::43f8:6d3f:89:e2a2/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:1044 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1044 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:85694 (83.6 KiB)  TX bytes:164932 (161.0 KiB)

veth2219def Link encap:Ethernet  HWaddr CA:67:C5:52:96:24  
          inet6 addr: fe80::c867:c5ff:fe52:9624/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:252 errors:0 dropped:0 overruns:0 frame:0
          TX packets:256 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:36844 (35.9 KiB)  TX bytes:25789 (25.1 KiB)

veth47a0a73 Link encap:Ethernet  HWaddr 86:B4:5D:64:28:5C  
          inet6 addr: fe80::84b4:5dff:fe64:285c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1419729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1109374 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:179355758 (171.0 MiB)  TX bytes:105898647 (100.9 MiB)

veth8ff0cd4 Link encap:Ethernet  HWaddr 26:04:A1:53:A0:D6  
          inet6 addr: fe80::2404:a1ff:fe53:a0d6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1568357 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1427074 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:213074155 (203.2 MiB)  TX bytes:500576391 (477.3 MiB)

wlan0     Link encap:Ethernet  HWaddr 06:C1:D8:74:EC:A6  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

I will have a go at those scripts while awaiting your answer. Just wanted to pass on some information first.

Again, thank you.
Best
Th3is

The device has now updated, the network configurations looks correct and the diagnostics page works. Did you guys do anything on you side, I did not, I just checked today and it was up and running on the correct release :stuck_out_tongue:

@th3is, for the better or the worse, :slight_smile: I don’t think our team took any action to fix that device. The output of ifconfig on a RPi I have here lists an interface named supervisor0, whereas the output you quote in a previous message did not list that interface name. The error message you were getting, “no such network - network supervisor0 not found,” makes sense in this context. There are some watchdogs in balenaOS – I am not sure if it was the case, but it’s possible that if the supervisor got in an unhealthy state for long enough (half an hour?), the watchdog could restart it, which would re-created that network interface. Let us know if the problem reoccurs, in which case it would be interesting to investigate the issue further. Thanks for reporting it.

Im having the same issue @th3is had.

Current configuration:
Host OS Version: balenaOS 2.31.5+rev1
Supervisor Version: 9.11.3

The device was updating at one point but is no longer updating. It is stuck on 2 releases. I tried rebooting to trigger an update but not able to ssh at times to debug further.