Device not downloading latest release

Product support
40

Normally the device should just pick up the changes without any further actions. Is it possible that the device is pinned to a specific release? You can see how to pin a device to a release here https://github.com/balena-io-projects/staged-releases, just check the scripts in the repo. You can just set a device to a specific release and see if that solves the problem. Let us know if that works.

42

@th3is, in case release pinning (as suggested by sradevski) is not the explanation:

  • The web dashboard now has device diagnostics page and you may find interesting to check the Supervisor State tab: compare the “current state” with the “target state”, and also compare working devices with the problematic device.

  • On a host OS terminal, you can check some logs for any error messages, in order of likelihood:

    • balena supervisor: journalctl -au resin-supervisor
    • balenaEngine: journalctl -au balena
    • overall system: journalctl -ab

As the log messages can be confusing, I suggest comparing the working devices with the problematic devices to spot the difference. There several warning/info messages that may sound like a problem but are actually advisory. In case you do find specific errors, consider searching the forums and whether it makes sense to start a new thread.

43

Hi and thank you guys. I should never have touched the pinnig, so I jumped directly to the the diagnostics. When trying to access the supervisor state I continously get errors:

image
image.png1928×1048 187 KB 

Running "journalctl -au resin-supervisor" I can see an error continously popping up:
    May 02 06:41:01 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:01.467Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at IncomingMessage.emit (events.js:185:7)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at endReadableNT (_stream_readable.js:974:12)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at _combinedTickCallback (internal/process/next_tick.js:80:11)
    May 02 06:41:01 5b667b7 resin-supervisor[16365]:     at process._tickCallback (internal/process/next_tick.js:104:9)
    May 02 06:41:02 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:02.091Z] Creating supervisor0 network
    May 02 06:41:16 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:16.265Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at IncomingMessage.emit (events.js:185:7)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at endReadableNT (_stream_readable.js:974:12)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at _combinedTickCallback (internal/process/next_tick.js:80:11)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]:     at process._tickCallback (internal/process/next_tick.js:104:9)
    May 02 06:41:16 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:16.795Z] Creating supervisor0 network
    May 02 06:41:30 5b667b7 resin-supervisor[16365]: [2019-05-02T06:41:30.870Z] Unhandled rejection Error: (HTTP code 404) no such network - network supervisor0 not found
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:112035
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at /usr/src/app/dist/app.js:576:111998
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:576:111508)
    May 02 06:41:30 5b667b7 resin-supervisor[16365]:     at emitNone (events.js:91:20)

For journalctl -au balena, it seems to be similar:
May 02 06:41:45 5b667b7 balenad[787]: [2019-05-02T06:41:45.654Z] Unhandled rejection Error: (HTTP code>
May 02 06:41:45 5b667b7 balenad[787]: at /usr/src/app/dist/app.js:576:112035
May 02 06:41:45 5b667b7 balenad[787]: at /usr/src/app/dist/app.js:576:111998
May 02 06:41:45 5b667b7 balenad[787]: at m.buildPayload (/usr/src/app/dist/app.js:576:112008)
May 02 06:41:45 5b667b7 balenad[787]: at IncomingMessage. (/usr/src/app/dist/app.js:576>

It seems like I must have messed up the network configuration, but I do not remember touching it at all for this particular device. Or do you have a better interpretation.

The raspberry is connected to a modem which is running dhcp and should receive an adrress in the range of 192.168.1.100 - 192.168.1.150. And it seems that this corresponds directly with the eth0 but not the br-245c9ed5116a which’s address is shown in the dashboard:
root@5b667b7:~# ifconfig
balena0 Link encap:Ethernet HWaddr 02:42:71:48:6E:7C
inet addr:10.114.101.1 Bcast:10.114.101.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

br-245c9ed5116a Link encap:Ethernet  HWaddr 02:42:F7:B9:C3:3F  
          inet addr:172.19.0.1  Bcast:172.19.255.255  Mask:255.255.0.0
          inet6 addr: fe80::42:f7ff:feb9:c33f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21779195 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17496919 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2374493744 (2.2 GiB)  TX bytes:2030715348 (1.8 GiB)

eth0      Link encap:Ethernet  HWaddr B8:27:EB:75:F5:B1  
          inet addr:192.168.1.133  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::7873:13b5:70fe:a02/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18767055 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21895219 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1909061511 (1.7 GiB)  TX bytes:2938177981 (2.7 GiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:742 errors:0 dropped:0 overruns:0 frame:0
          TX packets:742 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:68192 (66.5 KiB)  TX bytes:68192 (66.5 KiB)

resin-dns Link encap:Ethernet  HWaddr 9E:86:D7:09:C8:36  
          inet addr:10.114.102.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::9c86:d7ff:fe09:c836/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:197 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:16603 (16.2 KiB)

resin-vpn Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.240.54.165  P-t-P:52.4.252.97  Mask:255.255.255.255
          inet6 addr: fe80::43f8:6d3f:89:e2a2/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:1044 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1044 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:85694 (83.6 KiB)  TX bytes:164932 (161.0 KiB)

veth2219def Link encap:Ethernet  HWaddr CA:67:C5:52:96:24  
          inet6 addr: fe80::c867:c5ff:fe52:9624/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:252 errors:0 dropped:0 overruns:0 frame:0
          TX packets:256 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:36844 (35.9 KiB)  TX bytes:25789 (25.1 KiB)

veth47a0a73 Link encap:Ethernet  HWaddr 86:B4:5D:64:28:5C  
          inet6 addr: fe80::84b4:5dff:fe64:285c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1419729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1109374 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:179355758 (171.0 MiB)  TX bytes:105898647 (100.9 MiB)

veth8ff0cd4 Link encap:Ethernet  HWaddr 26:04:A1:53:A0:D6  
          inet6 addr: fe80::2404:a1ff:fe53:a0d6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1568357 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1427074 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:213074155 (203.2 MiB)  TX bytes:500576391 (477.3 MiB)

wlan0     Link encap:Ethernet  HWaddr 06:C1:D8:74:EC:A6  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

I will have a go at those scripts while awaiting your answer. Just wanted to pass on some information first.

Again, thank you.
Best
Th3is

44

The device has now updated, the network configurations looks correct and the diagnostics page works. Did you guys do anything on you side, I did not, I just checked today and it was up and running on the correct release :stuck_out_tongue:

46

@th3is, for the better or the worse, :slight_smile: I don’t think our team took any action to fix that device. The output of ifconfig on a RPi I have here lists an interface named supervisor0, whereas the output you quote in a previous message did not list that interface name. The error message you were getting, “no such network - network supervisor0 not found,” makes sense in this context. There are some watchdogs in balenaOS – I am not sure if it was the case, but it’s possible that if the supervisor got in an unhealthy state for long enough (half an hour?), the watchdog could restart it, which would re-created that network interface. Let us know if the problem reoccurs, in which case it would be interesting to investigate the issue further. Thanks for reporting it.

47

Im having the same issue @th3is had.

Current configuration:
Host OS Version: balenaOS 2.31.5+rev1
Supervisor Version: 9.11.3

The device was updating at one point but is no longer updating. It is stuck on 2 releases. I tried rebooting to trigger an update but not able to ssh at times to debug further.

48

@rudnyg can you share with us the dashboard url of that device and also grant us support access so that we can take a closer look?

50

Hi @rudnyg did you manage to debug this any further?

52

The device was in the field and was also have connection issues. Decided to get the device shipped back to do further debugging.

53

ah, that makes sense. Just let us know if we can help you with the debugging in anyway.