Unable to update HostOS on old devices

Product support
,
1

Hi, I’m trying to update the Host OS version on some of our old Raspberry Pi3 devices, but the update never seems to start.

When I try updating through the web console, I simply get a green notification saying the operation completed successfully, but the update never starts. If I try updating through the CLI (e.g. balena device os-update 8af61471a79d017362d03c8dcc78ec96), I get an error saying resinhup failed: exit code 60

The old devices are running ‘Resin OS 2.12.7+rev1’, and I’ve tried upgrading to version v2.115.1 and v2.113.18. Do you have an idea what the problem might be? I had no issues updating a batch of similar devices about two weeks ago.

1 Like
2

Hi, according to some old notes we have internally, we have seen a similar issue on a different old OS and a possible workaround is to set the network_mode: host on the docker-template temporarily until you manage to hup. Could you please try to do so for an affected device and let us know if that resolves the issue?

1 Like
4

Thanks for the suggestion. I forgot to mention this, but the devices in question are part of an Essentials fleet, so there is no docker-compose file. Is it still possible to change the network mode somehow?

I saw your docs also mentioned that there might be update logs in ‘/mnt/data/balenahup/’ or ‘/mnt/data/resinhup/’. I found log files in ‘/mnt/data/balenahup/’, and they only contain these 4 lines:

================upgrade-2.x.sh HEADER START====================
Wed May 24 08:21:22 UTC 2023
[000000000][LOG]Raw target version: 2.113.18
[000000001][LOG]Loading info from config.json
1 Like
8

Hello @Rune first of all welcome to the balena community.

Could you please try to update from 2.12.7+rev1 to 2.26.0+rev1 → 2.32.0+rev1 → 2.47.0+rev1 → latest?

Let us know if this works!

1 Like
10

Hi mpous,

Thanks! I tried updating to 2.26.0+rev1 on a few devices, but the same issue happens. The log message is almost the same:

================upgrade-2.x.sh HEADER START====================
Wed May 24 10:11:40 UTC 2023
[000000000][LOG]Raw target version: 2.26.0+rev1.prod
[000000000][LOG]Normalized target version: 2.26.0+rev1
[000000000][LOG]Loading info from config.json
1 Like
12

Hi again @mpous

I did a bit more digging, and I got the HostOS update to work! If I changed the apiEndpoint in /mnt/boot/config from https://api.resin.io to https://api.balena-cloud.com, then the update works without issues.

It looks like the update was previously failing silently at this point balenahup/upgrade-2.x.sh at master · balena-os/balenahup · GitHub. If I ran the curl command manually on the device

CURL_CA_BUNDLE=/tmp/tmp.xxxxxx curl -H "Authorization: Bearer xxxx" --retry 5 "https://api.resin.io/v6/device?\$select=is_of__device_type&\$expand=is_of__device_type(\$select=slug)&\$filter=uuid%20eq%20%27c4dabef71ed22dedace00410e9f2700b%27"

then I got this error:

curl: (60) server certificate verification failed. CAfile: /tmp/tmp.YJo4nEsflR CRLfile: none
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.

This is only an issue on our old devices running ‘Resin OS 2.12.7+rev1’. We have other devices running versions 2.90+ where apiEndpoint is also set to https://api.resin.io, but they can update HostOS without problems.

Do you know if there is a way to get around this issue without modifying the config.json?

1 Like
13

Glad to hear that it worked on your old OSs @Rune

Did you try GitHub - balena-os/configizer: Safe(r) balenaOS config.json updates remotely to update the apiEndpoint of the other devices running balenaOS 2.90+?

let us know if that works!

15

Thanks for the tool link! That’ll definitely be helpful when we have to update the rest of our devices.

I haven’t tried updating the apiEndpoint on our newer devices, as we don’t see any issues on them yet.