netdata cloud cannot reach Raspberry Pi4 running netdata-agent but Pi3 on outdated balenaOS detected just fine

Product support balenaOS
1

Hey. I read this issue but it seems we have a different issue.

I have Pi3 instance running for some time with version 2021.10.2 marked as outdated.
I have newer Pi4 instance that is updated to a 2023.1.0 (latest at the point).

I am running my app in docker compose and as a part of it I have Netdata container(not that much different from this config).

So the issue is that on Pi3 netdata-agent runs fine and I can see it from netdata cloud dashboard but on the Pi4 netdata container constantly restarted and I see some logs about Connection refused from different services in docker logs.

2

Hello, can you post some of the error/connection refused messages you are seeing? I have this block:

running on the same Pi4 2023.1.0 OS but not seeing any error messages. Which image are you running or are you building it yourself in a Dockerfile?

1 Like
3

we have Dockerfile with simple FROM netdata/netdata:v1.37.1 and then running run.sh from netdata/packaging/docker/run.sh (placed in /usr/sbin/run.sh obviously)

and docker-compose uses this setup as build field. In compose file all is basically the same as I linked in the OP.

If I understand correctly it is just correctly configured image that is spinned along with others components of compose file? But what if I have additional tuning in netdata.conf and go.d.conf?
Basically my netdata component of compose file look hell of a lot like the one from this netdatablock’s

4

I just found the issue.

Pi3 device is not 64bit (what is specifically mentioned in this netdatablock you linked).

So what happened was(my best guess, since that is the only possible way it worked on 32 but pi), on the older balenaOS version it checked the right platform for the Pi3 but in the latest, it fails to do that for some reason.
I mean it successfully chose linux/arm/v7 in the 2021.10.2 but chose linux/arm64 in the 2023.1.0
2021.10.2 → linux/arm/v7
2023.1.0 → linux/arm64

So the older one detected arm

# Pi3 2021.10.2

balena image inspect <image-id> --format="{{.Architecture}}"
arm

Pi4 (64) chooses arm64.

# Pi4 2023.1.0
balena image inspect <image-id> --format='{{.Architecture}}'
arm64

After we basically used netdatablocks startup.sh, Pi4 started working with netdata, but on Pi3 netdata container simply crashed because it pulled arm64 even when it is armv7l.

1 Like
6

Hello @alekspickle i only tested this on a Raspberry Pi 4 (arm64).

Could you please confirm if now it works for you?

7

Yeah, Pi4 (arm64) with the permissions setting block that sturtup.sh has - works perfectly, but the issue is that it somehow enforces arm64 arch on all images in fleet, therefore Pi3(armv7l) in the same fleet pull arm64 ones which they obviously cannot run and they crash.

I guess my previous comment about balenaOS version could be irrelevant since the arch of a built images is.

So does it makes Pi4(64) and Pi3(32) incompatible in the same fleet? And can enforce this any other way?