Hi,
I’m using the connector Balenablock. Recently I changed the image location to bh.cr/balenalabs/connector-aarch64 from balenablocks/connector:raspberrypi4-64. Unfortunately there was a problem with this image resulting in an ModuleNotFound Errors (I made a post about this on the forum and it seems this is a known problem indicated in the GitHub repo).
No problem I thought, I just changed the image location back to balenablocks/connector:raspberrypi4-64 and did a push up to my fleet which all seemed to work.
However now I’m getting the following rather strange error which I’m not convinced is entirely related just to the connector block. I’ve never seen this before and don’t see that I’ve changed anything my end.
I tried creating a new fleet, creating a new device, trying a previous OS version, all to no avail. I even powered up a device in this fleet that I know was working fine and was pinned to a working release but even this is now showing this error!
Does anybody have any ideas what might be causing this or anything I can try?
I’m wondering if this error means that the ‘connector’ is not able to get data from my sensor which makes its data available on port 7575 (as per connectors requirements) and yet I haven’t made any changes to the sensor code… very strange.
Phew looks like a fixed this. In case anybody comes across in a search in the future - I went back to a clean slate, created a new project in PyCharm using my main branch code, setup a new fleet and went back to Python 3.7 images. No errors now, everything is working.
So at the moment I’m not quite sure what caused this but I’ll keeping a close eye on things as I move forward now to see if I can isolate the cause. I’ll update here once I can be certain.
Hi @markysparks, glad you got your service working again. When you changed the image location on your app, did you change any other configurations? The balena.exceptions.RequestError looks like an authentication failure to the Balena API, so I’m wondering if maybe you could have mistakenly removed the io.balena.features.balena-api label in your docker-compose. Let us know if this sounds familiar
Hmm, it’s mysterious indeed. Since you point out the issue persisted even for new devices on the fleet, this really points out to some issue with the release, but I’m not really sure what it could be then. Could you maybe download the docker-compose from the failing release and paste it to the ticket? You can remove any sensitive information if you like, as long as you keep the format.
Sure, here we are - though this all started when I tried to point the connector image to bh.cr/balenalabs/connector-aarch64 from balenablocks/connector:raspberrypi4-64 though that maybe co-incidence of course. I changed the image location back to balenablocks/connector:raspberrypi4-64 as you can see in this docker-compose file. I don’t see why I would have changed anything else.
Also I’m not sure now if this docker-compose is in exactly the same state when I had the problems now as I needed to get it working again quickly and I have the deleted the old GitHub repo, didn’t think I would be needing it anymore! - sorry.
I do remember checking through the GitHub commits at the time and not seeing any changes to the docker-compose apart from that image location, that’s what I found so strange about this, I really tried to look for some unintended change but couldn’t find anything. It was at the point I went back to a ‘clean slate’ just to get it working again.
I know I also tried going back to previous working releases as well but the problem persisted, which was again very strange.
version: '2.1'
volumes:
settings:
dashboard-data:
sensor-data:
services:
scheduler:
restart: always
build: ./scheduler
privileged: true
wifi-connect:
image: bh.cr/balenalabs/wifi-connect-aarch64
restart: always
network_mode: host
privileged: true
labels:
io.balena.features.dbus: "1"
io.balena.features.firmware: "1"
hmt333:
privileged: true
build: ./hmt333
restart: always
expose:
- '7575'
volumes:
- 'settings:/data'
metoffice-wow-prod:
build: ./metoffice-wow-prod
restart: always
depends_on:
- hmt333
volumes:
- 'settings:/data'
influxdb:
image: influxdb@sha256:73f876e0c3bd02900f829d4884f53fdfffd7098dd572406ba549eed955bf821f
container_name: influxdb
restart: always
environment:
- INFLUX_DATA_DIR=/data
- PERSISTENT=1
volumes:
- 'sensor-data:/data'
connector:
image: balenablocks/connector:raspberrypi4-64
restart: always
depends_on:
- influxdb
- hmt333
labels:
io.balena.features.balena-api: '1' # necessary to discover services
io.balena.features.supervisor-api: 1 # necessary to discover services in local mode
privileged: true # necessary to change container hostname
dashboard:
image: balenablocks/dashboard:raspberrypi4-64
restart: always
volumes:
- 'dashboard-data:/data'
ports:
- '3000:80'
configuration:
build: ./configuration
restart: on-failure
privileged: true
depends_on:
- hmt333
ports:
- '8080'
volumes:
- 'settings:/data'
# The following 'browser section' should be commented out for the HMT datahub
browser:
# image: balenablocks/browser:raspberrypi4-64
# image: bhcr.io/balenablocks/browser-aarch64 # where <arch> is one of aarch64, arm32 or amd64
image: bh.cr/balenalabs/browser-aarch64
privileged: true # required for UDEV to find plugged in peripherals such as a USB mouse
network_mode: host
environment:
- LAUNCH_URL=http://0.0.0.0:3000
- ENABLE_GPU=0
- KIOSK=1
- LOCAL_HTTP_DELAY=8
- PERSISTENT=1
- WINDOW_SIZE=800,600
- WINDOW_POSITION=0,0
volumes:
- 'settings:/data' # Only required if using PERSISTENT flag (see below)
Thanks again for this info. I also don’t see anything wrong with the composition. Unfortunately I don’t have any more ideas here, you mention that you saw the problem even on a new device which made me suspect the release, however that doesn’t explain why rolling back to a working release.
If you see this issue again, please make sure to check the variables BALENA_API_KEY and SUPERVISOR_API_KEY when inspecting the container. You can get that information SSHing into the device and running
I have the same issue.
The device was working fine for 4 months and then the Connector started with the same error as shown here. I believe I’ve had the issue before and the only solution was to create the same project as a new project and flash it to the usb for the device, as mentioned in the solution here.
This time, I’ve been trying to fix the issue, but to no luck.
I can confirm that the BALENA_API_KEY from inspect is the same as /mnt/boot/config.json.
I have tried clearing all data from the device, Rebooting the device, Restarting the connector, Upgrading the OS, Upgrading Supervisor, Running a new build and Pushing it to the device. Even after upgrading the connector, the same error persists.
I can’t get ‘curl’ to work in the Host OS Terminal.