Hello,
My balena fin board is in a very strange state. This morning I tried purging data (as I have done many times before). Since the purge, I have not been able to push a new image. When I try to push a new image, I get the following output:
BUILD SUCCESSFUL in 2s
8 actionable tasks: 8 executed
[debug] original argv0="/usr/local/lib/balena-cli/bin/node" argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,push,192.168.2.113,–nolive] length=5
[debug] Using . as build source
[Debug] Checking we can access device
[Debug] Sending request to http://192.168.2.113:48484/ping
Could not communicate with local mode device at address 192.168.2.113
My balena fin is connected via Ethernet. I can ping it, and I can ssh to the Balena OS (using “balena ssh {ip_address}”). I cannot ssh to our application container, which appears to have been deleted with the purge.
I ran the Balena cloud diagnostics and the check_image_corruption test failed. I verified this is not due to a disk space issue.
What else can I do, or look at? I’d like to figure out what happened and how this can be repaired (if possible) in case this happens to any of our devices in the field.
Thank you!
Hi dstewart,
Can you tell us what version of balenaOS (and supervisor) you’re working with? Also, have you tried using the prune command from the Host OS shell ```balena system prune [–volumes]`` to clear away any previous artifacts`?
John
Hi John,
Thank you for the fast response! I ran “balena system prune --volumes”, but still get the same error when I try to push a new image.
Balena OS: 2.48.0+rev1
Supervisor: 10.8.0
Hi @dstewart,
Is the project you’re pushing public? I can try on a balenaFin and see if I get any errors. Do you get the same result when pushing different project code? Also, is the application in “local node”? If so, try disabling that (for now).
John
The project is not public. I have pushed this same project many times, so I don’t expect that to be part of the problem. The gateway is not running in local mode currently.
Would you like me to grant support access? Would that be helpful?
Hi @dstewart,
Go ahead and grant support access. There’s nothing on the device currently that you want, right? I’ll take a look at the logs and prompt you before I take any actions (unless you instruct me otherwise).
Thanks,
John
Okay, I just granted access. Correct - nothing we want. We are hoping to understand what happened and how we could recover the gateway if this happens again.
Thank you!
Can you provide the device UUID?
UUID: de16851e66820403c2e27712a180d90c
Hi,
If you’re not in local mode, try pushing using the application name, i.e. balena push gateway. Using the device IP address only works if you’re in local mode.
John
I have tried that approach too (balena push {gateway}), and get the same error.
Hi,
Can you add an environment variable DEBUG=1 and paste the complete logs you get? Also, the device ID you sent appears to show an active, running container.
John
Yes, here is the full output:
damon@:~/Development/github/gateway$ balena push de16851.local --nolive
[debug] original argv0="/usr/local/lib/balena-cli/bin/node" argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,push,de16851.local,–nolive] length=5
[debug] Using . as build source
[Debug] Checking we can access device
[Debug] Sending request to http://de16851.local:48484/ping
Could not communicate with local mode device at address de16851.local
If you need help, don’t hesitate in contacting our support forums at
https://forums.balena.io
For CLI bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/
Thanks. Can you try the push with the application name? balena push gateway?
It was running, but I got nervous and aborted it. We have a bunch of gateways. How does this know to only update the gateway here at my desk, and to not update all of our gateways?
Hi,
Do you have multiple applications named “gateway”? If so, that could be a problem. Try renaming the application “gateway-1” (or something like that) from application’s Actions menu. Then do balena push gateway-1.
John
I’m not familiar with the “balena push {application_name}” command. That seems to push the image to the balena cloud, right? If so, will all of my gateways in the field update to this image? This is a dev image and I need to be certain it will only be uploaded to the one gateway at my desk, and not all gateways.
If you have multiple devices in a single app, all will be updated at the same time with the latest unless you pin the other devices to a previous release. You can Pin from the chevron menu (next to the Restart button).
You can also create a new application (gateway-dev) and move the device to that application. Do your push to balena push gateway-dev). When you move the device back to “gateway” it will receive the last push you did to the application. There’s a 10-device limit on free accounts, but no limit on applications.
Generally speaking, all devices in an application are typically set to run the same version of the code you push, creating a fleet of that version. That way, each new device you deploy to any application automatically gets provisioned to that application. You can pin devices to avoid having them all (or some) get the latest push, but it can get tricky to manage all the pinned versions. Better to create separate applications for different versions of an application or use environment variables to manage specific difference.
John
One more question. Doing the balena push {application} will push a new image through the cloud, which may or may not work. But it doesn’t explain why I can’t push an image locally (when the gateway is in local mode). Do you have any idea why that isn’t working? Or do you know why the “check_image_corruption” test is failing? (The gateway is not in local mode now.)