Preloaded Image Doesn't Start

westerweerdt · October 28, 2020, 11:56am

Yeah, already did and got it working.

Thanks!

westerweerdt · October 30, 2020, 4:16am

Hmm, no preloaded image doesn’t communicate with my openBalena server:

Here is a log of supervisor:

root@yokoso:~# balena ps
CONTAINER ID        IMAGE                            COMMAND                  CREATED             STATUS                   PORTS               NAMES
6e5af6ad869e        balena/rpi-supervisor:v11.12.4   "/usr/src/app/entry.…"   6 minutes ago       Up 6 minutes (healthy)                       resin_supervisor
aca145dae1ab        9699d4ae6e16                     "/usr/bin/entry.sh b…"   2 months ago        Up 10 hours                                  yokoso_5_5
root@yokoso:~# balena logs 6
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
Warning: Ignoring extra certs from `/etc/ssl/certs/balenaRootCA.pem`, load failed: error:02001002:system library:fopen:No such file or directory
[info]    Supervisor v11.12.4 starting up...
[info]    Setting host to discoverable
[warn]    Invalid firewall mode: . Reverting to state: off
[info]    🔥 Applying firewall mode: of
[debug]   Starting logging infrastructure
[info]    Starting firewall
[success] 🔥 Firewall mode applie
[debug]   Starting api binder
[event]   Event: Supervisor start {}
[error]   LogBackend: unexpected error: Error: self signed certificate in certificate chain
[error]         at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
[error]       at TLSSocket.emit (events.js:310:20)
[error]       at TLSSocket._finishInit (_tls_wrap.js:917:8)
[error]       at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12)
[debug]   Performing database cleanup for container log timestamps
[info]    Previous engine snapshot was not stored. Skipping cleanup.
[debug]   Handling of local mode switch is completed
[debug]   Connectivity check enabled: true
[debug]   Starting periodic check for IP addresses
[info]    Reporting initial state, supervisor version and API info
[debug]   VPN status path exists.
[info]    VPN connection is not active.
(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
[info]    Waiting for connectivity...
[info]    Attempting to load any preloaded applications
[success] Preloading complete
[info]    Starting API server
[info]    Supervisor API successfully started on port 48484
[info]    Applying target state
[debug]   Ensuring device is provisioned
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[error]   LogBackend: unexpected error: Error: self signed certificate in certificate chain
[error]         at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
[error]       at TLSSocket.emit (events.js:310:20)
[error]       at TLSSocket._finishInit (_tls_wrap.js:917:8)
[error]       at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12)
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[debug]   Finished applying target state
[success] Device state apply success
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[api]     GET /v1/healthy 200 - 182.323 ms
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}
[event]   Event: Device bootstrap {}
[info]    New device detected. Provisioning...
[event]   Event: Device bootstrap failed, retrying {"delay":30000,"error":{"message":""}}

Is there any other thing I should check? Seems certification issue?

Here are commmands I had used to preload the image file:

balena os configure balena.img --app yokosoApp
balena preload balena.img --add-certificate <OPENBALENA CERT FILE> --app yokosoApp --commit current --splash-image my_splash.png

Thanks!

codewithcheese · October 30, 2020, 6:17am

Hi Wester, we just updated openBalena components and documentation. Please have a look at the updated getting started guide to make sure you have your openBalena installation setup as described there. Hopefully that fixes your issue. https://www.balena.io/open/docs/getting-started/ Once your all updated let us know if your issue persists.

codewithcheese · October 30, 2020, 6:19am

Also check out the README to find the new requirements https://github.com/balena-io/open-balena

"The current release of openBalena has the following minimum version requirements:

balenaOS v2.58.3
balena CLI v12.23.4

If you are updating from previous openBalena versions, ensure you update the balena CLI and reprovision any devices to at least the minimum required versions in order for them to be fully compatible with this release, as some features may not work."

westerweerdt · October 30, 2020, 6:24am

Thanks for the update!

But where is the latest balenaOS image?

codewithcheese · October 30, 2020, 6:38am

I see the latest image for PiZero is BalenaOS 2.54.2+rev1 ( https://www.balena.io/os/#download) give that a try with an upgraded openBalena, and I will ask my colleagues about compatibility for devices that don’t have the new OS version yet.

westerweerdt · October 30, 2020, 6:41am

Would be better to wait for the updated version? Seems waste of time?

codewithcheese · October 30, 2020, 7:01am

Ok if you would prefer not to risk it. I will update you when I hear back from my colleagues. In the mean time with your certificate issue I noticed “WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
Warning: Ignoring extra certs from /etc/ssl/certs/balenaRootCA.pem, load failed: error:02001002:system library:fopen:No such file or directory” can you confirm that the --add-certificate <OPENBALENA CERT FILE> file path is correct.

westerweerdt · October 30, 2020, 7:04am

Yes, without that parameter, balena preload command didn’t work. See this link - https://github.com/balena-io/balena-cli/issues/2068#issuecomment-713178034

codewithcheese · October 30, 2020, 7:09am

Ok so, version 2.58.6 for pi-zero is available on staging (finally testing before release) you can download it using the following. BALENARC_BALENA_URL="balena-staging.com" balena os download raspberrypi --version=2.58.6+rev4 -o rapsberrypi-2.58.6+rev4.img

codewithcheese · October 30, 2020, 7:12am

Are you setting export NODE_EXTRA_CA_CERTS='/path/to/ca.crt' before running CLI?

westerweerdt · October 30, 2020, 7:13am

Of course, yes

codewithcheese · October 30, 2020, 7:19am

And you have completed these steps “Install self-signed certificates on the local machine” https://www.balena.io/open/docs/getting-started/#Install-self-signed-certificates-on-the-local-machine

westerweerdt · October 30, 2020, 7:20am

Dude, I have been using balenaOS for over a year! lol

codewithcheese · October 30, 2020, 7:21am

Just checking since I thought those steps might have changed with the new update.

westerweerdt · October 30, 2020, 7:21am

Will try the new version once the image is ready. Thanks!

codewithcheese · October 30, 2020, 7:25am

I will ask my colleagues who are involved in preload and openBalena development if they know the solution to the certificate issue on the version you are running.

westerweerdt · November 1, 2020, 2:54am

Hi, @codewithcheese

Any update here?

richbayliss · November 2, 2020, 2:21pm

Wester,

We need to reproduce this to see what’s causing the certificate to not be expanded at runtime into the correct FS locations. You can see this yourself in the start-resin-supervisor script but I am not sure why your process has resulted in the failure.

To confirm, you have preloaded successfully and wifi-connect ran on boot. Then you connected to the internet and the API comms in the Supervisor is failing. Is that correct?

Could you also, for the sake of clarity, confirm the versions of:

balenaOS on the PiZero
balenaCLI used to configure and preload the PiZero image
openBalena stack

Thanks.

westerweerdt · November 2, 2020, 8:11pm

HI, @richbayliss

Yes, preloaded successfully, but wifi-connect wasn’t executed as my app didn’t launch automatically.
Does the resin-supervisor need internet connection to start the containers at the booting time?

Versions I have used:

openBalena: v2.0.0 on EC2 instance.
Used balena cli v12.11.0 to deply my app.
Used balena cli v11.7.10 to preload my application to the image file.
balenaOS: 2.54.2+rev1

Please check my original question to see why I had to select these versions…