How to manage OpenBalena Certificate expiration

openBalena
1

Hello,
we are hosting an OpenBalena Server and we are planning to deploy a lot of devices under it.
We have installed certificates as in the guide and everything is running fine by now, but we have some conerns about what happen when the certificates expires.
OpenBalena gives a self-signed certificate, as far as I know it is a 10 years CA certificate and 2 years client certificate.
What happens after 2 years?
Do we need to perform some actions on each individual device?
It will be a lot of them… so we have to have a clear plan of action.
Does the certificates have some auto-renewal or is there a way to massively update the certificates via API?

Thank you,
Max

1 Like
2

I’m making some integration on the previous request.
We tried to set the Production certificate using letsencrypt using the quickstart -c option.
At first we got some errors, but we managed to overcome them thanks to this post:

Commenting the lines 182:183 on cert-provider.sh AND update the Dockerfile to use version 3.0.1 of ACME script “seems” to have solved the installation.
Now the log actually say it has installed a PRODUCTION certificate.

But actually when we test the api.mydomain.com we still don’t have a trusted certificate.
The issuer appear to be (STAGING) Artificial Apricot R3

Why it isn’t a PRODUCTION certificate?
Could we use a self-signed certificate that expires in 100 years?
It’s important to us not to interrupt workload and communications for a certificate expiration, and we will have a lot of devices.

Here is the log from the cert-providers container:

[Info] VALIDATION not set. Using default: http-01
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Failed. Retrying in 5 seconds...
[Info] (2/3) Connecting...
[Info] (2/3) Failed. Retrying in 5 seconds...
[Info] (3/3) Connecting...
[Info] (3/3) Failed!
[Info] Unable to access api.eci2.cloud on port 80. This is needed for certificate validation. Retrying in 30 seconds...
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Success!
cat: can't open '/usr/src/app/certs/last_run_mode': No such file or directory
[Info] Last acquired certificate for 
[Info] Using STAGING mode
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Success!
[Info] Issuing certificates...
[Mon Nov 15 16:29:10 UTC 2021] Using ACME_DIRECTORY: https://acme-staging-v02.api.letsencrypt.org/directory
[Mon Nov 15 16:29:10 UTC 2021] Using CA: https://acme-staging-v02.api.letsencrypt.org/directory
[Mon Nov 15 16:29:10 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:10 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:10 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:11 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:11 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:11 UTC 2021] Create account key ok.
[Mon Nov 15 16:29:11 UTC 2021] Registering account: https://acme-staging-v02.api.letsencrypt.org/directory
[Mon Nov 15 16:29:12 UTC 2021] Registered
[Mon Nov 15 16:29:12 UTC 2021] ACCOUNT_THUMBPRINT='vWqfkRV8I-oTrgg6PCEPIiphzi8RZF17KtjS3iLAiBo'
[Mon Nov 15 16:29:12 UTC 2021] Creating domain key
[Mon Nov 15 16:29:12 UTC 2021] The domain key is here: /usr/src/app/certs/api.eci2.cloud/api.eci2.cloud.key
[Mon Nov 15 16:29:12 UTC 2021] Multi domain='DNS:api.eci2.cloud,DNS:registry.eci2.cloud,DNS:s3.eci2.cloud,DNS:vpn.eci2.cloud,DNS:tunnel.eci2.cloud'
[Mon Nov 15 16:29:12 UTC 2021] Getting domain auth token for each domain
[Mon Nov 15 16:29:18 UTC 2021] Getting webroot for domain='api.eci2.cloud'
[Mon Nov 15 16:29:18 UTC 2021] Getting webroot for domain='registry.eci2.cloud'
[Mon Nov 15 16:29:18 UTC 2021] Getting webroot for domain='s3.eci2.cloud'
[Mon Nov 15 16:29:18 UTC 2021] Getting webroot for domain='vpn.eci2.cloud'
[Mon Nov 15 16:29:18 UTC 2021] Getting webroot for domain='tunnel.eci2.cloud'
[Mon Nov 15 16:29:18 UTC 2021] Verifying: api.eci2.cloud
[Mon Nov 15 16:29:18 UTC 2021] Standalone mode server
[Mon Nov 15 16:29:19 UTC 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Mon Nov 15 16:29:22 UTC 2021] Success
[Mon Nov 15 16:29:22 UTC 2021] Verifying: registry.eci2.cloud
[Mon Nov 15 16:29:22 UTC 2021] Standalone mode server
[Mon Nov 15 16:29:23 UTC 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Mon Nov 15 16:29:26 UTC 2021] Success
[Mon Nov 15 16:29:26 UTC 2021] Verifying: s3.eci2.cloud
[Mon Nov 15 16:29:26 UTC 2021] Standalone mode server
[Mon Nov 15 16:29:28 UTC 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Mon Nov 15 16:29:30 UTC 2021] Success
[Mon Nov 15 16:29:30 UTC 2021] Verifying: vpn.eci2.cloud
[Mon Nov 15 16:29:30 UTC 2021] Standalone mode server
[Mon Nov 15 16:29:32 UTC 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Mon Nov 15 16:29:34 UTC 2021] Success
[Mon Nov 15 16:29:34 UTC 2021] Verifying: tunnel.eci2.cloud
[Mon Nov 15 16:29:34 UTC 2021] Standalone mode server
[Mon Nov 15 16:29:36 UTC 2021] Pending, The CA is processing your order, please just wait. (1/30)
[Mon Nov 15 16:29:38 UTC 2021] Success
[Mon Nov 15 16:29:38 UTC 2021] Verify finished, start to sign.
[Mon Nov 15 16:29:38 UTC 2021] Lets finalize the order.
[Mon Nov 15 16:29:38 UTC 2021] Le_OrderFinalize='https://acme-staging-v02.api.letsencrypt.org/acme/finalize/33811898/1024739498'
[Mon Nov 15 16:29:42 UTC 2021] Downloading cert.
[Mon Nov 15 16:29:42 UTC 2021] Le_LinkCert='https://acme-staging-v02.api.letsencrypt.org/acme/cert/faafb47d6f8f567c71b75813a432e106291d'
[Mon Nov 15 16:29:42 UTC 2021] Cert success.
-----BEGIN CERTIFICATE-----
[Edited: Obfuscated certificate]
-----END CERTIFICATE-----
[Mon Nov 15 16:29:42 UTC 2021] Your cert is in: /usr/src/app/certs/api.eci2.cloud/api.eci2.cloud.cer
[Mon Nov 15 16:29:42 UTC 2021] Your cert key is in: /usr/src/app/certs/api.eci2.cloud/api.eci2.cloud.key
[Mon Nov 15 16:29:42 UTC 2021] The intermediate CA cert is in: /usr/src/app/certs/api.eci2.cloud/ca.cer
[Mon Nov 15 16:29:42 UTC 2021] And the full chain certs is there: /usr/src/app/certs/api.eci2.cloud/fullchain.cer
[Info] Installing certificates...
[Mon Nov 15 16:29:42 UTC 2021] Installing cert to: /tmp/cert.pem
[Mon Nov 15 16:29:42 UTC 2021] Installing key to: /tmp/key.pem
[Mon Nov 15 16:29:42 UTC 2021] Installing full chain to: /tmp/fullchain.pem
[Mon Nov 15 16:29:42 UTC 2021] Run reload cmd: cat /tmp/fullchain.pem /tmp/key.pem > /certs/open-balena.pem
[Mon Nov 15 16:29:42 UTC 2021] Reload success
[Info] Using PRODUCTION mode
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Success!
[Info] Issuing certificates...
[Mon Nov 15 16:29:43 UTC 2021] Using CA: https://acme.zerossl.com/v2/DV90
[Mon Nov 15 16:29:43 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:43 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:43 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:43 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:43 UTC 2021] Standalone mode.
[Mon Nov 15 16:29:44 UTC 2021] Create account key ok.
[Mon Nov 15 16:29:44 UTC 2021] No EAB credentials found for ZeroSSL, let's get one
[Mon Nov 15 16:29:44 UTC 2021] acme.sh is using ZeroSSL as default CA now.
[Mon Nov 15 16:29:44 UTC 2021] Please update your account with an email address first.
[Mon Nov 15 16:29:44 UTC 2021] acme.sh --register-account -m my@example.com
[Mon Nov 15 16:29:44 UTC 2021] See: https://github.com/acmesh-official/acme.sh/wiki/ZeroSSL.com-CA
[Mon Nov 15 16:29:44 UTC 2021] Please add '--debug' or '--log' to check more details.
[Mon Nov 15 16:29:44 UTC 2021] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
[Info] Installing certificates...
[Mon Nov 15 16:29:44 UTC 2021] Installing cert to: /tmp/cert.pem
[Mon Nov 15 16:29:44 UTC 2021] Installing key to: /tmp/key.pem
[Mon Nov 15 16:29:44 UTC 2021] Installing full chain to: /tmp/fullchain.pem
[Mon Nov 15 16:29:44 UTC 2021] Run reload cmd: cat /tmp/fullchain.pem /tmp/key.pem > /certs/open-balena.pem
[Mon Nov 15 16:29:44 UTC 2021] Reload success
[Success] Done!
[Info] Running cron...
crond: crond (busybox 1.33.1) started, log level 7
crond: USER root pid 3617 cmd run-parts /etc/periodic/15min
[Info] VALIDATION not set. Using default: http-01
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Failed. Retrying in 5 seconds...
[Info] (2/3) Connecting...
[Info] (2/3) Failed. Retrying in 5 seconds...
[Info] (3/3) Connecting...
[Info] (3/3) Failed!
[Info] Unable to access api.eci2.cloud on port 80. This is needed for certificate validation. Retrying in 30 seconds...
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Success!
[Info] Last acquired certificate for PRODUCTION
[Success] Done!
[Info] Running cron...
crond: crond (busybox 1.33.1) started, log level 7
crond: USER root pid  26 cmd run-parts /etc/periodic/15min
crond: USER root pid  27 cmd run-parts /etc/periodic/15min
crond: USER root pid  28 cmd run-parts /etc/periodic/hourly
[Info] VALIDATION not set. Using default: http-01
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Failed. Retrying in 5 seconds...
[Info] (2/3) Connecting...
[Info] (2/3) Failed. Retrying in 5 seconds...
[Info] (3/3) Connecting...
[Info] (3/3) Failed!
[Info] Unable to access api.eci2.cloud on port 80. This is needed for certificate validation. Retrying in 30 seconds...
[Info] Waiting for api.eci2.cloud to be available via HTTP...
[Info] (1/3) Connecting...
[Info] (1/3) Success!
[Info] Last acquired certificate for PRODUCTION
[Success] Done!
[Info] Running cron...
crond: crond (busybox 1.33.1) started, log level 7
crond: USER root pid  26 cmd run-parts /etc/periodic/15min

UPDATE:
I still cannot find a solution.
I’ve done

docker system prune -a
docker volume prune
quickstart -U {email} -P {password} -c -d {domain}

But no luck so far…

I’ve connected to cert-provider container and made some tests with the file cert-provider.sh
STAGING certificate does install, but doesn’t verify, so the operation is aborted.
Commenting lines 182:183 as suggested in the other topic skips to the PRODUCTION certificate, but it gives this error:
cat: can't open '/usr/src/app/certs/{domain}/{domain}.cer': No such file or directory

That’s the reason we cannot get a PRODUCTION certificate I guess…
Can someone point me to the right direction?

4

Hello @maxferretti78 Thanks for posting these questions! Let me try to answer them one at a time and then I’ll probably need some clarification on a few:

For a development scenario, using the self-signed certs is fine. But for anything production, as you have more or less already seen, you’ll want to use a cert provided by a provider like LetsEncrypt. The self signed certs are only needed if you don’t have a better option.

We have installed certificates as in the guide and everything is running fine by now, but we have some conerns about what happen when the certificates expires.

If you used self-signed certs, you would have to put the new certs on the devices. Not a great option.

OpenBalena gives a self-signed certificate, as far as I know it is a 10 years CA certificate and 2 years client certificate.
What happens after 2 years? Do we need to perform some actions on each individual device?

What exactly are you seeing? What is the message you are geting? How are you determining that the cert is staging versus production?

But actually when we test the api.mydomain.com we still don’t have a trusted certificate.
The issuer appear to be (STAGING) Artificial Apricot R3 Why it isn’t a PRODUCTION certificate?

Unfortunately, certs are limited to three years. See this link regarding certs

Could we use a self-signed certificate that expires in 100 years?

In your log I see what appears to be validation occurring. Where is the abortion occurring? Can you explain?

STAGING certificate does install, but doesn’t verify, so the operation is aborted.

I have a colleague who is an OpenBalena nerd. If sees something else to comment on, he may add it in.