Set up SSL certificate on balenaOS devices (RPI3s)

Hi,

I’m wondering whether it’s possible to install a ssl certificate for my balena devices.

I’m currently running pi3s with the supervisor version 12.2.11 and balenaOS 2.65.0+rev1.

The devices are behind a router which I have a .crt and key key for.

I tried modifying the config.json on the devices. In these I added the balenaRootCA field including my base64 encoded .crt string.

This doesn’t work.

Could you provide any guidance?

Follow up …

So basically what I’m trying to do is to access a service (one container on the device - MQTT broker) via a device URL

The router my PI is behind is configured to be available at an address, e.g.: <protocol>://someaddress.com

The router is pointing at a static ip with a few forwarded ports.

The static IP is setup with the balena device and it does work to access the MQTT broker via the websocket protocoll e.g. ws://someaddress.com works.

However, I need to access the secured websocket, which is also offered by the service. E.g. wss://someaddress.com. This doesn’t work.

If it was somehow possible to connect to the websocket via the public balena device url, this would be even better. E.g.: wss://<balena_device_uuid>.balena-devices.com

I’m not sure what to do here :confused:

Another follow up…

I tried installing the cert as described here. This doesn’t work because:

  1. /usr/local/ doesn’t exist
  2. /usr/* is read only
  3. It doesn’t specify where to put the .key file

I tried to install it via the balenaRootCA entry in the config.json. This also doesn’t work and also doesn’t specify where to put the .key file.

I have Googled for different ways for different OS to deduct how to do this, but nothing seems to work on balenaOS. I just can’t figure out how to make this work…

Hello @jebraat thank you for your messages!

Could you please share your docker-compose to try to understand if this is a problem of the privileges of the containers or similar?

Remember that the public device URL is only available on port 80, 443 and 8080.

Looking forward to get more information to help you :slight_smile:

Hey @mpous thank you for getting back to me! :slightly_smiling_face:

Here is my docker compose:

version: '2'

services:
  cloud-relay:
    build: ./cloud-relay
    depends_on:
      - mqtt-broker
    restart: unless-stopped
    labels:
      io.balena.features.supervisor-api: '1'
  gps-rtk-service:
    build: ./gps-rtk
    depends_on:
      - mqtt-broker
    privileged: true
    restart: unless-stopped
    devices:
      - "/dev/ttyACM0:/dev/ttyACM0"
      - "/dev/ttyACM1:/dev/ttyACM1"
    volumes:
      - gps-data:/usr/data
  rtcm-service:
    build: ./rtcm-service
    privileged: true
    restart: unless-stopped
    devices:
      - "/dev/ttyACM0:/dev/ttyACM0"
      - "/dev/ttyACM1:/dev/ttyACM1"
  mqtt-broker:
    image: emqx/emqx:4.3.10
    ports:
      - "80:18083" # Dashboard
      - "1883:1883" # MQTT/TCP protocol port
      - "11883:11883" # MQTT/TCP Protocol internal port, only used for local client connection
      - "8883:8883" # MQTT/SSL protocol port
      - "8083:8083" # MQTT/WS protocol port
      - "8084:8084" # MQTT/WSS protocol port

volumes:
  gps-data:

It’s the mqtt broker I want to access on either of the secured ports. Are you suggesting the broker needs privileged access?

Ultimately I would prefer to use the balena device url via wss. If that doesn’t work because it can only do http, the second best option is to install a certificate .crt and key .key on the device which is provided by a third party. They are controlling the gateway/router and are forwarding the emqx ports (see above).

I assume I need to configure a certificate for EMQX, as described here or here in the docker environment.

However, I’m not sure where to put the .crt and .key file in the context of balena, given the directories are read only on the host. I guess I need to include the certificate and key in the build?!

Thanks in advance!

Cheers

1 Like

wow this looks amazing @jebraat

did you try to add a nginx reverse proxy to manage all the ports that you would like to manage? Check this example to get some ideas → GitHub - balena-io-examples/nginx-reverse-proxy: Access multiple ports over public URL using nginx reverse proxy.

On the other hand, you have a cacert.pem on the right place? i checked the docs from emqx → Configuration | EMQX 4.4 Documentation

Let us know if you make this work!

Thanks. It’s a work in progress @mpous
Building a telemetry solution for tower cranes on commercial construction sites

I would like to stay away from reverse proxies if I can as this makes the setup unecessarily complicated.

I read in the docs the public device URL only supports HTTP. Since WSS isn’t technically HTTP but and upgrade, does this mean the balena device URL does not support ws/s?

Regarding the cert. I’m not sure how to get it on the device. As described here:

I tried installing the cert as described here. This doesn’t work because:

  1. /usr/local/ doesn’t exist
  2. /usr/* is read only
  3. It doesn’t specify where to put the .key file

I tried to install it via the balenaRootCA entry in the config.json. This also doesn’t work and also > > doesn’t specify where to put the .key file.

I have Googled for different ways for different OS to deduct how to do this, but nothing seems to work > on balenaOS. I just can’t figure out how to make this work…

How can I get the cert onto the device?

1 Like

Hello @jebraat the public URL can’t provide wss we only provide http on 443, 80 and 8080 ports.

Answering about where to install the certificates, you need to create the volume on the container where you are going to use the certificate or just use the /app folder if i’m not wrong.

More questions! Is the device who is trying to access to your broker connected on the same network? if so, can you use the local address? if you want to use the edge device as your global MQTT/wss broker probably you will need to add extra-containers.

I found this forums message that maybe can help you → Public URL and Socket.io

Finally, let me share here a project that is using wss service with letsencrypt digital certificates. You can find it here → basicstation/runner at master · xoseperez/basicstation · GitHub and here copying the cacert.pem on the app directory → basicstation/start.sh at fe59373667060da3f86c5757545362d74160aafe · xoseperez/basicstation · GitHub

Let me know if this can help!

Hi @mpous

Ok, makes sense it doesn’t work with the device URL

To your questions

Is the device who is trying to access to your broker connected on the same network?

No the device is not in the same network.

The client is in an unknown remote network. The balena device is behind a gateway, which is accessible via the internet. The Gateway forwards specific ports (8083, 8084, 1883, and 8883) and points at the balena device in it’s own network.

Think of it like having a balena device at your home and your router forwards these ports. The router (gateway) has an address someaddress.com.

This address is accessible via the internet and this already works with the unsecured websocket connection.

E.g. ws://someaddress.com:8083 works as intended.

if you want to use the edge device as your global MQTT/wss broker probably you will need to add extra-containers

It’s not clear to me, why there should be additional containers. It already works with an unsecured websocket connection, so I assume it’s a certificate issue.

Either we set up a device wide certificate, which was the initial question of this thread or we somehow make it work for the emqx container.

It’s still unclear to me where to save the certificate on the balena host. E.g. which directory holds the ssl certificates, and which directory holds the certificate keys?

I have tried something else.

I created a new image with the certificates copied into the container filesystem. However this also doesn’t work.

FROM emqx/emqx:4.3.10

COPY domain.crt /etc/certs/emqx.crt
COPY domain.key /etc/certs/emqx.key

CMD ["/opt/emqx/bin/emqx",  "foreground"]

@jebraat how did this go? Could you get an SSL certificate for your local ip address on the RPI3?

I’m looking to do this in pubs, and would appreciate understanding your solution if you could get it working.

@mpous, have you had a chance to assess if there are options for providing this? It seems like Let’s Encrypt certificates for OpenBalena can be easily generated, is there any ways to get such certificates for a device on a local network?

Hey @SpencerWF,

Never got this to work and had to put it aside for lack of time.

I’m currently shifting the comms to AWS IoT.

I would still be interested in a solution if you find one

2 Likes

Thanks for the reply @jebraat, why are you shifting to AWS IoT? I haven’t looked into it yet, I hope it goes well! If I find a solution I’ll let you know.

Users should most of the time have internet access for us, which means that an external site can be used if needed, so I think that might have to be the solution I use.