Failing to persist data to host OS and access it from main

Product support
1

Hi

I have problems persisting data to the host OS. I have followed this guide: https://www.balena.io/docs/learn/develop/runtime/#persistent-storage and read several post on the forum with no luck.

My project is based on the Google Cloud IoT integration: https://www.balena.io/docs/learn/develop/integrations/google/
and i need to persist certificate files between restarts of my Raspberry Pi

My understanding of how the persistence of data works with the Balena container is that everything placed in the /usr/src/app/data folder of the project will be persisted into the hostOS at /var/lib/docker/volumes/resin-data/_data as my Balena OS version is 2.29.2+rev2 and then be accessible from /usr/src/app/data folder in the container automatically after restart.

So the gist of my bash script generating the certificates is as follows:

#!/bin/bash

if [ ! -f ./data/service.json ]; then

    mkdir -p data

    echo "$GOOGLE_IOT_SERVICE_JSON" > data/service.json

    cd data
        
    # Create keys
    openssl req -x509 -newkey rsa:2048 -keyout rsa-priv.pem -nodes -out rsa-cert.pem -subj "//CN=unused"
    openssl ecparam -genkey -name prime256v1 -noout -out rsa-ec_private.pem
    openssl ec -in rsa-ec_private.pem -pubout -out rsa-ec_public.pem

    cd -
fi

node dist/main.js

However as mentioned, the data is not persisted and new certificates are generated every time the Pi is restarted.

When pushing the project to the Pi I am able to cd to the var/lib/docker/volumes/resin-data/_data folder in the hostOS but it remains empty.

I have attempted pushing from both Windows and mac.
When running the bash script on my dev computer, it works perfectly.

I have assumed it just working out of the box, but is there any settings or configs i need to set before data persistency works?

Thanks!

4

You want /data not /usr/src/app/data

#!/bin/bash

if [ ! -f /data/service.json ]; then

    echo "$GOOGLE_IOT_SERVICE_JSON" > /data/service.json

    pushd /data
        
    # Create keys
    openssl req -x509 -newkey rsa:2048 -keyout rsa-priv.pem -nodes -out rsa-cert.pem -subj "//CN=unused"
    openssl ecparam -genkey -name prime256v1 -noout -out rsa-ec_private.pem
    openssl ec -in rsa-ec_private.pem -pubout -out rsa-ec_public.pem

    popd
fi

node dist/main.js
6

(I used pushd and popd because I wasn’t familiar with cd -)

7

Hi @TobiasEmil,
@jason10 response is indeed accurate, you need to use /data from within your container to access the persistent storage directory.

Just to correct the above, as mentioned earlier, /usr/src/app/data should be /data. Moreover, instead of /var/lib/docker/volumes/resin-data/_data, the correct host OS path is /var/lib/docker/volumes/<APPID>_resin-data/_data.

Please also keep in mind that the resin-data named volume is only automatically linked in a single-container application. If you have a docker-compose.yml in your project then you should also specify the used volumes in there.

Give us a note if you still face issues with using persistent storage on your app.

Kind regards,
Thodoris

8

Hi @TobiasEmil,
@jason10 response is indeed accurate, you need to use /data from within your container to access the persistent storage directory.

TobiasEmil:

My understanding of how the persistence of data works with the Balena container is that everything placed in the /usr/src/app/data folder of the project will be persisted into the hostOS at /var/lib/docker/volumes/resin-data/_data as my Balena OS version is 2.29.2+rev2 and then be accessible from /usr/src/app/data folder in the container automatically after restart.

Just to correct the above, as mentioned earlier, /usr/src/app/data should be /data. Moreover, instead of /var/lib/docker/volumes/resin-data/_data, the correct host OS path is /var/lib/docker/volumes/_resin-data/_data.
Please also keep in mind that the resin-data named volume is only automatically linked in a single-container application. If you have a docker-compose.yml in your project then you should also specify the used volumes in there.
Give us a note if you still face issues with using persistent storage on your app.
Kind regards,
Thodoris

11

I got it to work from your advice, thanks! I have spent ages figuring out the right paths as my work environment is Windows.

Now that the /data dir does not exists in my workdir anymore my JS application has trouble finding the certificates when running on my dev machine:

    const privateKeyFile = '/data/rsa-priv.pem'
    const jwt = this.createJwt(this.projectId, privateKeyFile, this.algorithm)

This is not at problem when pushing to the device, however I when i run the bash script and subsequently the JS app, the app looks for the file in ‘C:\data’

I have fixed this right now by copying the files from the /data folder into the workdir again in my bash script:

    mkdir -p ./data
    cp -r /data/* ./data

    node dist/main.js

and changed the file path in the JS file to:

    const privateKeyFile = './data/rsa-priv.pem'

But om not sure whether this is a preferable way to handle it?

12

Thanks!

13

I’d probably go with changing the file path in the JS file to the persistent storage

    const privateKeyFile = '/data/rsa-priv.pem'

Or, you can add a link from the persistent storage to the local directory:

ln -s /data ./data

Or, you can use a variable in your JS file, and fetch the value from the environment or rewrite the variable/marker when you deploy.

Questions to ask:

  1. Will this be part of a multicontainer build, and might you have to share the keys across containers. If yes, you will eventually have to move from using /data to /some-other-shared-volume that you will have to specify in your docker-compose.yml file
  2. Is this path likely to change across deployments? If you have non-balena.io deployments where the path might be different, you might want to use a variable or a deployment time rewrite.
  3. Do you already have configuration values in a configuration file or in the runtime environment that this path could move into?
16

Thank you for the answer!

The application will remain fairly simple so I do not plan on dividing it into multiple containers for now

I think I will go with the link, as it works on my dev environment.