Safe way to edit config.json on a remotely deployed device

Hi,

I was trying to look around and could not find anything too specific relating to editing a config.json with a fail-safe option, to be able to revert to a previous state if there is an issue/mistake that may prevent the device from booting.

I came across: config.json is single point of failure and it does get corrupted often

I have looked into some of the mentioned links/articles but I wanted a confirmation from the team, is there any possible way to edit a config.json, remotely, and if there is an issue or a mistake in the edited file, to somehow revert to last known working configuration or anything along those lines?

It would not be possible to reach the device physically, if the edited configuration caused it to “break” so I am cautious of attempting to do this.

I am planning on simply adding a udev rule to the config file of a remotely deployed device if that is of any help.

Thank you! :grin:

1 Like

Hello @balena101 there is this project however this is no longer maintained, so use it under your consideration!

More importantly in general we will not be able to help fix an unbootable device if you edit the config.json remotely. You will need to be onsite, and likely reflash the device. So you can proceed at your own risk.

1 Like

Okay, great!

Thank you for that, I understand, it’s kind of a last resort type of thing but I will gladly test this out in-office to see how it works. :grin:

1 Like

We’ve run into the same issue and had some long road-trips to fix that config.json file. We don’t have a solid fix either, but we now copy the original config.json, make our changes to the copy, then use jq to verify the changes resulted in valid JSON. Finally we replace the original with the copy.

In our case, most of our issues were syntactical, missing comma, unneeded comma, missing curly brace… etc. If we’re testing a change to the config.json that we haven’t used before, we test on a local device to make sure it behaves as expected, then enact the change on the remote device.

Kind of a bummer that the config.json contains breaking configuration items right next to mundane, non-breaking configuration items. Maybe someday they’ll get broken out into a few different files along with some redundancies.

1 Like

Let us know how does this work @balena101

Thanks for sharing @jchristensen

Hi @mpous,

So I tested out the configizer, it runs nicely, it connects to the device automatically and updates the config.json, it also makes it easier to read as before it was a messy looking json file whereas with the configizer it gets “pretty”, as in each key:value is on a new line, making it easier to follow.

I did change a couple lines in the run-one.sh:

This:

# don't run if the device has already been processed

grep -a -q '{} : DONE' config.log 2>/dev/null && exit 0

# these settings need to be handled separately because they need to be different

# per-device even when processing a large batch

newKey=$(grep "${uuid}" new_keys | sed 's/.*\s*->\s*//')

To:

# don't run if the device has already been processed
grep -a -q "${uuid} : DONE" config.log 2>/dev/null && exit 0

# Check if the new_keys file exists and process accordingly
if [[ -f new_keys ]]; then
    newKey=$(grep "${uuid}" new_keys | sed 's/.*\s*->\s*//')
else
    newKey=""
fi

As before, I would get the:

grep: new_keys: No such file or directory

If I wasn’t trying to change the API keys, hence there is no new_keys file, whereas with this update it won’t print that error. Not that it caused any issues but I just didn’t like how it showed that all the time.

I also made some changes in the config.sh code, mainly just for the udevRules option.

I changed the following function:

# Handling udevRules
 udevrulesInsert() {
     echo "Inserting udevRules values"
     local TEMPWORK
     TEMPWORK=$(tempwork)
     for key in "${!UDEVRULES[@]}" ; do
         jq  ".os.udevRules.\"${key}\" = \"${UDEVRULES[$key]}\"" "$WORKCONFIGFILE" > "$TEMPWORK" || finish_up "Couldn't insert rule ${key}."
         if [[ "$(jq -e ".os.udevRules.\"${key}\"" "$TEMPWORK")" == "" ]] ; then
             finish_up "Failed to insert ${key} rule into config.json."
         fi
         mv "${TEMPWORK}" "${WORKCONFIGFILE}" || finish_up "Failed to update working copy of config.json"
     done
 }

To (Unsure why it won’t format as code in this post):

udevrulesInsert() {
    echo "Updating udevRules values"
    local TEMPWORK
    TEMPWORK=$(tempwork)

    # Iterate through existing keys and remove any that are not in the new configuration
    existing_keys=$(jq -r 'try .os.udevRules | keys_unsorted[]' "$WORKCONFIGFILE")
    for key in $existing_keys; do
        if [[ -z "${UDEVRULES[$key]}" ]]; then
            jq "del(.os.udevRules.\"$key\")" "$WORKCONFIGFILE" > "$TEMPWORK" || finish_up "Couldn't remove rule ${key}."
            mv "${TEMPWORK}" "${WORKCONFIGFILE}" || finish_up "Failed to update working copy of config.json"
        fi
    done

    # Add or update the new rules
    for key in "${!UDEVRULES[@]}"; do
        jq  ".os.udevRules.\"${key}\" = \"${UDEVRULES[$key]}\"" "$WORKCONFIGFILE" > "$TEMPWORK" || finish_up "Couldn't insert rule ${key}."
        if [[ "$(jq -e ".os.udevRules.\"${key}\"" "$TEMPWORK")" == "" ]]; then
            finish_up "Failed to insert ${key} rule into config.json."
        fi
        mv "${TEMPWORK}" "${WORKCONFIGFILE}" || finish_up "Failed to update working copy of config.json"
    done
}

In the main() I also changed:

This:

if (( ${#UDEVRULES[@]} > 0 )); then
    DO_UDEVRULES="yes"
    anytask="yes"
fi

To this:

DO_UDEVRULES="yes"
anytask="yes"

So that it executes the udevrulesInsert function whether or not there is a udevRule present.

This way, when you comment out or remove the UDEV rule you added, it will remove it from the config.json and leave behind:

“os”: {
“udevRules”: {}
}

This is just a quick change, in case somebody decides to use this, they can change/clean it up as they wish. :smiley:

If anyone will be using this with Git Bash on a windows machine, I just had to set GitBash to use the native SSH option:

> export PATH="/c/Windows/System32/OpenSSH:$PATH

Otherwise it wouldn’t ssh into the device. :slight_smile:

Other than that it worked great, for now I was just testing it on the devices that I can access here, just in case I broke anything.

If anyone is interested, I was adding this rule: Auto-Mount External Storage into the configuration of my device.

Also @jchristensen, thanks for suggesting the use of jq, as that helps to check for simple syntax issues if not wanting to use the configizer. I would suggest if you can SSH into the device, copy the config.json file, insert the changes you would like to make, test this with jq (for peace of mind) and then you can enter the relevant change using the configizer as this will restart the balena supervisor to push the updated config.json. As, I tested removing a comma from the udevrule that i was adding and it still pushed to the device without giving any error. Perhaps if itnerested, someone can integrate the jq check on the config file into the configizer itself.

To check the actual config.json file I simply ran the below code using powershell, as I was checking this on a windows machine:

jq . config.json > $null
echo $LASTEXITCODE

This will return an error for syntax if there is one, otherwise it will return 0.
Once I know there is no syntax issues I can SSH into the device, manually edit the config.json by copy pasting my updated config file. Otherwise, the configizer will make the changes for you.