We are trying to use the Balena Offline Update process to deploy application updates to a region without Balena Cloud support. Our application uses local device-specific environment variables to configure our application (log directory, logging behavior, application type, etc.)
Is there a way that we can specify these device-specific environment variables when creating an offline update?
Thank you for any help!
In our use case, we’re trying to deploy updates to pre-existing offline Balena devices. These devices already have device-specific environment variables assigned to them, which I would like to be included as part of the balena preload process.
In our own experimentation, we found that only the fleet-level default variables are included in a preloaded image, so there’s no way to customize these offline deployments with device-specific variables. I also don’t see a way to manually the config.json to override any environment variables. I’ve read a few things about configuring apps.json, but there’s no documentation on the format for us to try creating/editing this file, and this file may be deprecated anyways.
In order to accomplish our goal, it looks like we’ll need to modify the fleet-level default environment variables before each preload operation, which I find risky. Our fleet is a mix of online and offline devices, so this triggers updates to all the online devices. If this Github issue were addressed, that would satisfy our use case for offline updates.
Is there any way we can specify device variables using balena preload without modifying the fleet default variables? Thanks!
Hi! We’re still having this problem in multiple Balena fleets that we own. Some of our fleets are a mix of online and offline devices, so when we update the fleet-level default variables, there is the risk of our online devices restarting as the fleet-level default environment variable changes.
Additionally, I have to go and check that each device in the fleet overrides the default environment variable value, so that its behavior doesn’t change.
The Balena preload tool allows us to input a specific device UUID, so could you all use that as a source for the environment variable values? This would save our team a lot of process and re-work as we prepare offline updates for our services.
Can anyone on the Balena team please weigh in on this, and let us know if this can be considered for a new feature? Thank you!
@ntardif-glowforge, I’m not sure how this one slipped through the net.
I’m a little confused by some of the explanations there around the environment variables pre- and post- offline update. I am going to throw a link in here to something created by a user for offline updates, which seems to have some element of variable configuration options: Interactive Bash script that performs a Balena Offline Update on an SD card. Does this help at all with your initial question?
Hi Maggie, thanks for sharing! I took a look through that, but it doesn’t provide a solution for the problem I’ve brought up here. I just ran through the offline update process one more time to validate that I’m seeing this behavior. I’ll provide a concrete example for clarity.
We have a fleet that serves an app with a GUI interface. This serves customers both in the US and in Mexico. We use a Balena-configured environment variable called GUI_LANGUAGE to determine whether to display the GUI in English or Spanish. The fleet-level default is set to SPANISH.
I’m attempting to deploy an offline update to an English customer. In the Balena device page, I can see that the Balena cloud has configured the GUI_LANGUAGE value on this device to ENGLISH. I’ve attached a picture to illustrate.
However, when I follow the offline update process, which includes inputting the device UUID, after deploying the offline update, the GUI displays in Spanish. This tells me that something in the offline update generation process does not fetch the device-specific environment variable overrides.
As a workaround, I believe you can edit the environment variables under the following key paths in apps.json, a file stored in root of the resin-data partition of the pre-loaded image:
apps.<uuid>.releases.<uuid>.services.<serviceName>.environment
apps.<uuid>.releases.<uuid>.services.<serviceName>.composition.environment
I looked into this a bit, and it appears that the balena-cli GitHub issue you linked above will block you from achieving an automated solution. In my tests, balena preload will only add the fleet default environment variables. My interactive offline update script won’t help you either, because if you specify environment variables, the script uses the balena-cli to update them in balenaCloud. It then uses the balena preload command to generate the apps.json, which leads you back to the blocking root issue. I don’t have plans to add a workaround in my script, as it would be a much better use of time to fix the issue upstream in the CLI.
I don’t believe that apps.json will be deprecated. It has gone through a couple versions, so it’s possible you saw something related to deprecating v2.
The format of apps.json is documented as a TypeScript definition in code here, but that’s a little opaque, so here’s an approximation of a schema:
type AppsJsonFormat = {
config: {
[x: string]: string;
};
apps: {
[x: string]: {
id: number;
name: string;
class: "fleet" | "app" | "block";
releases: {
[x: string]: {
id: number;
services: {
[x: string]: {
id: number;
image_id: number;
image: string;
environment: {
[x: string]: string;
};
labels: {
[x: string]: string;
};
} & {
running?: boolean | undefined;
contract?: {
[x: string]: unknown;
} | undefined;
composition?: {
capAdd?: string[];
capDrop?: string[];
command?: string[] | string;
cgroupParent?: string;
devices?: string[];
dns?: string | string[];
dnsOpt?: string[];
dnsSearch?: string | string[];
tmpfs?: string | string[];
entrypoint?: string | string[];
environment?: { [envVarName: string]: string };
expose?: string[];
extraHosts?: string[];
groupAdd?: string[];
healthcheck?: ComposeHealthcheck;
image: string;
init?: string | boolean;
labels?: { [labelName: string]: string };
running?: boolean;
networkMode?: string;
networks?: string[] | ServiceNetworkDictionary;
pid?: string;
pidsLimit?: number;
ports?: string[];
securityOpt?: string[];
stopGracePeriod?: string;
stopSignal?: string;
sysctls?: { [name: string]: string };
ulimits?: {
[ulimitName: string]: number | { soft: number; hard: number };
};
usernsMode?: string;
volumes?: ServiceVolumeConfig[];
restart?: string;
cpuShares?: number;
cpuQuota?: number;
cpus?: number;
cpuset?: string;
domainname?: string;
hostname?: string;
ipc?: string;
macAddress?: string;
memLimit?: string;
memReservation?: string;
oomKillDisable?: boolean;
oomScoreAdj?: number;
privileged?: boolean;
readOnly?: boolean;
shmSize?: string;
user?: string;
workingDir?: string;
tty?: boolean;
} | undefined;
};
};
volumes: {
...;
};
networks: {
...;
};
};
};
} & {
...;
} & {
...;
};
};
} & {
pinDevice?: boolean
}
Very cool! Thank you for the insight here, and descriptions of the schema! I should be able to find a way to insert our desired environment variables into the apps.json file during our build process. Thanks for the information on where this is located, and what to change.
I’ll keep an eye on that Github issue for the upstream fix in the CLI, but I believe I should be unblocked on this for now.