Failed to preload an application with openBalena

westerweerdt · October 20, 2020, 11:57am

I was able to build an openBalena server and deploy a raspberry pi zero w application, but failed to preload my image:

ubuntu@ubuntu-desktop:/media$ balena apps
ID APP NAME  SLUG      DEVICE TYPE  ONLINE DEVICES DEVICE COUNT
1  yokosoApp yokosoapp raspberry-pi 0              0
ubuntu@ubuntu-desktop:/media$ balena preload ../balena-rpi-configured.img --app 1 --commit current --splash-image splash.jpg --debug
Building Docker preloader image. [===                     ] 12%
Step 1/7 : FROM docker:17.12.0-ce-dind
Building Docker preloader image. [======                  ] 25%
Step 2/7 : RUN apk update && apk add --no-cache python3 parted btrfs-progs util-linux sfdisk file coreutils sgdisk
 ---> Using cache
Building Docker preloader image. [=========               ] 37%
Step 3/7 : COPY ./requirements.txt /tmp/
 ---> Using cache
Building Docker preloader image. [============            ] 50%
Step 4/7 : RUN pip3 install -r /tmp/requirements.txt
 ---> Using cache
Building Docker preloader image. [===============         ] 62%
Step 5/7 : COPY ./src /usr/src/app
 ---> Using cache
Building Docker preloader image. [==================      ] 75%
Step 6/7 : WORKDIR /usr/src/app
 ---> Using cache
Building Docker preloader image. [=====================   ] 87%
Step 7/7 : CMD ["python3", "/usr/src/app/preload.py"]
 ---> Using cache
 ---> 95c91d1abee6
Successfully built 95c91d1abee6
Building Docker preloader image. [========================] 100%
| Checking that the image is a writable file
| Finding a free tcp port and getting balena settings
| Checking if the image is an edison zip archive
| Creating preloader container
| Starting preloader container
- Fetching application 1
- Cleaning up temporary files
BalenaRequestError: Request error: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /v6/application(1)</pre>
</body>
</html>


BalenaRequestError: Request error: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /v6/application(1)</pre>
</body>
</html>

    at /snapshot/versioned-source/node_modules/balena-request/build/request.js:189:17
From previous event:
    at /snapshot/versioned-source/node_modules/balena-request/build/request.js:181:62
From previous event:
    at Object.module.exports.exports.send (/snapshot/versioned-source/node_modules/balena-request/build/request.js:180:8)
    at /snapshot/versioned-source/node_modules/balena-pine/es2018/index.js:20:22
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
    at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
    at BalenaPine._request (/snapshot/versioned-source/node_modules/balena-pine/es2018/index.js:12:34)
    at BalenaPine.request (/snapshot/versioned-source/node_modules/pinejs-client-core/es2018/index.js:952:29)
    at BalenaPine.get (/snapshot/versioned-source/node_modules/pinejs-client-core/es2018/index.js:701:21)
    at /snapshot/versioned-source/node_modules/balena-preload/node_modules/balena-sdk/es2018/models/application.js:245:26
From previous event:
    at Object.get (/snapshot/versioned-source/node_modules/balena-preload/node_modules/balena-sdk/es2018/models/application.js:239:32)
    at Object.get (/snapshot/versioned-source/node_modules/balena-preload/node_modules/balena-sdk/es2018/util/callbacks.js:19:27)
    at _fetchApplication._runWithSpinner.Promise.try (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:487:49)
From previous event:
    at _fetchApplication._runWithSpinner (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:485:25)
    at Preloader._runWithSpinner (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:305:12)
    at Preloader._fetchApplication (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:481:17)
    at prepare._build.then.then.spread.then.then.then.then.then.then.then.then (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:628:19)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
    at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
    at Preloader.prepare (/snapshot/versioned-source/node_modules/balena-preload/lib/preload.js:627:6)
    at prepareAndPreload (/snapshot/versioned-source/build/actions/preload.js:173:21)
    at /snapshot/versioned-source/build/actions/preload.js:347:25
    at new Promise (<anonymous>)
    at Command.action (/snapshot/versioned-source/build/actions/preload.js:345:19)

Additional information may be available with the `--debug` flag.
For help, visit our support forums: https://forums.balena.io
For bug reports or feature requests, see: https://github.com/balena-io/balena-cli/issues/

ubuntu@ubuntu-desktop:/media$ balena version -a
balena-cli version "12.11.0"
Node.js version "10.21.0"

I even tried balena-cli with 12.2.2, but another error:

ubuntu@ubuntu-desktop:/media$ balena preload ../balena-rpi-configured.img --app yokosoApp --commit current --splash-image splash.jpg --debug
Building Docker preloader image. [===                     ] 12%
Step 1/7 : FROM docker:17.12.0-ce-dind
Building Docker preloader image. [======                  ] 25%
Step 2/7 : RUN apk update && apk add --no-cache python3 parted btrfs-progs util-linux sfdisk file coreutils sgdisk
 ---> Using cache
Building Docker preloader image. [=========               ] 37%
Step 3/7 : COPY ./requirements.txt /tmp/
 ---> Using cache
Building Docker preloader image. [============            ] 50%
Step 4/7 : RUN pip3 install -r /tmp/requirements.txt
 ---> Using cache
Building Docker preloader image. [===============         ] 62%
Step 5/7 : COPY ./src /usr/src/app
 ---> Using cache
Building Docker preloader image. [==================      ] 75%
Step 6/7 : WORKDIR /usr/src/app
 ---> Using cache
Building Docker preloader image. [=====================   ] 87%
Step 7/7 : CMD ["python3", "/usr/src/app/preload.py"]
 ---> Using cache
 ---> c0c061732ecc
Successfully built c0c061732ecc
Building Docker preloader image. [========================] 100%
| Checking that the image is a writable file
| Finding a free tcp port and getting balena settings
| Checking if the image is an edison zip archive
/ Creating preloader container
- Starting preloader container
\ Fetching application yokosoApp
\ Reading image informationWaiting for Docker to start...
- Reading image informationDocker started
\ Reading image information
- Resizing partitions and waiting for dockerd to startReplacing splash image
/ Resizing partitions and waiting for dockerd to startExpanding extended partition n°4 of /img/balena.img
Expanding logical partition n°6 of /img/balena.img
| Resizing partitions and waiting for dockerd to startResizing ext4 filesystem of partition n°6 of /img/balena.img using /dev/loop9
/ Resizing partitions and waiting for dockerd to startFile system OK
| Resizing partitions and waiting for dockerd to startWaiting for Docker to start...
\ Resizing partitions and waiting for dockerd to startDocker started

\ Cleaning up temporary files
(HTTP code 500) server error - Get https://registry.yokosotv.com/v2/: x509: certificate signed by unknown authority

Error: (HTTP code 500) server error - Get https://registry.yokosotv.com/v2/: x509: certificate signed by unknown authority
    at /snapshot/versioned-source/node_modules/dockerode/node_modules/docker-modem/lib/modem.js:257:17
    at IncomingMessage.<anonymous> (/snapshot/versioned-source/node_modules/dockerode/node_modules/docker-modem/lib/modem.js:284:9)
    at IncomingMessage.emit (events.js:203:15)
    at IncomingMessage.EventEmitter.emit (domain.js:448:20)
    at endReadableNT (_stream_readable.js:1143:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

Additional information may be available with the `--debug` flag.
For help, visit our support forums: https://forums.balena.io
For bug reports or feature requests, see: https://github.com/balena-io/balena-cli/issues/


root@ubuntu-desktop:/media# balena apps
ID APP NAME  SLUG      DEVICE TYPE  ONLINE DEVICES DEVICE COUNT
1  yokosoApp yokosoapp raspberry-pi 0              0
root@ubuntu-desktop:/media# balena version -a
balena-cli version "12.2.2"
Node.js version "10.17.0"

How can we fix this issue?
As you could see, I can see the application that was deployed in advance.

And another issue: Cannot SSH into a local development image with 12.2.2!

ubuntu@ubuntu-desktop:~$ sudo balena scan
Reporting scan results
-
  host:          yokoso.local
  address:       192.168.1.115
  dockerInfo:
    Containers:        4
    ContainersRunning: 2
    ContainersPaused:  0
    ContainersStopped: 2
    Images:            87
    Driver:            aufs
    SystemTime:        2020-10-20T11:38:43.173895986Z
    KernelVersion:     4.19.118
    OperatingSystem:   balenaOS 2.54.2+rev1
    Architecture:      armv6l
  dockerVersion:
    Version:    19.03.13-dev
    ApiVersion: 1.40
ubuntu@ubuntu-desktop:~$ sudo balena ssh 192.168.1.115 --debug
[debug] new argv=[/home/ubuntu/balena-cli/balena,/snapshot/versioned-source/bin/balena,ssh,192.168.1.115] length=4
Part of the CLI could not be loaded. This typically means your CLI install is in a broken state.
You can normally fix this by uninstalling and reinstalling the CLI.

Error: Cannot find module 'lodash/reduce'
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:634:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1351:46)
    at Function.resolveFilenameOptimized [as _resolveFilename] (/snapshot/versioned-source/node_modules/fast-boot2/index.js:57:33)
    at Function.Module._load (internal/modules/cjs/loader.js:560:25)
    at Module.require (internal/modules/cjs/loader.js:690:17)
    at Module.require (pkg/prelude/bootstrap.js:1230:31)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Promise.resolve.then (/snapshot/versioned-source/build/utils/device/ssh.js:7:55)

Additional information may be available with the `--debug` flag.
For help, visit our support forums: https://forums.balena.io
For bug reports or feature requests, see: https://github.com/balena-io/balena-cli/issues/

ubuntu@ubuntu-desktop:~$ balena version -a
balena-cli version "12.2.2"
Node.js version "10.17.0"

Thanks!

dfunckt · October 20, 2020, 12:04pm

Seems the preload module of the CLI installed on your computer uses an API version that is currently unavailable on openBalena. That’s bad module versioning from our side but regardless, can you try the 12.2.2 standalone CLI release please?

westerweerdt · October 20, 2020, 12:25pm

Yes, I have downloaded the standalone executable from here - https://github.com/balena-io/balena-cli/releases/tag/v12.2.2

dfunckt · October 20, 2020, 12:43pm

The CLI installation may have issues. I’d try running the CLI from a new local folder (ie. cd /path/to/extracted/balena-cli-standalone && ./balena preload ...) and if that fails again I’d give an older CLI version a go – maybe https://github.com/balena-io/balena-cli/releases/tag/v11.7.10

westerweerdt · October 20, 2020, 12:59pm

Yes, 11.7.10 works with ssh command. And also 12.11.0 works too!

Only 12.2.2 doesn’t work with ssh command… weird.

And preload - same with 11.7.10…

ubuntu@ubuntu-desktop:~/balena-cli_11.7.10$ ./balena preload balena-cloud-raspberry-pi-2.54.2+rev1-dev-v11.12.4.img --app yokosoApp --commit current --splash-image splash.jpg --debug
Building Docker preloader image. [========================] 100%



| Creating preloader container
/ Starting preloader container
\ Fetching application yokosoApp
- Reading image information
- Resizing partitions and waiting for dockerd to start
- Cleaning up temporary files
(HTTP code 500) server error - Get https://registry.yokosotv.com/v2/: x509: certificate signed by unknown authority

Additional information may be available by setting a DEBUG=1 environment
variable: "set DEBUG=1" on a Windows command prompt, or "export DEBUG=1"
on Linux or macOS.

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/

ubuntu@ubuntu-desktop:~/balena-cli_11.7.10$
ubuntu@ubuntu-desktop:~/balena-cli_11.7.10$ ./balena version -a
balena-cli version "11.7.10"
Node.js version "10.15.3"

The problem is that I can login and get the application list via cli.

Do I need another certification? Or a silly issue on my openBalena server?

Thanks!

dfunckt · October 20, 2020, 1:03pm

That’s good progress. Preload fails with a different error this time, but at least we got the version right. You will need to follow the Getting Started Guide for openBalena again from the section where it talks about installing the custom certificates on the system.

westerweerdt · October 20, 2020, 1:04pm

Dude, I already tried 3 times! haha

saintaardvark · October 20, 2020, 8:36pm

Hi there – can you please let us know the full steps you took to install the certificates on your system, and the steps you followed in the Getting Started Guide? Without knowing more about what steps you have taken, it will be hard for us to offer advice. You may also wish to consider re-installing from scratch, just to make sure that you’re starting from a clean slate. If you run into problems at particular steps, please let us know what you’re seeing.

All the best,
Hugh

westerweerdt · October 20, 2020, 11:28pm

Thanks for your patience!

I got this working by following this - https://github.com/balena-io/balena-cli/issues/2068

Thanks!