balena-cli compilation issues in Yocto

We are developing a custom flasher image in Yocto (3.2.1) which serves as a “master key” flasher for our app, where each time it flashes the hardware (in this case a CM4 module) it uses balena-cli to login, register the device with our app in Balena, inject the newly generated config.json into a copy of the BalenaOS image, and flash that image to the EMMC on the CM4. The hardware is arm64 so we require an arm64 version of balena-cli, which has proven to not be so straightforward. The way we are using balena-cli on the flasher today is using docker to run the klutchell/balena-cli image, but this is a lot of unnecessary overhead and we would like to natively run balena-cli on the CM4, compiling and deploying it via Yocto similar to all of the other balena recipes.

The issue is that when doing so, it generates errors, which look like they have to do with the shrinkwrap file. When I create a new recipe with the following bb file:

SRC_URI = " \
    npm://registry.npmjs.org;package=balena-cli;version=12.39.1 \
    npmsw://${THISDIR}/npm-shrinkwrap.json \
"
S = "${WORKDIR}/npm"
inherit npm

For the npmsw reference I am using the npm-shrinkwrap file from the 12.39.1 release. When building an image with this layer, first the fetch stage generates errors around the formatting of the git references in the file, for example:

github:balena-io-modules/unbzip2-stream#942fc218013c14adab01cf693b0500cf6ac83193",

needs to be modified to:

"git+https://github.com/resin-io-modules/unbzip2-stream.git#942fc218013c14adab01cf693b0500cf6ac83193",

And a number of other commit references need to be updated as well. Then when I get it through the fetch stage, it fails on compile, which also seems to be related to version references. Does anyone know if the shrinkwrap file is up to date / if not how can I get a working shrinkwrap file?

Does anyone know if the shrinkwrap file is up to date / if not how can I get a working shrinkwrap file?

The balena CLI’s npm-shrinkwrap.json file is always used when the CLI is installed via npm install. It is “always up to date” and, if it was broken, the CLI could not be installed. These are the lines containing the quoted values for v12.39.1:

I’ve just tested CLI v12.39.1 to confirm. As far as I can tell, it is working! :slight_smile:

needs to be modified to: "git+https:// […]

Could it be that the Yocto build is using an old version of npm that doesn’t understand the github package notation in the format github:<githubname>/<githubrepo>[#<commit-ish>]? This format is documented at: https://docs.npmjs.com/cli/v6/commands/npm-install

By the way, the balena CLI currently requires a Node.js version >= 10.20.0 and < 13.0.0:

The versions of npm that ship with some key versions of Node.js are:

$ nvm use 10.20.0
Now using node v10.20.0 (npm v6.14.4)

$ nvm use 12.0.0
Now using node v12.0.0 (npm v6.9.0)

$ nvm use 12
Now using node v12.20.2 (npm v6.14.11)

The output above effectively means that the minimum tested version of npm is 6.9.0, and the latest tested version of npm is 6.14.11.

More generally, an interesting question would be “how do I build the balena CLI using a Yocto recipe”? To be honest, I don’t know the answer! But at least I know that the the balena CLI can be built on ARM using Linux distributions like Debian and Ubuntu, and using nvm (Node Version Manager) to install Node.js and npm, along the lines of:

$ apt-get update && apt-get install -y curl git python g++ make
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash
$ nvm install 12
$ npm install balena-cli -g --production --unsafe-perm
$ which balena
  /usr/local/nvm/versions/node/v12.20.2/bin/balena
$ balena version -a
  balena-cli version "12.40.0"
  Node.js version "12.20.2"

(More detailed steps can be found in this sample Dockerfile.)

Let us know if this information helps, and if you can check the version of npm and update it.

Hey there @drcnyc, I tried reproducing the issue in my yocto environment and found similar issues, but the npm-shrinkrap file is correct for most other environments as Paolo showed.

The problem is that bitbake has a custom npmsw handler for these files and it doesn’t support things like the github: prefix or from: repos that point to branches not merged to master. Check out the source here:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/npmsw.py?id=507a47a4e5077d5f8f76d9629be6b871dfd8eb90#n125

So the best workaround for now may be the following in your yocto copy of npm-shrinkwrap.json:

  1. replace github: prefixes with git+https: as you’ve already done
  2. for modules using commits from a branch not merged to master, append ;branch=foobar between the url and the commit hash and avoid changing the commit itself

eg.

git+https://github.com/balena-io-modules/bonjour.git;branch=fixed-mdns#e018851dc823b4b3f670f658f71d0c1c7f3e637c
git+https://github.com/balena-io-modules/multicast-dns.git;branch=listen-on-all-interfaces#a15c63464eb43e8925b187ed5cb9de6892e8aacc
git+https://github.com/balena-io-modules/resin-discoverable-services.git;branch=find-on-all-interfaces#afca9e4700ec5ef82aa897f14bd5a46f06518061
git+https://github.com/balena-io-modules/unbzip2-stream.git;branch=core-streams#942fc218013c14adab01cf693b0500cf6ac83193

@pdcastro @klutchell thank you both for the detailed follow-up. I came to the same conclusion re: the yocto fetcher, and actually went down the opposite path of modifying the (forbidden to modify) poky layer to get it to understand git+ links. Basically ends up at the same place - it was able to build after that. But I then undid the changes, because obviously modifying poky is not a good idea. To make it usable for your yocto universe you might want to consider making the git references conform to the older standard, which shouldn’t break anything for the newer clients.

For my purpose (having the flasher image register the target device) I was able to accomplish everything I needed via the balena API in the end, which was a much lighter weight solution than having to build the CLI into the flasher, so we went with that instead. Kudos to you guys - you clearly have a very comprehensive product that has given us so much flexibility to be able to do this multiple ways.

Thanks for the kind words @drcnyc! Good to know that the balena API was good enough for you. I have pinged klutchell so that we get the feedback you gave about the git references to the right engineers :slight_smile:

For additional visibility of this issue and hopefully a fix, I’ve filed the following issues in the Yocto Project Bugzilla and the balena CLI repo: