We are developing a custom flasher image in Yocto (3.2.1) which serves as a “master key” flasher for our app, where each time it flashes the hardware (in this case a CM4 module) it uses balena-cli to login, register the device with our app in Balena, inject the newly generated config.json into a copy of the BalenaOS image, and flash that image to the EMMC on the CM4. The hardware is arm64 so we require an arm64 version of balena-cli, which has proven to not be so straightforward. The way we are using balena-cli on the flasher today is using docker to run the klutchell/balena-cli image, but this is a lot of unnecessary overhead and we would like to natively run balena-cli on the CM4, compiling and deploying it via Yocto similar to all of the other balena recipes.
The issue is that when doing so, it generates errors, which look like they have to do with the shrinkwrap file. When I create a new recipe with the following bb file:
For the npmsw reference I am using the npm-shrinkwrap file from the 12.39.1 release. When building an image with this layer, first the fetch stage generates errors around the formatting of the git references in the file, for example:
And a number of other commit references need to be updated as well. Then when I get it through the fetch stage, it fails on compile, which also seems to be related to version references. Does anyone know if the shrinkwrap file is up to date / if not how can I get a working shrinkwrap file?
Does anyone know if the shrinkwrap file is up to date / if not how can I get a working shrinkwrap file?
The balena CLI’s npm-shrinkwrap.json file is always used when the CLI is installed via npm install. It is “always up to date” and, if it was broken, the CLI could not be installed. These are the lines containing the quoted values for v12.39.1:
I’ve just tested CLI v12.39.1 to confirm. As far as I can tell, it is working!
needs to be modified to: "git+https:// […]
Could it be that the Yocto build is using an old version of npm that doesn’t understand the github package notation in the format github:<githubname>/<githubrepo>[#<commit-ish>]? This format is documented at: npm-install | npm Docs
By the way, the balena CLI currently requires a Node.js version >= 10.20.0 and < 13.0.0:
The versions of npm that ship with some key versions of Node.js are:
$ nvm use 10.20.0
Now using node v10.20.0 (npm v6.14.4)
$ nvm use 12.0.0
Now using node v12.0.0 (npm v6.9.0)
$ nvm use 12
Now using node v12.20.2 (npm v6.14.11)
The output above effectively means that the minimum tested version of npm is 6.9.0, and the latest tested version of npm is 6.14.11.
More generally, an interesting question would be “how do I build the balena CLI using a Yocto recipe”? To be honest, I don’t know the answer! But at least I know that the the balena CLI can be built on ARM using Linux distributions like Debian and Ubuntu, and using nvm (Node Version Manager) to install Node.js and npm, along the lines of:
$ apt-get update && apt-get install -y curl git python g++ make
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash
$ nvm install 12
$ npm install balena-cli -g --production --unsafe-perm
$ which balena
/usr/local/nvm/versions/node/v12.20.2/bin/balena
$ balena version -a
balena-cli version "12.40.0"
Node.js version "12.20.2"
Hey there @drcnyc, I tried reproducing the issue in my yocto environment and found similar issues, but the npm-shrinkrap file is correct for most other environments as Paolo showed.
So the best workaround for now may be the following in your yocto copy of npm-shrinkwrap.json:
replace github: prefixes with git+https: as you’ve already done
for modules using commits from a branch not merged to master, append ;branch=foobar between the url and the commit hash and avoid changing the commit itself
@pdcastro@klutchell thank you both for the detailed follow-up. I came to the same conclusion re: the yocto fetcher, and actually went down the opposite path of modifying the (forbidden to modify) poky layer to get it to understand git+ links. Basically ends up at the same place - it was able to build after that. But I then undid the changes, because obviously modifying poky is not a good idea. To make it usable for your yocto universe you might want to consider making the git references conform to the older standard, which shouldn’t break anything for the newer clients.
For my purpose (having the flasher image register the target device) I was able to accomplish everything I needed via the balena API in the end, which was a much lighter weight solution than having to build the CLI into the flasher, so we went with that instead. Kudos to you guys - you clearly have a very comprehensive product that has given us so much flexibility to be able to do this multiple ways.
Thanks for the kind words @drcnyc! Good to know that the balena API was good enough for you. I have pinged klutchell so that we get the feedback you gave about the git references to the right engineers