Trying to run balenaOS on Arm's Virtual Hardware service

Product support balenaOS
1

Arm offers a Virtual Hardware service (AVH) that emulates devices in the cloud, including a RaspberryPi 4. You can use either a pre-packaged firmware to boot these virtual devices, or provide your own.

I am attempting to make a custom firmware for balenaOS, using the RaspberryPi 4 images provided. I found this guide and script for converting a RaspberryPi image into a custom AVH firmware.

I ran the script on the balenaOS image file, but it didn’t like running zcat boot/kernel8.img > kernel saying it wasn’t a gzip’d file. so I changed that line to just cp boot/kernel8.img kernel and the script finished happily.

However, after uploading the resulting zip file to AVH, when it started to boot the virtual device it gave the error in the screenshot below:

Screenshot 2023-03-01 at 2.38.52 PM
Screenshot 2023-03-01 at 2.38.52 PM1188×740 18.8 KB

Since the only thing I changed was that zcat/cp line, I’m wondering if the kernel8.img file in the balenaOS image is using some other kind of compression other than gzip, but running file on it says it’s a PCX (graphics file) format.

So, finally getting to my actual questions:

  1. Is the kernel8.img file compressed? If so, what compression is it using?
  2. Is there anything different between the balenaOS image and a standard Raspbian OS image that I should be taking into consideration here?
1 Like
6

Hello @mhall119 welcome to the balena community!

I pinged internally the balenaOS team to see how we can help you more!

We will keep you updated here!

7

Hi Michael. The balenaOS kernel is a zImage that embeds an initramfs. It might be that AVH requires the kernel and the initramfs separately. I would suggest you build the repository yourself and use the standalone kernel and initramfs images.

1 Like
8

Also, balenaOS uses u-boot, so kernel8.img is not the kernel image, but the u-boot image.

10

I’m trying to build the repository, but I’m getting the following error running ./balena-yocto-scripts/build/barys:

[000000003][ERROR]Please install the 'jq' package before running this script
11

Okay, I’ve gotten around this in two ways:

  1. Installed jq command in my Ubuntu environment using multipass on my Mac
  2. Installed jq directly on my Mac using Homebrew

#1 gives me the following errors:

17125 11:53:04.115673 --- Starting bitbake server pid 17125 at 2023-03-03 11:53:04.115630 ---
Traceback (most recent call last):
  File "/home/ubuntu/Home/Projects/balena/balena-raspberrypi/layers/poky/bitbake/bin/bitbake-server", line 52, in <module>
    bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface)
  File "/home/ubuntu/Home/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/server/process.py", line 541, in execServer
    sock.bind(os.path.basename(sockname))
PermissionError: [Errno 1] Operation not permitted
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/ubuntu/Home/Projects/balena/balena-raspberrypi/build/bitbake-cookerdaemon.log' mode='a+' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=3 mode='w' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>

#2 gives me the following errors:

Traceback (most recent call last):
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/bin/bitbake", line 37, in <module>
    sys.exit(bitbake_main(BitBakeConfigParameters(sys.argv),
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/main.py", line 378, in bitbake_main
    return ui_module.main(server_connection.connection, server_connection.events,
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/ui/knotty.py", line 415, in main
    params.updateToServer(server, os.environ.copy())
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/cookerdata.py", line 75, in updateToServer
    raise Exception("Unable to update the server configuration with local parameters: %s" % error)
Exception: Unable to update the server configuration with local parameters: Traceback (most recent call last):
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/command.py", line 90, in runCommand
    result = command_method(self, commandline)
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/command.py", line 286, in updateConfig
    command.cooker.updateConfigOpts(options, environment, cmdline)
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/cooker.py", line 535, in updateConfigOpts
    self.reset()
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/cooker.py", line 1774, in reset
    self.initConfigurationData()
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/cooker.py", line 360, in initConfigurationData
    self.setupConfigWatcher()
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/bb/cooker.py", line 235, in setupConfigWatcher
    self.configwatcher = pyinotify.WatchManager()
  File "/Users/michal02/Projects/balena/balena-raspberrypi/layers/poky/bitbake/lib/pyinotify.py", line 1723, in __init__
    raise InotifyBindingNotFoundError()
pyinotify.InotifyBindingNotFoundError: Couldn't find any inotify binding
15

Okay, I have the repo building in an Ubuntu VM now, but I keep running out of space. I have 60GB allocated to the VM but it still runs out of space.

How much disk space is needed to build just the raspberrypi4-64 image?

16

Hi, probably around 150GB, unless you use the --rm-work barys option to use the rm_work class to remove intermediate object.

1 Like
18

Hello @mhall119 did you try with 150GB or more or ith the --rm-work that my colleague recommended?

Let us know if this works!

19

@mpous @alexgg I tried with the --rm-work flag and after several attempts (do_fetch on raspberrypi/linux kept timing out) was able to get it almost to the point of building.

The latest problem is coming from go-native in do_configure saying unknown architecture: aarch64 , screenshot attached.

Screenshot 2023-03-08 at 2.06.23 PM
Screenshot 2023-03-08 at 2.06.23 PM2556×1608 259 KB

1 Like
21

Hi Michael,

Is this balenaOS source tree modified at all? The aarch64 target architecture should be mapped to go’s equivalent of arm64 in layers/poky/meta/classes/goarch.bbclass:63 in the function go_map_arch. It looks like that didn’t work right for some reason.

22

@mhall119,
Just a quick direct ping to make sure you saw Joseph’s reply above. We’re internally pretty excited to see this working, so here to help. :slight_smile:

23

No, I did a fresh git clone and have been trying to build from there. Git commit b6ff0d9985071425de6b54701f1109ac4b3a0a75

24

I’ve also got internal AVH support interested in this now. This was the latest I got from them:

The Raspberry Pi BalenaOS image utilizes a single-stage boot process, where AVH is expecting a second-stage boot process. We are working on support for these types of images, but it will take some time to implement.

For now I’m leaving it with them, and will come back to it once they have a solution.

1 Like
25

Thanks for sharing this @mhall119

Keep us updated on the AVH development!

28

Hey @mpous, just wanted to let you know that Package Custom U-Boot Firmware for AVH | Arm AVH Help Center was published and shows how to run Balena OS on AVH, I’m trying it out now!

1 Like
29

Happy to read this @mhall119 did you succeed testing this?

I will put this on my to-do list :slight_smile:

30

My Learning Path for Arm’s Developer Hub is up: Deploy IoT apps using Balena Cloud and Arm Virtual Hardware | Arm Learning Paths

1 Like
31

This is great @mhall119 thanks for sharing!