Qemu worker issue in autokit

Product support
1

Hi balena team,
I am planning autokit test for my custom board, so I use raspberry pi for autokit demonstration. I am running Qemu autokit test on virtualbox running Debian 12 bookworm. When I make local-test in leviathan folder, qemu autokit test shows “Couldn’t retrieve http://worker worker’s state. Querying http://worker and received StatusCodeError: 502”, here is my config:

module.exports = {
deviceType: “raspberrypi3-64”, // replace this with your device type slug. A contract must exist for this in the leviathan/core/contracts submodule.
suite: ${__dirname}/../suites/e2e,
config: {
networkWired: false,
networkWireless: false,
downloadVersion: ‘latest’,
balenaApiKey: ‘XXXXXX’, // api key is right, I omit it.
balenaApiUrl: ‘balena-cloud.com’,
organization: ‘xxxxx’, //organizaion is right, I omit it.
},
debug: {
unstable: [“Kill the device under test”],
},
image: ${__dirname}/balena.img.zip, // can also set image: false to auto download - only if the image is aailable through balena cloud
workers: [‘http://worker’], // or local ip address of the autokit (recommended)
}

by the way, leviathan-worker-1 reports an error: modprobe: FATAL: Module sg not found in directory /lib/modules/6.1.0-18-amd64, but I found sg module does exist.

By the way do I have to create a Qemu device on my balena cloud fleet?

attachment is the full log
balenaos-log.txt (21.1 KB)

Thanks

2

Hello Jason, Leviathan maintainer here, thanks for reaching out. QEMU is a great worker to run your first Leviathan test suite ideally the e2e test suite that we have. This ensures everything is setup correctly and ready to go. What custom board are you looking to start testing with autokit?

To run the QEMU tests, you can follow the Getting Started guide: QEMU worker | Leviathan Helpers

A fix in the config.js is the device type to be tested with the QEMU worker type. QEMU can only run generic device-type images. Hence, the RPi3-64 images won’t be able to run in QEMU. Replace the deviceType with genericx86-64-ext and use false in image property to get started on testing the e2e suite.

No, you won’t need to create a QEMU device in the balenaCloud fleet. Follow along with the instructions in the Getting Started guide, let us know of any feedback or questions you have, happy to help.

1 Like
3

Thanks for your reply, now it worked.

My custom board is based on I.MX8m plus, and my custom board balena yocto image has done, the device has been registered to my Balena cloud fleet, with raspberry pi device type. So my next step is pass autokit test, then I will contact Balena sales to add my device type to Balena cloud.

Do I have to pass qemu autokit test? Or I only need to pass physical autokit test?

Thanks

4

Hi @vipulgupta2048 ,
I’m trying to use QEMU worker with balena-staging.com as I’m scared to point Leviathan to our production environment.
I followed the start guide and made sure I have Docker, node and npm.
I also made sure my balena vars are valid by issuing a fleet list API call using curl. It looks working.

leviathan.run.txt (21.5 KB)

This is my config.json:

module.exports = [{
                deviceType: "genericx86-64-ext",
                suite: `${__dirname}/../suites/os`,
                config: {
                        networkWired: false,
                        networkWireless: false,
                        balenaApiKey: process.env.BALENACLOUD_API_KEY,
                        balenaApiUrl: 'balena-staging.com',
                        organization: process.env.BALENACLOUD_ORG,
                },
                image: false,
                workers: ['http://worker']
        }]

and this is the output I’m getting:

leviathan-worker-1  | Worker http listening on port 80
leviathan-client-1  | Logged in with <MYUSER>'s account on balena-staging.com using balenaSDK
leviathan-client-1  | No workers found...NO TESTS RAN
leviathan-client-1  | Exiting with 1, client = 1, children: 
leviathan-client-1  | *******  Final Test Result of Leviathan Run  *******
leviathan-client-1  | No tests ran, check for errors .. Quitting
leviathan-client-1  | /usr/src/app/lib/config-validator.js:22
leviathan-client-1  |     if (config.debug.dev === true ) {
leviathan-client-1  |                      ^
leviathan-client-1  | 
leviathan-client-1  | TypeError: Cannot read properties of undefined (reading 'dev')
leviathan-client-1  |     at testConfig (/usr/src/app/lib/config-validator.js:22:22)
leviathan-client-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
leviathan-client-1  |     at async /usr/src/app/bin/multi-client.js:314:26
leviathan-client-1  | 
leviathan-client-1  | Node.js v18.16.0
leviathan-client-1 exited with code 1

Should this work?

Thank you,
-Sergey.

5

Ok, apparently the debug section is not optional in config.js.
After adding it and changing suite name to e2e the tests fail with being unable to netbtoot.
dut-serial-worker-e2e.log (4.0 KB)

6

@jason You would need to pass both tests as mentioned in the Getting Started guide to make sure everything works with your device type.

7

If I set image to false, the suite fails to download an image:

leviathan-client-1  | [2025-03-26T10:50:34.354Z][worker-e2e] 		 Power cycling the DUT
leviathan-client-1  | [2025-03-26T10:50:34.354Z][worker-e2e] 		 Is the DUT reachable?
leviathan-client-1  | [2025-03-26T10:50:34.354Z][worker-e2e] 	 Serial test
leviathan-client-1  | [2025-03-26T10:50:34.354Z][worker-e2e] 		 Recording DUT serial output
leviathan-client-1  | [2025-03-26T10:50:34.402Z][worker-e2e] TAP version 13
leviathan-client-1  | [2025-03-26T10:50:34.727Z][worker-e2e] (I added this print) this.suite.options.balenaOS: {"config":{"uuid":"7708a2b7a061c099226aef2fbf255672"},"network":{"wired":{"nat":true}}}
leviathan-client-1  | [2025-03-26T10:50:34.734Z][worker-e2e] # Subtest: Testbot Diagnostics
leviathan-client-1  | [2025-03-26T10:50:34.735Z][worker-e2e]     not ok 1 - Cannot read property 'version' of undefined
leviathan-client-1  | [2025-03-26T10:50:34.735Z][worker-e2e]       ---
leviathan-client-1  |       stack: |
leviathan-client-1  | [2025-03-26T10:50:34.735Z][worker-e2e]         Proxy.run (/data/suite/suite.js:295:61)
leviathan-client-1  |       at:
leviathan-client-1  | [2025-03-26T10:50:34.735Z][worker-e2e]         line: 295
leviathan-client-1  |         column: 61
leviathan-client-1  |         file: /data/suite/suite.js
leviathan-client-1  | [2025-03-26T10:50:34.735Z][worker-e2e]         function: Proxy.run
leviathan-client-1  |       type: TypeError
leviathan-client-1  |       test: Testbot Diagnostics
leviathan-client-1  | [2025-03-26T10:50:34.736Z][worker-e2e]       source: >
leviathan-client-1  | [2025-03-26T10:50:34.736Z][worker-e2e]         							.get()
leviathan-client-1  |         								.cloud.fetchOS(this.suite.options.balenaOS.download.version, this.suite.deviceType.slug)}`
leviathan-client-1  |         ------------------------------------------------------------^
leviathan-client-1  | [2025-03-26T10:50:34.736Z][worker-e2e]         							: undefined,

If I provide an image downloaded from dashboard (qemux86-64-2.83.18+rev5-dev-v12.10.3.img.gz), it appears to fail booting:

BdsDxe: failed to load Boot0001 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0): Not Found
BdsDxe: failed to load Boot0002 "UEFI QEMU QEMU USB HARDDRIVE 1-0000:00:02.0-1" from PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0): Not Found

>>Start PXE over IPv4.
  PXE-E16: No valid offer received.
BdsDxe: failed to load Boot0003 "UEFI PXEv4 (MAC:5254009EA7CB)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5254009EA7CB,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0): Not Found

>>Start PXE over IPv6.
  PXE-E16: No valid offer received.
BdsDxe: failed to load Boot0004 "UEFI PXEv6 (MAC:5254009EA7CB)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5254009EA7CB,0x1)/IPv6(0000:0000:0000:0000:0000:0000:0000:0000,0x0,Static,0000:0000:0000:0000:0000:0000:0000:0000,0x40,0000:0000:0000:0000:0000:0000:0000:0000): Not Found

>>Start HTTP Boot over IPv4.....
  Error: Could not retrieve NBP file size from HTTP server.

  Error: Server response timeout.
BdsDxe: failed to load Boot0005 "UEFI HTTPv4 (MAC:5254009EA7CB)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5254009EA7CB,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found

>>Start HTTP Boot over IPv6.
  Error: Could not retrieve NBP file size from HTTP server.

  Error: Unexpected network error.
BdsDxe: failed to load Boot0006 "UEFI HTTPv6 (MAC:5254009EA7CB)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5254009EA7CB,0x1)/IPv6(0000:0000:0000:0000:0000:0000:0000:0000,0x0,Static,0000:0000:0000:0000:0000:0000:0000:0000,0x40,0000:0000:0000:0000:0000:0000:0000:0000)/Uri(): Not Found
BdsDxe: loading Boot0007 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
BdsDxe: starting Boot0007 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)

UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
Mapping table
      FS0: Alias(s):HD1a0b:;BLK2:
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(1,MBR,0x00000000,0x2000,0x14000)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)
     BLK1: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)
     BLK3: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(2,MBR,0x00000000,0x16000,0xA0000)
     BLK4: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(3,MBR,0x00000000,0xB6000,0xA0000)
     BLK5: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(4,MBR,0x00000000,0x156000,0x410000)
     BLK6: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(4,MBR,0x00000000,0x156000,0x410000)/HD(1,MBR,0x00000000,0x158000,0xA000)
     BLK7: Alias(s):
          PciRoot(0x0)/Pci(0x2,0x0)/USB(0x0,0x0)/HD(4,MBR,0x00000000,0x156000,0x410000)/HD(2,MBR,0x00000000,0x164000,0x402000)




Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell>
8

Poking around and testing what QEMU command line would let the VM boot I deleted following arguments:

-nodefaults
firmwareArgs
internalStorage

this made tests complete with:

leviathan-client-1  | [
leviathan-client-1  |   {
leviathan-client-1  |     suite: 'Testbot Diagnostics',
leviathan-client-1  |     stats: { tests: 5, ran: 2, skipped: 3, passed: 2, failed: 0 },
leviathan-client-1  |     tests: {
leviathan-client-1  |       'Kill the device under test': 'skipped',
leviathan-client-1  |       'Flashing an image': 'passed',
leviathan-client-1  |       'Power cycling the DUT': 'skipped',
leviathan-client-1  |       'Is the DUT reachable?': 'passed',
leviathan-client-1  |       'Recording DUT serial output': 'skipped'
leviathan-client-1  |     },
leviathan-client-1  |     dateTime: 'Thu Mar 27 2025 15:33:32 GMT+0000 (Coordinated Universal Time)'
leviathan-client-1  |   }
leviathan-client-1  | ]