# CM4 Secure Boot **URL:** \ **Category:** balenaOS\ **Created:** [July 13, 2022, 12:48pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620 "2022-07-13T12:48:06Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![kwaremburg](https://avatars.discourse-cdn.com/v4/letter/k/87869e/32.png) [@kwaremburg](https://forums.balena.io/u/kwaremburg)\ **Post date:** [July 13, 2022, 12:48pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/1 "2022-07-13T12:48:06Z")
Hi, we are at the stage in our product development where we need to lock down the the CM4 to protect company IP. The official RPi usbboot repo has a good tutorial on how to enable and run secure boot. The [usbboot/secure-boot-example at master · raspberrypi/usbboot · GitHub](https://github.com/raspberrypi/usbboot/tree/master/secure-boot-example) provides a very basic boot.img. I’ve attempted to follow the tutorial and package up a boot.img of a Balena OS install but without fail run into a Firmware not found error on boot. I’m assuming I’m not the only one that desires to use the secure boot functionality of the CM4 alongside Balena OS so I’m curious if anyone else has already done this and gotten it to work. ---
**Author:** ![anujdeshpande](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/anujdeshpande/32/3155_2.png) [@anujdeshpande](https://forums.balena.io/u/anujdeshpande)\ **Post date:** [July 13, 2022, 1:14pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/3 "2022-07-13T13:14:41Z")
Hi I have pinged the engineer who is working on secure boot for balena devices. We’ll get back with an answer soon! ---
**Author:** ![mtoman](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/mtoman/32/3599_2.png) [@mtoman](https://forums.balena.io/u/mtoman)\ **Post date:** [July 15, 2022, 11:32am UTC](https://forums.balena.io/t/cm4-secure-boot/359620/4 "2022-07-15T11:32:50Z")
Hi, our secure boot efforts are currently focused on x86 device types. Once that is finished, we are likely to look into RPi 4 and CM4 but at this moment it is not actively being worked on. It should be possible to make it work following the turorial though - have you been able to make it work with the minimal `boot.img` provided as example? If that works for you, could you describe a little more how you generated the `boot.img` for balenaOS? Are you booting off an SD card or something else? It might also be a good idea to enable `BOOT_UART` in the bootloader: [Raspberry Pi Documentation - Raspberry Pi Hardware](https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#BOOT_UART) That way you would get more debug messages via the serial console. ---
**Author:** ![kwaremburg](https://avatars.discourse-cdn.com/v4/letter/k/87869e/32.png) [@kwaremburg](https://forums.balena.io/u/kwaremburg)\ **Post date:** [July 15, 2022, 3:47pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/6 "2022-07-15T15:47:28Z")
I’m able to get the minimal boot.img work. I’m booting off the EMMC of a CM4 module. I might be wrong as I don’t have a lot of experience around how linux boots and loads the kernel but their example loads the FS into RAM and then boots from there. This is what I’m currently doing. Mount the `resin-boot` partition. Copy contents of that parition to a folder called `secure-boot-files` My hunch is that I’m messing something up in this step. I noticed that the Balena cmdline.txt doesn’t have a root specified. But Balena uses a boot script? `boot.scr`. which pointing it to that might also be my problem. > #### Set the kernel root device > > Since the boot file-system for the firmware is now in a signed disk image the OS cannot write to this. Therefore, any changes to `cmdline.txt` must be made before the `boot.img` file is signed. > > - Verify that `cmdline.txt` in `secure-boot-files` points to the correct UUID for the root file-system. Alternatively, for testing, you can specify the root device name e.g. `root=/dev/mmcblk0p2`. > - Remove `init-resize.sh` from `cmdline.txt` Specifically setting the root properly in cmdline.txt Then I continue on with the tutorial and make the `.img` > sudo tools/make-boot-image -d secure-boot-files -o boot.img -b cm4 -a 64 Then I sign it. > tools/rpi-eeprom-digest -i boot.img -o boot.sig -k “${KEY\_FILE}”` I then copy `boot.img` and `boot.sig` to the `resin-boot` partition and try to boot it. > RPi: BOOTLOADER release VERSION:507b2360 DATE: 2022/04/26 TIME: 11:24:28 BOOTMODE: 0x00000006 part: 0 BUILD\_TIMESTAMP=1650968668 0x93e9c5a3 0x00d03140 0x0006a297 > PM\_RSTS: 0x00001000 > part 00000000 reset\_info 00000000 > uSD voltage 3.3V > Initialising SDRAM ‘Micron’ 32Gb x2 total-size: 64 Gbit 3200 > DDR 3200 1 0 64 152 > > Boot mode: SD (01) order f254 > SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > SD HOST: 250000000 CTL0: 0x00000f00 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > EMMC > SD retry 1 oc 0 > SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > OCR c0ff8080 [0] > CID: 00150100424a54443452030dadb69e58 > SD HOST: 250000000 CTL0: 0x00000f04 BUS: 25000000 Hz actual: 25000000 HZ div: 10 (5) status: 0x1fff0000 delay: 4 > SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 > MBR: 0x00002000, 81920 type: 0x0c > MBR: 0x00016000, 638976 type: 0x83 > MBR: 0x000b2000, 638976 type: 0x83 > MBR: 0x0014e000,59703296 type: 0x0f > Trying partition: 0 > type: 32 lba: 8192 oem: ‘mkfs.fat’ volume: ’ resin-boot ’ > rsc 32 fat-sectors 630 c-count 80628 c-size 1 > root dir cluster 2 sectors 0 entries 0 > FAT32 clusters 80628 > secure-boot > Loading boot.img … > SIG boot.sig 4daaf788365e038e8ac581ff4d70f908cf6603d22924c43836a9a2cef1c92dde 1657563380 > Verifying > RSA verify > rsa-verify pass (0x0) > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > Trying partition: 0 > type: 12 lba: 0 oem: ‘mkfs.fat’ volume: ’ V ^ ’ > rsc 1 fat-sectors 12 c-count 3576 c-size 4 > root dir cluster 1 sectors 16 entries 256 > FAT12 clusters 3576 > Read config.txt bytes 216 hnd 0x26 > Firmware not found > ERROR: 4 > Boot mode: USB-MSD (04) order f25 > PCIe timeout: 0x0000008f > USB xHC init failed > Boot mode: BCM-USB-MSD (05) order f2 > XHCI-STOP > xHC ver: 272 HCS: 01000140 0c0000f1 07ff000a HCC: 0220fe65 > xHC ports 1 slots 64 intrs 1 ---
**Author:** ![mtoman](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/mtoman/32/3599_2.png) [@mtoman](https://forums.balena.io/u/mtoman)\ **Post date:** [July 18, 2022, 12:01pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/9 "2022-07-18T12:01:50Z")
Hi, thanks for the detailed description, it is perfect. It looks like you did everything correctly, the signature check passes yet it looks like the `boot.img` is missing some file(s) - the “Firmware not found” message is coming from the bootloader which means balenaOS has not yet started. `make-boot-images` performs a firmware cleanup by board type, would you be able to inspect your `boot.img` as described in the tutorial: [GitHub - raspberrypi/usbboot: Raspberry Pi USB booting code, moved from tools repository](https://github.com/raspberrypi/usbboot#verify-the-boot-image) to check which files actually ended up being packaged? Maybe even compare them to the original contents of `resin-boot`. The bullets from “Set the kernel root device” do not apply to balenaOS - we do not specify the root device on the command line because the root device is determined dynamically between `rootA` and `rootB`. The initrd, which should be packaged with the kernel, takes care of that and at this point I see no obvious reason for it not to work in secure boot mode. ---
**Author:** ![kwaremburg](https://avatars.discourse-cdn.com/v4/letter/k/87869e/32.png) [@kwaremburg](https://forums.balena.io/u/kwaremburg)\ **Post date:** [July 18, 2022, 3:18pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/11 "2022-07-18T15:18:41Z")
Hi, thanks again for the help. I left it out of what I’ve tried but I did verify the root dir of the image contents matched the `resin-boot` partition. Originally it was stripping out the 64 bit kernel because the example was for a Pi4 and it wasn’t obvious that I need to specify `-a 64`. Once I worked through that problem the contents of the `boot.img` appeared to match the `resin-boot` partition. I will attempt it again and verify the contents more closely. My gut tells me all the files are present but something isn’t pointing it to the correct files. When it says `Firmware not found` `error 4` Is that a specific file that it can’t find or id it more generically saying 1 or more files in a list of N required files can’t be found? ---
**Author:** ![kwaremburg](https://avatars.discourse-cdn.com/v4/letter/k/87869e/32.png) [@kwaremburg](https://forums.balena.io/u/kwaremburg)\ **Post date:** [July 18, 2022, 4:29pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/12 "2022-07-18T16:29:47Z")
I verified the contents and discovered that if I leave the `-b cm4` in the `make-boot-image` command then it “works”. Running with `-b cm4` strips the `fixup4cd.elf` and `start4cd.elf`. It isn’t clear to me how it selects which `fixup*.elf` and `start*.elf` to use but it appears it wanted the `cd.*` set. I am now getting significantly further into the boot process now but running into a filesystem error now. I’m curious if you have any thoughts on this as it appears to be related to the multiple partition structure that Balena OS uses. > RPi: BOOTLOADER release VERSION:507b2360 DATE: 2022/04/26 TIME: 11:24:28 BOOTMODE: 0x00000006 part: 0 BUILD\_TIMESTAMP=1650968668 0x93e9c5a3 0x00d03140 0x0006a297 > PM\_RSTS: 0x00001000 > part 00000000 reset\_info 00000000 > uSD voltage 3.3V > Initialising SDRAM ‘Micron’ 32Gb x2 total-size: 64 Gbit 3200 > DDR 3200 1 0 64 152 > > Boot mode: SD (01) order f254 > SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > SD HOST: 250000000 CTL0: 0x00000f00 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > EMMC > SD retry 1 oc 0 > SD HOST: 250000000 CTL0: 0x00000000 BUS: 200000 Hz actual: 200000 HZ div: 1250 (625) status: 0x1fff0000 delay: 540 > OCR c0ff8080 [0] > CID: 00150100424a54443452030dadb69e58 > SD HOST: 250000000 CTL0: 0x00000f04 BUS: 25000000 Hz actual: 25000000 HZ div: 10 (5) status: 0x1fff0000 delay: 4 > SD HOST: 250000000 CTL0: 0x00000f04 BUS: 50000000 Hz actual: 41666666 HZ div: 6 (3) status: 0x1fff0000 delay: 2 > MBR: 0x00002000, 81920 type: 0x0c > MBR: 0x00016000, 655360 type: 0x83 > MBR: 0x000b6000, 655360 type: 0x83 > MBR: 0x00156000, 458752 type: 0x0f > Trying partition: 0 > type: 32 lba: 8192 oem: ‘mkfs.fat’ volume: ’ resin-boot ’ > rsc 32 fat-sectors 630 c-count 80628 c-size 1 > root dir cluster 2 sectors 0 entries 0 > FAT32 clusters 80628 > secure-boot > Loading boot.img … > SIG boot.sig da364247609a1c5bb4cfa1aae3a30a25e05bb61890f91c72fe4c44493567ed19 1658161151 > Verifying > RSA verify > rsa-verify pass (0x0) > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > MBR: 0x00000000, 0 type: 0x00 > Trying partition: 0 > type: 16 lba: 0 oem: ‘mkfs.fat’ volume: ’ V ^ ’ > rsc 1 fat-sectors 16 c-count 4087 c-size 4 > root dir cluster 1 sectors 16 entries 256 > FAT16 clusters 4087 > Read config.txt bytes 36396 hnd 0x59 > Read start4cd.elf bytes 795708 hnd 0x71c > Read fixup4cd.dat bytes 3183 hnd 0x70 > Firmware: a48d332c35ee1c1c1ab433228e23317f62dcc5fb Apr 21 2021 15:48:07 > 0x00d03140 0x00000000 0x000003ff > MEM GPU: 16 ARM: 998 TOTAL: 1014 > Starting start4cd.elf @ 0xff000200 partition 0 > + > > U-Boot 2021.04-rc3 (Mar 03 2021 - 15:10:34 +0000) > > DRAM: 7.9 GiB > RPI Compute Module 4 (0xd03140) > MMC: mmcnr@7e300000: 1, mmc@7e340000: 0 > Loading Environment from FAT… \*\*\* Warning - bad CRC, using default environment > > In: serial > Out: serial > Er: serial > Net: eth0: ethernet@7d580000 > PCIe BRCM: link down > starting USB… > No working controllers found > Hit any key to stop autoboot: 2 1 0 > switch to partitons #0, OK > mmc0(part 0) is current device > Scanning mmc 0:1… > libfdt fdt\_check\_header(): FDT\_ERR\_BADMAGIC > 7[r[999;999H[6n8Card did not respon to voltage select! : -110 > Scanning disk mmcnr@7e300000.blk… > Disk mmcnr@7e300000.blk not ready > Scanning disk mmc@7340000.blk… > \*\* Unrecognized filesystem type \*\* > Found 7 disks > No EFI system partition > BootOrder not defined > EFI boot manager: Cannot load any image > Card did nt respond to voltage select! : -110 > starting USB… > No working controllers found > USB is stopped. Please issue ‘usb strt’ first. > starting USB… > No working controllers found > PCIe BRCM: link down > ethernet@7d580000 Waiting for PHY auto negotiation tocomplete… TIMEOUT ! > bcmgenet: PHY startup failed: -110 > missing environment variable: pxeuuid > missing environment variable: boofile > Retrieving file: pxelinux.cfg/01-e4-5f-01-50-0e-76 ---
**Author:** ![bkaindl](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/bkaindl/32/8283_2.png) [@bkaindl](https://forums.balena.io/u/bkaindl)\ **Post date:** [July 26, 2022, 7:48pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/14 "2022-07-26T19:48:22Z")
@kwaremburg Hi Kyle, from what I can tell, having a plan what implementing a proper Chain of Trust for BalenaOS Secure Boot on the CM4 would be a good start for this project. As you wrote, your goal is to “lock down the the CM4 to protect company IP.” IMHO, the word “protect” is not very well defined, and as far as I can tell, it really depends on the class of adversaries you want to protect against. If your application supports network communication over accessible networks, of course a threat analysis for such protection might also have to include the security of the network communication of the devices as well as all the possible communication endpoints (as well as middle mans, e.g. for MITM attacks). E.g. if your devices talk to Balena Cloud or your own OpenBalena server(s), then also the security of these need to be considered in a threat model. > **[Threat model](https://en.wikipedia.org/wiki/Threat_model)** > > Threat modeling is a process by which potential threats, such as structural vulnerabilities or the absence of appropriate safeguards, can be identified and enumerated, and countermeasures prioritized. The purpose of threat modeling is to provide defenders with a systematic analysis of what controls or defenses need to be included, given the nature of the system, the probable attacker's profile, the most likely attack vectors, and the assets most desired by an attacker. Threat modeling answers Co... I am going to write more followup answers, I just wanted to get started from the root. ---
**Author:** ![bkaindl](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/bkaindl/32/8283_2.png) [@bkaindl](https://forums.balena.io/u/bkaindl)\ **Post date:** [July 26, 2022, 8:51pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/15 "2022-07-26T20:51:16Z")
Copying the Secure Boot example from [usbboot/README.md at master · raspberrypi/usbboot · GitHub](https://github.com/raspberrypi/usbboot/blob/master/secure-boot-example/README.md) into BalenaOS gives you: - The Raspberry Pi CM4 boot-loader have to be correctly signed and it is verified on boot. - Since the CM4 device tree and CM4 device tree overlay files are inside the `boot.img`, these are part of the signed an verified image as well. - As part of the signed `boot.img` using the file `kernel8.img`, the U-Boot for BalenaOS is loaded by the CM4 boot loader is signed and verified on boot. Normally, `kernel8.img` contains the kernel, but in BalenaOS, it contains U-Boot. (The macro `resin_set_kernel_root` which is built into the BalenaOS U-Boot appears to select the partition on which loading and booting the BalenaOS Kernel happens) Because the boot ROM of the BCM2711 will only boot a signed bootloader, you also get this: - Prevention of unauthorized modifications of the Boot EEPROM (early boot loader, boot order and secure boot config) of the CM4 - Prevention of unauthorized modifications of the eMMC over USB using rpiboot mode (by placing the CM4 into USB gadget mode and then sending arbitrary code over USB) Especially the last two give you protection against simple reverse-engineering and hacking by someone with physical access to a production device by using the USB debug connector to use the rpiboot mode to boot the CM4 into USB mass-storage gadget mode which otherwise would give full read/write access to the eMMC of the CM4. If this is is enough, just using the slight changes you started to make (and completing them) are sufficient and offer quite some protection. If this satisfies your requirements then stop reading here. If local access to the CM4 USB-Gadget port can be disabled by our HW or physical access to the boards can be restricted to trusted persons, then that would be essentially equivalent. Since the rest of the boot chain and update procedure would use the existing BalenaOS partitions and schemes, this does not protect against modifications done over the network (which could be hostile): If due to some kind of security breach over the Network, an adversary gains privileged access to BalenaOS, a privileged container, a container which has access to your intellectual property, or a container with capabilities to access your intellectual property or the Balena host OS, these further changes would still be possible for an adversary (of course, these need another breach before they could be done, and require a really custom, targeted attack on your infrastructure, which is likely very expensive to do): 1. Remote modification of the BalenaOS Linux kernel 2. Remote modification of any other files on the boot, root and data partitions 3. Remote read of your container’s data if the remote access to the container or the Baleana host OS could be gained - Note: This 3rd point is not just a question of the security of BalenaOS and the Balena server and OS and application update infrastructure, but also a question of the whole chain of development and delivery of containers deployed on the CM4. This page gives some steps to address the first three of these challenges: > **[Secure Boot and Chain of Trust - KOAN](https://koansoftware.com/secure-boot-and-chain-of-trust/)** > > Ensure your device is not running tampered software by verifying its authenticity before execution. Establish software authenticity all the way from the bootloader to user applications. Our secure boot services help implement: Verified bootloader... Here is a copy of these steps, with comments by me: - **Verified bootloader** - Is taken care of by the CM4 secure boot example. - **Kernel verification** - Would be taken care of by the CM4 secure boot example, but: - Because BalenaOS uses U-Boot as another bootloader before starting the kernel, this does not verify the kernel, but the U-Boot binary, and also not it’s boot scripts. - Solution: Build `boot.scr` into U-Boot and make it verify a signed BalenaOS kernel image. - **Root filesystem verification (dm-verity, IMA/EVM)** - Not addressed by the CM4 Secure boot example. - It would have to be supported by BalenaOS. - `dm-verity` would be a good possibility. - IMA/EVM was made for enterprise server hardware and is very slow on embedded devices like the CM4. - **Filesystem cryptography or block device encryption (dm-crypt)** - Not addressed by BalenaOS so far. ---
**Author:** ![bkaindl](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/bkaindl/32/8283_2.png) [@bkaindl](https://forums.balena.io/u/bkaindl)\ **Post date:** [July 26, 2022, 9:23pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/16 "2022-07-26T21:23:37Z")
Note: This has been updated considerably as more analysis gained more insight. AFAICS, based on [[usbboot/secure-boot-example/README.md at master · raspberrypi/usbboot · GitHub](https://github.com/raspberrypi/usbboot/blob/master/secure-boot-example/README.md)](Secure Boot Quickstart), the steps of booting an implementation of a fully secure boot with verified boot on the CM4 would be these: 1. The burnt OTP fuses inside the BCM2711 on the CM4 irrevocably limit the chip to only boot with your signed secure boot firmware. (the silicon die of it is your root of trust) 2. The signed firmware image loaded from the boot EEPROM only executes signed `boot.img`. 3. For BalenaOS, the signed `boot.img` contains the device tree overlay files and `kernel8.img` (in BalenaOS, it contains the Balena U-Boot which selects the root partition for booting). 4. Currently, Balena U-Boot loads the file `boot.scr` from the boot partition, which then boots BalenaOS. Because the contents in `boot.scr` control the boot process (selection which root partition to boot and loading the kernel from it), Balena U-Boot needs to be modified to include the contents of `boot.src` as an U-Boot macro which is compiled into U-Boot. 5. The `boot.scr` macros have to calculate a checksum over the BalenaOS kernel it loaded from the selected root partition and check that this checksum is signed before booting the kernel. - U-Boot has support for checking signatures using RSA, such step would have to be added. - Alternatively, U-Boot’s implementation of verified boot using signed FIT images could be used. 6. To verify the partitions before mounting them, the BalenaOS kernel needs use a (likewise verified or preferably built-in RAMdisk) to mount the root partition and it’s overlays. - To support this, the active/inactive sysroot partitions `/dev/mmcblk0p2` and `/dev/mmcblk0p3` would not be plain `ext4` filesystems but `dm-verity` volumes, and the root hash/key for `dm-verity` would have to be provided in the verified initramfs (could be built into the kernel image) 7. Other partitions like the data partition mounted by BalenaOS would have to be verified as well for fully locking down the boot (likewise using `dm-verity`) 8. Depending on the application and if physical access is to storage is also a concern, data-at-rest on additional storage devices like a SATA/NVMe SSD (or network storage buckets) might have to be protected against unauthorized read/write by encrypting data at rest on these storage locations. This could be done by using encrypted files, an encrypted filesystem or `dm-crypt`. Since encryption layers do not strictly provide data integrity, a verification layer like `dm-verity` on the physical volume would complete the lock down. 9. Finally, if you need to trust that a remote device is in a trusted state (secure and verified boot was performed), one could remotely verify the checksums gathered during verified boot in a TPM and use TPM remote attestation for confidence that the remote TPM really recorded a verified boot and the checksums are not faked. Of course how far you need to go (and in turn how massively you’d have to change BalenaOS into an OS capable of fully verified boot) depends on the threat you have at hand. ### Security/Key architecture Then comes the question on how to architect the deployment and updates. If all devices share the same, single boot-loader signing key, the compromise of that key makes all devices vulnerable for changes and compromising your IP. If each device uses it’s own unique bootloader signing key, each device also needs it’s own unique signed `boot.img` and `boot.sig` pair, and when delivering updates, these must of couse only get their matching updated pair of files. Any failure on always delivering the correctly matching signed files/updates renders them not bootable, only to be recovered by booting a signed mass-storage recovery bootloader over USB as described here: [usbboot/secure-boot-example/README.md at master · raspberrypi/usbboot · GitHub](https://github.com/raspberrypi/usbboot/blob/master/secure-boot-example/README.md#mount-the-cm4-sdemmc-after-enabling-secure-boot) ### Excursion into EFI land Secure boot on the CM4 is really simple compared to EFI secure boot. For EFI secure boot, Microsoft established a very powerful but also very daunting, complicated model. EFI secure boot has 4 states of provisioning with transitions between them using a complicated state machine. It has key provisioning and the possibility to update keys, even thru the running OS. The OS makes calls to EFI running in system management mode behind the back of the OS. Using them and a big API, EFI stores variables for booting and the different layers of EFI secure boot keys. For a full implementation, you need the shim boot-loader, the grub boot-loader and software infrastructure like MOK manager (machine owner key manager) when using MOK keys for development. ### Further ideas Based on the Secure boot Quickstart implementation, the BCM2711 appears to have only one set of fuses for one boot-loader key. Once the fuses are burnt, a bootloader key can no longer be changed or revoked. One should likely only burn the OTP fuses when the CM4 goes into production use. When the OTP fuses are burnt, the bootloader key essentially becomes an “HW owners key”: Who has the key, “owns” the inital step of loading and updating the bootloader as well as flashing the eMMC with new software over USB. Therefore, for the later stages of a secure boot, I guess it good to use another key which you can replaced, updated and/or revoke for the purpose of transferring the right to sign a BalenaOS kernel from one user to another and/or revoking that right when needed. #### Non-Bootloader keys for kernel and rootfs deployment, with revocation For this you could store the certificates/public keys of many pre-created keys inside the signed `boot.img`, and only use one or a few of them at a time and if the device changes e.g. from one set of users to another, revoke the older keys by incrementing a counter in the eMMC `rpmb` partition where the count value of the counter is the number of old keys already revoked. The rpmb partition is specifically designed for this kind of roll-back prevention: It can be read, but it can only be updated by using the secret rpmb key with which the rpmb partition is provisioned by you. U-Boot has code to do all operations supported by with the eMMC rpmb partition: > [Linus Walleij](https://patchwork.kernel.org/project/linux-mmc/list/?submitter=18191) wrote a great [primer on RPMB](https://patchwork.kernel.org/project/linux-mmc/patch/20210303135500.24673-2-alex.bennee@linaro.org/#24027683) which helps to understand what RPMB is. #### Yes it’s a lot… I would have split this into more parts, but the forums limited me to 3 answers. Possibly this should be put into a design spec in a github repo. #### Balena, what is your take on all that so far? @anujdeshpande It would be helpful to see what the Balena team member who is working on secure boot for EFI on x86 has come up with so far. ---
**Author:** ![bkaindl](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/bkaindl/32/8283_2.png) [@bkaindl](https://forums.balena.io/u/bkaindl)\ **Post date:** [July 27, 2022, 4:29pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/19 "2022-07-27T16:29:27Z")
## Step where Kyle’s secure boot attempt fails to boot (and where no signature is checked): @kwaremburg Kyle, the analysis on the `boot.scr` is this (newlines inserted by me): ```auto root@BalenaOS-CM4# grep boot.scr /mnt/boot/kernel8.img boot_scripts=boot.scr.uimg boot.scr ... scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done ``` This means that Balena U-Boot looks for `${prefix}boot.scr` in the path `${devtype} ${devnum}:${distro_bootpart}`. If it is not found, it is skipped and fallbacks are executed instead. To run a verified `boot.scr`, you’d have to replace the macro `scan_dev_for_boot` inside of the Balena U-Boot to not look for `boot.scr` (via `scan_dev_for_scripts`) on mmc 0:1 (you see the message `Scanning mmc 0:1…` from the macro `scan_dev_for_boot` in your log), but instead just contain the contents of `boot.scr` as part of the Macro, so it would look like this: ```auto run resin_set_kernel_root;fdt addr ${fdt_addr};fdt get value bootargs /chosen bootargs;env set bootargs "${bootargs} ${resin_kernel_root} rootwait";fdt rm /chosen bootargs;load ${resin_dev_type} ${resin_dev_index}:${resin_root_part} ${resin_tmp_loadaddr} /boot/Image.gz;unzip ${fileaddr} ${resin_kernel_load_addr};booti ${resin_kernel_load_addr} - ${fdt_addr} ``` Yes, you have to build the BalenaOS U-Boot image file for this, so you need to run your your own BalenaOS image build starting from cloning [GitHub - balena-os/balena-raspberrypi: Balena support for RaspberryPI boards](https://github.com/balena-os/balena-raspberrypi) and running `balena-yocto-scripts/build/barys --machine raspberrypi4-64` on a strong server with a fast SSD with preferably around 40GB of space for the build. Here is the definition of the `scan_dev_for_boot` macro which you have to replace: > **It may only contain a copy of `boot.scr`, extendend with a signature check if you want of check the signature of the kernel (and so on…)**, and not run `scan_dev_for_extlinux` before, to not leave any door open where secure boot could be circumvented. ---
**Author:** ![kwaremburg](https://avatars.discourse-cdn.com/v4/letter/k/87869e/32.png) [@kwaremburg](https://forums.balena.io/u/kwaremburg)\ **Post date:** [August 22, 2022, 1:42pm UTC](https://forums.balena.io/t/cm4-secure-boot/359620/25 "2022-08-22T13:42:08Z")
Thanks for the multiple replies and wealth of information you shared. I haven’t had a chance to fully digest and understand it all at a level where I would be able to implement it but I get the general ideas. As far as protecting company IP we are well aware we need to secure the device while running (ssh, console, etc). This exercise is interested in preventing someone from pulling the CM4 or EEMC from the CM4 to gain access to the information on it. `dm-crypt` and the `secure boot` functionality should get us there if I’m not mistaken.