Hi all,
Currently, we are using the CAN bus of the Balena devices (Jetson AGX) to receive messages from some devices. Recently, we updated the Balena OS to newer version (v2.105.2) and realized CAN bus does not work with newer Balena OS.
Previously, we were using v2.80.5 Balena OS and the CAN bus connection was working. However, after upgrading to the newer OS versions (v2.99.27, v2.105.2), we no longer see any CAN bus messages.
The setup scripts we used to test CAN bus on the Balena side are as follows:
#!/bin/bash
busybox devmem 0x0c303000 32 0x0000C400
busybox devmem 0x0c303008 32 0x0000C458
busybox devmem 0x0c303010 32 0x0000C400
busybox devmem 0x0c303018 32 0x0000C458
modprobe can
modprobe can_raw
modprobe mttcan
ip link set down can0
ip link set can0 type can bitrate 500000 dbitrate 2000000 berr-reporting on fd on
ip link set up can0
exit 0
We used the command candump can0 to receive the CAN messages after the setup on the Balena device side. On the sender side, we sent CAN messages using cansend can0 02a#0000.
We have confirmed that the CAN messages are received with Balena OS versions 2.80.5 and 2.67.2, but not with versions 2.99.27 and 2.105.2. Can you please let me know if there have been any changes between the old and new versions regarding the CAN bus? If there have been changes, do you have any suggestions on how to make the CAN bus work with the newer Balena OS versions?
Thank you!
After digging about this issue, I realized that the same issue happens with JetPack 4.6 ubuntu and also that the issue came from the clock source of mttcan kernel module. Nvidia recently changed the clock source to pll_c from pll_aon to save the power. I am trying to use 500,000 bit rate and pll_c does not support that high bitrate for canbus, so I needed to change the default clock for mttcan following this instruction: https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3271/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/clocks.html#wwpID0E04B0HA
Basically to change the default clock for mttcan, I needed to modify two dtbs:
Linux_For_Tegra/kernel/dtb/tegra194-p2888-0001-p2822-0000.dtb
Linux_For_Tegra/bootloader/t186ref/tegra194-a02-bpmp-p2888-a04.dtb
I updated these two files and flashed it, I could receive the CAN messages on JetPack Ubuntu.
Now I am trying to do the same thing for the balenaOS, and what I have done is
-
Added the modified kernel dtb file (tegra194-p2888-0001-p2822-0000.dtb) to
balena-jetson/layers/meta-balena-jetson/recipes-kernel/linux/linux-tegra
, and updated
balena-jetson/layers/meta-balena-jetson/recipes-kernel/linux/linux-tegra_%.bbappend accordingly.
-
After that downloaded L4T BSP and update the bootloader dtb (tegra194-a02-bpmp-p2888-a04.dtb), compressed it to tar file and open the local webserver, and update jetson-flash/lib/resin-jetson-flash.js accordingly to use the updated BSP.
After doing this, it seems like the kernel dtb changes are applied (I confirmed the clock name is changed to pllaon using cat /proc/device-tree/mttcan@c310000/pll_source), but in the dmesg, there was this error
[ 38.288323] mttcan c310000.mttcan: unable to set CAN_CLK parent
[ 38.318556] mttcan c310000.mttcan: probe failed
[ 38.320861] mttcan c320000.mttcan: unable to set CAN_CLK parent
[ 38.341751] mttcan c320000.mttcan: probe failed
Because the allowed-parents is defined in the bootloader dtb (tegra194-a02-bpmp-p2888-a04.dtb), so I suspected that the 2nd step that I took didn’t work properly.
Is there anything I am missing to apply the modified dtb file? If you could give me any guidance for this step, it would be appreciated!
3 Likes
Hi @acostach
Thanks so much for helping! I tried the local Yocto build with your new branch in a clean repo with docker 18.09.03 version, but I got some errors. I thought it was a docker-related issue, so I tried with the most recent docker version (24.0.2) and got another build issue. Could you take a look at these errors?
18.09.3.error.txt (1.1 KB)
24.0.2.error.txt (108.5 KB)
DEBUG: Mirror fetch failure for url https://mirrors.kernel.org/yocto-sources/bitflags-1.2.1.crate (original url: https://crates.io/api/v1/crates/bitflags/1.2.1/download)
...
ERROR: Bitbake Fetcher Error: FetchError('Unable to fetch URL from any source.', 'crate://crates.io/bitflags/1.2.1')
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished
How does the git symlink look like if you do a ‘which git’? I wonder if it’s pointing to a git from another build directory? It should point to /usr/bin/git but I’ve seen similar errors when it was pointing to a different build directory, other than the one used for the build. For example: /home/acostach/work/jetson_repos/balena-jetson/layers/poky/scripts/git If it does not point to the right git binary, remove the existing symlink and try again: ./balena-yocto-scripts/build/barys -b build_agx_orin -m jetson-agx-orin-devkit -d
If git is working fine then it’s worth checking if you are able to clone those repositories from your build environment, it might as well be a firewall that’s preventing you from downloading them.
Also, please note that we use Ubuntu 18.04 for Yocto builds and other distros may not work as expected, see https://github.com/balena-os/balena-yocto-scripts/blob/master/automation/Dockerfile_yocto-build-env
Hi @acostach
I cloned the branches in a new folder again, and tried to build the image, and built this time. So I could flash an AGX based on your branch of jetson-flash and balena-jetson. AGX powered up but seems like the AGX is having the same issue that I got previously.
[ 0.000000] OF: fdt:memory scan node memory, reg size 48,
[ 1.243963] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[ 1.244404] tegra-pwm c340000.pwm: PWM clk can sleep in ops
[ 1.245036] tegra-pwm 32c0000.pwm: PWM clk can sleep in ops
[ 1.245403] tegra-pwm 32f0000.pwm: PWM clk can sleep in ops
[ 1.667146] FAN: can't find tach_gpio
[ 41.304266] mttcan c310000.mttcan: unable to set CAN_CLK parent
[ 41.330153] mttcan c310000.mttcan: probe failed
[ 41.339455] mttcan c320000.mttcan: unable to set CAN_CLK parent
[ 41.361686] mttcan c320000.mttcan: probe failed
[ 697.236165] can: controller area network core (rev 20120528 abi 9)
[ 697.266268] can: raw protocol (rev 20120528)
The output of cat /sys/kernel/debug/bpmp/debug/clk/can1/parent is pll_c (which is supposed to be pll_aon And the output of cat /proc/device-tree/mttcan@c310000/pll_source is pllaon.
So seems like the tegra194-p2888-0001-p2822-0000.dtb is updated, but the allowd-parents are not updated.
So just for fun, I tried to add this below on line 169 in layers/meta-balena-jetson/recipes-bsp/tegra-binaries/tegra194-flash-dry_32.7.3.bb, because this part worked in the same situation, but I got a build error.
[log.do_configure.9576.txt|attachment](upload://4ySKal50mbH2UkuKtTJLpWysfJw.txt) (3.7 KB)
for dtnode in can1 can2; do
fdtput -t x tegra194-a02-bpmp-p2888-a04.dtb "/clocks/clock@$dtnode" "allowed-parents" "121" "5b" "13a" "5e"
done
Error message is as follows:
log.do_configure.9576.txt (3.7 KB)
I got this error too previously:
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_configure
Error at '/clocks/clock@can1': FDT_ERR_BADMAGIC
it it turned out it happens because the bpmp dtbs were compressed with lz4, so in the PR tegra194-flash-dry: Set pllaon as parent for can1 and can2 in bpmp by acostach · Pull Request #625 · balena-os/balena-jetson · GitHub , more exactly in commit tegra194-flash-dry: Set pllaon as parent for can1 and can2 in bpmp by acostach · Pull Request #625 · balena-os/balena-jetson · GitHub I switched to doing the switch prior to the bpmp compression performed by the meta-tegra tegra-bootfiles_32.7.3.bb recipe.
Also, the bpmp you edit seems a bit different, it’s tegra194-a02-bpmp-p2888-a04.dtb whereas the one used in my build is tegra194-flash-dry: Set pllaon as parent for can1 and can2 in bpmp by acostach · Pull Request #625 · balena-os/balena-jetson · GitHub
Ah okay. Seems like that was the issue. I think we need to modify tegra194-a02-bpmp-p2888-a04.dtb not tegra194-a02-bpmp-p2888-a01.dtb to change the list of allowed parent clock for mttcan. I tried to change a01 to a04 here: tegra194-flash-dry: Set pllaon as parent for can1 and can2 in bpmp by acostach · Pull Request #625 · balena-os/balena-jetson · GitHub, and I built a new image again, the parent clock was successfully changed to pllaon and I could receive the canbus message.
Is there a reason we need to modify a01, not a04? The instruction I followed before for Jetpack Ubuntu (https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3271/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/clocks.html#wwpID0E04B0HA) was modifying a04.
Thank you! Just confirmed that I could receive the can message!
Perfect, thank you! I will merge the PR, do a release for the AGX Xavier and let you know as soon as it becomes available in balena cloud.
1 Like
Hi @changkk , the v3.1.3+rev1 image is now available in balena-cloud for the Xavier AGX and it includes the latest changes you validated above. Jetson Flash v0.5.49 should be used if you want to provision the AGX at this version, as the BSP has been bumped to L4T 32.7.3.
2 Likes