Getting linux_for_tegra into a container on balena-os

So I understand you have balenalib base images, I’m using jetson-xavier-ubuntu:bionic.

What is the recommended way to install linux_for_tegra into a container based on this base image?

I have tried manually copying over Jetson_Linux_R32.4.3_aarch64.tbz2, unzipping and running apply_binaries.sh but that runs into
dpkg: error processing archive /opt/nvidia/l4t-packages/userspace/nvidia-l4t-firmware_32.4.3-20200625213407_arm64.deb (--install): unable to clean up mess surrounding './lib/firmware/nv-WIFI-Version' before installing another version: Read-only file system dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

I also tried using the nvidia l4t base image https://ngc.nvidia.com/catalog/containers/nvidia:l4t-base but that doesn’t seem to have any of the required drivers or tools installed when I run it. I used this in conjunction with the balenalib base image via two consecutive FROM calls in the dockerfile.

Any suggestion how to get linux_for_tegra installed in a container on balena-os?

Hi,

I found this NVIDIA thread that appears to be related: https://forums.developer.nvidia.com/t/guide-how-to-install-ubuntu-18-04-on-tx2/67176/10. There seemed to be a lack of support for Ubuntu Bionic when this thread originated.

Have you explored these steps?

John

I am using l4t R32.4.3 which I believe is compatible with 18.04.
The steps in the referenced discussion are related to using Jetpack but when working with Balena Jetpack is not involved.

BalenaOS supports the Jetson Xavier and uses the l4t (linux for tegra) tools to actually flash the image onto the device. Given that, I assumed that installing the full gamut of l4t drivers/libraries/tools onto a balena device would be well documented and explored - was just hoping for those details.

So without balena the operation is simple, Jetpack (SDKManager) will download l4t and install all the drivers/libraries/tools into a rootfs from which it generates an image and flashes to the camera. Then when the device comes up Jetpack installed additional libraries/packages over ethernet on the device.
Its these actions I want to achieve on a device running balenaOS that I’m having troubles with.

Hi Jonathan, I forwarded your question to the OS team, hopefully they will be able to guide you

Hi Jonathan, you should be able to install any of the userspace tools you need via apt. I see that we have included the nvidia apt sources in debian, see: https://github.com/balena-io-library/base-images/blob/master/balena-base-images/device-base/jetson-xavier/debian/buster/run/Dockerfile#L3-L5 so I believe for you to use those in ubuntu 18.04 you would just need to copy those 3 lines into your dockerfile. You can see those sources are targeting r32 and the t194 is the internal platform name for the Xavier boards. You should be able to install everything you need via that in your build. I have also created an issue here: https://github.com/balena-io-library/base-images/issues/654 to make sure those are included in our base images and our team is also in the process of converting a few of our example projects to use these source installs. Hope this helps.

So I added those nvidia sources to my sources lists
cat /etc/apt/sources.list.d/nvidia-l4t-apt-source.list deb https://repo.download.nvidia.com/jetson/common r32.4 main deb https://repo.download.nvidia.com/jetson/t194 r32.4 main

Ran apt-get update
root@b202bef:/etc/apt/sources.list.d# apt-get update Get:1 file:/var/cuda-repo-10-0-local-10.0.326 InRelease Ign:1 file:/var/cuda-repo-10-0-local-10.0.326 InRelease Get:2 file:/var/cuda-repo-10-0-local-10.0.326 Release [574 B] Get:2 file:/var/cuda-repo-10-0-local-10.0.326 Release [574 B] Get:3 https://repo.download.nvidia.com/jetson/common r32.4 InRelease [2552 B] Get:4 https://repo.download.nvidia.com/jetson/t194 r32.4 InRelease [2562 B] Hit:5 http://ports.ubuntu.com/ubuntu-ports bionic InRelease Hit:7 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease Hit:8 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease Hit:9 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease Get:10 https://repo.download.nvidia.com/jetson/common r32.4/main arm64 Packages [13.5 kB] Get:11 https://repo.download.nvidia.com/jetson/t194 r32.4/main arm64 Packages [15.8 kB] Fetched 34.4 kB in 3s (13.2 kB/s) Reading package lists... Done

Then confirmed I the nvidia-l4t-firmware package is visible to apt
root@b202bef:/etc/apt/sources.list.d# apt search nvidia-l4t-firmware Sorting... Done Full Text Search... Done nvidia-l4t-firmware/stable 32.4.3-20200625213407 arm64 NVIDIA Firmware Package

Tried to install it
root@b202bef:/etc/apt/sources.list.d# apt-get install nvidia-l4t-firmware Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: nvidia-l4t-firmware 0 upgraded, 1 newly installed, 0 to remove and 52 not upgraded. 11 not fully installed or removed. Need to get 1703 kB of archives. After this operation, 5219 kB of additional disk space will be used. Get:1 https://repo.download.nvidia.com/jetson/t194 r32.4/main arm64 nvidia-l4t-firmware arm64 32.4.3-20200625213407 [1703 kB] Fetched 1703 kB in 0s (5107 kB/s) debconf: delaying package configuration, since apt-utils is not installed (Reading database ... 53419 files and directories currently installed.) Preparing to unpack .../nvidia-l4t-firmware_32.4.3-20200625213407_arm64.deb ... Unpacking nvidia-l4t-firmware (32.4.3-20200625213407) ... dpkg: error processing archive /var/cache/apt/archives/nvidia-l4t-firmware_32.4.3-20200625213407_arm64.deb (--unpack): unable to clean up mess surrounding './lib/firmware/nv-WIFI-Version' before installing another version: Read-only file system dpkg-deb: error: paste subprocess was killed by signal (Broken pipe) Errors were encountered while processing: /var/cache/apt/archives/nvidia-l4t-firmware_32.4.3-20200625213407_arm64.deb E: Sub-process /usr/bin/dpkg returned an error code (1) root@b202bef:/etc/apt/sources.list.d#

And STILL run into this read-only file system error when trying to install this package.

I don’t understand this error. I don’t see this file anywhere on the filesystem in either the host balenaOS nor the container i’m running in.

Well despite not finding a reference to nv-WIFI-Version anywhere,
I discovered my /lib/firmware and /lib/modules directories are read-only…

Confirmed that if I do all this in the dockerfile I don’t hit this read-only file system error.
Maybe this will be helpful to some other docker noobs :smiley:

Hmm not quite there.

My dockerfile:
ARG version=0.0.2
FROM balenalib/jetson-xavier-ubuntu:bionic
RUN echo "deb https://repo.download.nvidia.com/jetson/common r32.4 main" > /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
RUN echo "deb https://repo.download.nvidia.com/jetson/t194 r32.4 main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
RUN apt-key adv --fetch-key http://repo.download.nvidia.com/jetson/jetson-ota-public.asc
RUN apt-get update
RUN apt-get install nvidia-l4t-core -y

I get:


Preparing to unpack …/21-nvidia-l4t-core_32.4.3-20200625213407_arm64.deb …
/var/lib/dpkg/tmp.ci/preinst: line 40: /proc/device-tree/compatible: No such file or directory
dpkg: error processing archive /tmp/apt-dpkg-install-KsEMHe/21-nvidia-l4t-core_32.4.3-20200625213407_arm64.deb (–unpack):
new nvidia-l4t-core package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
/tmp/apt-dpkg-install-KsEMHe/21-nvidia-l4t-core_32.4.3-20200625213407_arm64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

The docker build command is just docker build --platform=aarch64 .

I’m building an arm based container (I have qemu installed for host emulation). I can run the image just prior to this and poke around in the image.

Just to reiterate balena supports nvidia jetson xavier devices so it seems like it should have a pretty straight forward way of installing the linux-for-tegra packages that typical applications would depend on.

How can I install these packages in a container and deploy it to the xavier?

nvidia-l4t-3d-core_32.4.2-20200408182620_arm64.deb
nvidia-l4t-graphics-demos_32.4.2-20200408182620_arm64.deb nvidia-l4t-oem-config_32.4.2-20200408182620_arm64.deb
nvidia-l4t-apt-source_32.4.2-20200408182620_arm64.deb nvidia-l4t-gstreamer_32.4.2-20200408182620_arm64.deb nvidia-l4t-tools_32.4.2-20200408182620_arm64.deb
nvidia-l4t-camera_32.4.2-20200408182620_arm64.deb nvidia-l4t-init_32.4.2-20200408182620_arm64.deb nvidia-l4t-wayland_32.4.2-20200408182620_arm64.deb
nvidia-l4t-configs_32.4.2-20200408182620_arm64.deb nvidia-l4t-initrd_32.4.2-20200408182620_arm64.deb nvidia-l4t-weston_32.4.2-20200408182620_arm64.deb
nvidia-l4t-core_32.4.2-20200408182620_arm64.deb nvidia-l4t-jetson-io_32.4.2-20200408182620_arm64.deb nvidia-l4t-x11_32.4.2-20200408182620_arm64.deb
nvidia-l4t-cuda_32.4.2-20200408182620_arm64.deb nvidia-l4t-multimedia_32.4.2-20200408182620_arm64.deb nvidia-l4t-xusb-firmware_32.4.2-20200408182620_arm64.deb
nvidia-l4t-firmware_32.4.2-20200408182620_arm64.deb nvidia-l4t-multimedia-utils_32.4.2-20200408182620_arm64.deb

Hi Jonathan,

Here’s a Dockerfile I used to install these packages.

FROM balenalib/jetson-xavier-ubuntu:bionic

ENV DEBIAN_FRONTEND noninteractive

WORKDIR /usr/src/app

# nvidia-l4t-core is a dependency for the rest
# of the packages, and is designed to be installed directly
# on the target device. This because it parses /proc/device-tree
# in the deb's .preinst script. Looks like we can bypass it though:
RUN \
    echo "deb https://repo.download.nvidia.com/jetson/common r32.4 main" >> /etc/apt/sources.list && \
    echo "deb https://repo.download.nvidia.com/jetson/t194 r32.4 main" >> /etc/apt/sources.list && \
    apt-key adv --fetch-key http://repo.download.nvidia.com/jetson/jetson-ota-public.asc && \
    mkdir -p /opt/nvidia/l4t-packages/ && \
    touch /opt/nvidia/l4t-packages/.nv-l4t-disable-boot-fw-update-in-preinstall && \
    apt-get update && \
    apt-get install -y --no-install-recommends nvidia-l4t-core

RUN \
    apt-get install --no-install-recommends -y \
    nvidia-l4t-firmware \
    nvidia-l4t-multimedia-utils \
    nvidia-l4t-multimedia \
    nvidia-l4t-cuda \
    nvidia-l4t-x11 \
    nvidia-l4t-camera \
    nvidia-l4t-tools \
    nvidia-l4t-graphics-demos \
    nvidia-l4t-gstreamer \
    nvidia-l4t-jetson-io \
    nvidia-l4t-configs \
    nvidia-l4t-3d-core \
    nvidia-l4t-oem-config

ENV UDEV=1

CMD ["bash"]

Some of the packages you listed don’t seem very useful in container unless they include some configuration you really need. They can probably be dropped to save some space, for instance the nvidia-l4t-initrd or nvidia-l4t-xusb-firmware…

Perfect! This was what I needed. Thanks!!