Hi all,
I need some help with debugging Gstreamer blacklisting Deepstream plugins in my docker container. The build is fine, but for some reason gstreamer is not registering the deepstream plugin, looking at logs output there is no real indication of why it fails to load the library too.
The OS is built on L4T 32.6.1, and I’m using a barebone TX2NX base image provided by Balena. The top layers of the image are shown below. We are using cuda 10.2, tensorrt 8.0.1 and deepstream 6.0.0.
# base image common to both build and run images
FROM balenalib/jetson-tx2-ubuntu:bionic-20220622 as base
# Don't prompt with any configuration questions
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
# Update apt sources for 32.6
RUN echo "deb https://repo.download.nvidia.com/jetson/common r32.6 main" > /etc/apt/sources.list.d/nvidia.list && \
echo "deb https://repo.download.nvidia.com/jetson/t186 r32.6 main" >> /etc/apt/sources.list.d/nvidia.list
# Download and install BSP binaries for L4T 32.6.1
RUN apt-get update && apt-get install -y wget tar lbzip2 python3 libegl1 binutils && \
wget https://developer.nvidia.com/embedded/l4t/r32_release_v6.1/t186/jetson_linux_r32.6.1_aarch64.tbz2 && \
tar xf jetson_linux_r32.6.1_aarch64.tbz2 && \
cd Linux_for_Tegra && \
sed -i 's/config.tbz2\"/config.tbz2\" --exclude=etc\/hosts --exclude=etc\/hostname/g' apply_binaries.sh && \
sed -i 's/install --owner=root --group=root \"${QEMU_BIN}\" \"${L4T_ROOTFS_DIR}\/usr\/bin\/\"/#install --owner=root --group=root \"${QEMU_BIN}\" \"${L4T_ROOTFS_DIR}\/usr\/bin\/\"/g' nv_tegra/nv-apply-debs.sh && \
sed -i 's/LC_ALL=C chroot . mount -t proc none \/proc/ /g' nv_tegra/nv-apply-debs.sh && \
sed -i 's/umount ${L4T_ROOTFS_DIR}\/proc/ /g' nv_tegra/nv-apply-debs.sh && \
sed -i 's/chroot . \// /g' nv_tegra/nv-apply-debs.sh && \
./apply_binaries.sh -r / --target-overlay && \
rm -rf /jetson_linux_r32.6.1_aarch64.tbz2 && \
rm -rf /Linux_for_Tegra && \
echo "/usr/lib/aarch64-linux-gnu/tegra" > /etc/ld.so.conf.d/nvidia-tegra.conf && ldconfig
# Install required NVIDIA packages
RUN apt-get update && apt-get install --yes \
cuda-cudart-10-2=10.2.300-1 \
cuda-nvprof-10-2=10.2.300-1 \
cuda-driver-dev-10-2=10.2.300-1 \
cuda-cudart-dev-10-2=10.2.300-1 \
cuda-nvcc-10-2=10.2.300-1 \
cuda-nvrtc-10-2=10.2.300-1 \
libcublas10=10.2.3.300-1 \
libcudnn8=8.2.1.32-1+cuda10.2 \
libcurand-10-2=10.1.2.300-1 \
libcurand-dev-10-2=10.1.2.300-1 \
libnvinfer8=8.0.1-1+cuda10.2 \
libnvinfer-plugin8=8.0.1-1+cuda10.2 \
libnvonnxparsers8=8.0.1-1+cuda10.2 \
libnvparsers8=8.0.1-1+cuda10.2 \
python3-libnvinfer=8.0.1-1+cuda10.2 \
&& rm -rf /var/lib/apt/lists/*
# Install gstreamer
RUN apt-get update && apt-get install --yes \
libgstreamer1.0-0=1.14.5-0ubuntu1~18.04.2 \
gstreamer1.0-tools=1.14.5-0ubuntu1~18.04.2 \
gstreamer1.0-plugins-good=1.14.5-0ubuntu1~18.04.3 \
gstreamer1.0-plugins-bad=1.14.5-0ubuntu1~18.04.1 \
gstreamer1.0-plugins-ugly=1.14.5-0ubuntu1~18.04.1 \
gstreamer1.0-libav=1.14.5-0ubuntu1~18.04.1 \
libgstreamer-plugins-base1.0-dev=1.14.5-0ubuntu1~18.04.3 \
libgstrtspserver-1.0-0=1.14.5-0ubuntu1~18.04.1 \
libgstrtspserver-1.0-dev=1.14.5-0ubuntu1~18.04.1 \
&& rm -rf /var/lib/apt/lists/*
# CUDA Varibles
ENV CUDA_ROOT="/usr/local/cuda-10.2"
ENV PATH="$CUDA_ROOT/bin:${PATH}"
ENV CPATH="$CUDA_ROOT/targets/aarch64-linux/include:${CPATH}"
ENV LD_LIBRARY_PATH="$CUDA_ROOT/targets/aarch64-linux/lib:${LD_LIBRARY_PATH}"
# Install important apt packages
RUN apt-get update && apt-get install -y --allow-unauthenticated --no-install-recommends \
libusb-1.0-0-dev \
openssh-client \
python3-gi=3.26.1-2ubuntu1 \
python3-numpy=1:1.13.3-2ubuntu1 \
python3-pip \
python3-venv=3.6.7-1~18.04 \
python3-dev=3.6.7-1~18.04 \
nfs-common \
isc-dhcp-client\
kmod \
libssl-dev \
net-tools \
usbutils \
python3-dbus \
iputils-ping \
cryptsetup \
gdisk \
libjson-glib-dev=1.4.2-3ubuntu0.18.04.1 \
libssl1.1=1.1.1-1ubuntu2.1~18.04.23 \
libjansson4=2.11-1 \
libyaml-cpp-dev=0.5.2-4ubuntu1 \
&& rm -rf /var/lib/apt/lists/*
# Install ffmpeg version 4.3.2 and its relevant required libs
RUN apt-get update && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:jonathonf/ffmpeg-4 && \
apt-get install -y ffmpeg && \
apt-get install -y --no-install-recommends \
libavformat-dev \
libavcodec-dev \
libavdevice-dev \
libavutil-dev \
libavfilter-dev \
libswscale-dev \
libswresample-dev && \
rm -rf /var/lib/apt/lists/* && \
apt-get remove -y software-properties-common
# Install deepstream package (also remove installation of test app)
WORKDIR /
RUN wget https://developer.download.nvidia.com/assets/Deepstream/DeepStream_6.0/deepstream_sdk_v6.0.0_jetson.tbz2 && \
tar -xvf deepstream_sdk_v6.0.0_jetson.tbz2 -C / && \
sed -i '47,68d' /opt/nvidia/deepstream/deepstream-6.0/install.sh && \
/opt/nvidia/deepstream/deepstream-6.0/install.sh && \
ldconfig && \
rm -rf /deepstream_sdk_v6.0.0_jetson.tbz2 && \
rm -rf /opt/nvidia/deepstream/deepstream-6.0/bin && \
rm -rf /opt/nvidia/deepstream/deepstream-6.0/doc && \
rm -rf /opt/nvidia/deepstream/deepstream-6.0/samples
... rest of our application
The installation seems straightforward, and is very similar to the installation on Jetpack 4.6, apart from the BSP download and applying debs manually.
For some reason, gstreamer is blacklisting some of the deepstream plugins. I’ve manually check if the dependencies for the shared library exist by using ldd, and they all shows up fine. I also checked the output of ldconfig -p to ensure the gstreamer plugins exist, and they do. I’ve attach the output when doing GST_DEBUG=4 gst-inspect-1.0 -b after removing the cache file in /root/.cache/gstreamer-1.0.
inspect-output.txt (151.5 KB)
There’s a few stderr out regarding nvbuf_utils: Could not get EGL display connection, but the DISPLAY env var is already unset, and this device does not have any display output as well.
Any help pointing towards where to look into fixing this issue would be appreciated. Thanks!