Error in Container: mount: /sys/kernel/debug: nodev already mounted or mount point busy.

Product support
1

Hi, when i run a particular container (which is new just a script to test why), i get this warning. also, if i do anything more in the script it just becomes worse.

mount: /sys/kernel/debug: nodev already mounted or mount point busy.

i also get this, mount: /sys/kernel/debug: nodev already mounted or mount point busy. CNDT /usr/bin/entry.sh: line 38: 30 Killed udevadm trigger &> /dev/null

here is the Docker file, which will eventually run an Ai Project hence the libs. i have added another Express project to see if this is just a container or the whole proejct.

FROM balenalib/jetson-nano-emmc-ubuntu:bionic

# Don't prompt with any configuration questions
ENV DEBIAN_FRONTEND noninteractive
# Set our working directory
WORKDIR /usr/src/app
# Install CUDA, CUDA samples and some utilities
RUN \
    #apt-get update && apt-get install -y cuda-toolkit-10-2 cuda-samples-10-2 \
    apt-get update && apt-get install -y cuda-toolkit-10-2 \
    lbzip2 wget tar python3 python3-pip libegl1 libcudnn8 \
    nvidia-cudnn8 nvidia-cuda

# # use `install_packages` if you need to install dependencies,
# # for instance if you need git, just uncomment the line below.
RUN apt-get install -y git libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran
RUN apt-get install -y build-essential libatlas-base-dev libsm6 libxext6 libxrender1 libfontconfig1 
RUN apt-get install -y pkg-config 
RUN apt-get update
RUN apt-get install -y vim

RUN apt-get update && apt-get install -y wget tar lbzip2 python3 libegl1 && \
    wget https://developer.nvidia.com/embedded/l4t/r32_release_v5.2/t210/jetson-210_linux_r32.5.2_aarch64.tbz2 && \
    tar xf jetson-210_linux_r32.5.2_aarch64.tbz2 && \
    cd Linux_for_Tegra && \
    sed -i 's/config.tbz2\"/config.tbz2\" --exclude=etc\/hosts --exclude=etc\/hostname/g' apply_binaries.sh && \
    sed -i 's/install --owner=root --group=root \"${QEMU_BIN}\" \"${L4T_ROOTFS_DIR}\/usr\/bin\/\"/#install --owner=root --group=root \"${QEMU_BIN}\" \"${L4T_ROOTFS_DIR}\/usr\/bin\/\"/g' nv_tegra/nv-apply-debs.sh && \
    sed -i 's/LC_ALL=C chroot . mount -t proc none \/proc/ /g' nv_tegra/nv-apply-debs.sh && \
    sed -i 's/umount ${L4T_ROOTFS_DIR}\/proc/ /g' nv_tegra/nv-apply-debs.sh && \
    sed -i 's/chroot . \//  /g' nv_tegra/nv-apply-debs.sh && \
    ./apply_binaries.sh -r / --target-overlay && cd .. \
    rm -rf Tegra210_Linux_R32.4.4_aarch64.tbz2 && \
    rm -rf jetson-210_linux_r32.5.2_aarch64.tbz2 && \
    echo "/usr/lib/aarch64-linux-gnu/tegra" > /etc/ld.so.conf.d/nvidia-tegra.conf && ldconfig


# pip install python deps from requirements.txt on the resin.io build server
RUN pip3 install setuptools==49.6.0 cython
RUN pip3 install -U --no-deps numpy==1.19.4 future==0.18.2 mock==3.0.5 keras_preprocessing==1.1.2 keras_applications==1.0.8 gast==0.4.0 protobuf==3.19.4 pybind11 cython pkgconfig 
RUN env H5PY_SETUP_REQUIRES=0 pip3 install -U h5py==2.9.0
#RUN pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v45 tensorflow-gpu==1.14.0+nv19.7
RUN pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v45 tensorflow==1.15.4+nv20.12
RUN install_packages python3-sklearn
RUN pip3 install --upgrade pip
RUN pip3 install scikit-build
RUN pip3 install keras==2.3.1 opencv-python-headless
RUN pip3 install numpy==1.18.5

# This will copy all files in our root to the working  directory in the container
COPY . ./
RUN chmod +x start.sh

# scriptwill run when container starts up on the device
CMD ["bash","start.sh"]

The Docker Compose file:

version: '2.1'

services:
  CNDT:
    build: ./CNDT2
    network_mode: host
    privileged: true
    restart: always
    ports:
      - "8085:8085"
    expose:
      - "80"
    environment:
      - UDEV=1
      - 'DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket'
    labels:
      io.balena.features.dbus: '1'
      io.balena.features.kernel-modules: "1"
      io.balena.features.sysfs: "1"
      io.balena.features.supervisor-api: "1"
  express:
    build: ./Express
    network_mode: host
    privileged: true
    restart: always
    ports:
      - "3000:3000"
    expose:
      - "80"
    environment:
      - UDEV=1
      - 'DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket'
    labels:
      io.balena.features.dbus: '1'
      io.balena.features.kernel-modules: "1"
      io.balena.features.sysfs: "1"
      io.balena.features.supervisor-api: "1"

any ideas?

Tim

3

Hi Tim

While trying to reproduce your issue I found that if I specify only one of io.balena.features.sysfs: '1' and UDEV=on, the container starts with no error.

I’ve asked some other balena devs for more insights about this. But you may want to try this workaround.

I’m also curious about the different labels you’re using, and if you need them all to be enabled.

Regards
Ramiro

6

Hi Ramiro, sorry for the late reply. yes this works a charm! as for the labels, we needed access to some kernel and hardware devices. these are all working now. thanks for the solution!

Cheers, Tim

8

Hi @ramirogm was there ever any update on this issue? We too are seeing the same error when enabling both sysfs and udev.

mount: /sys/kernel/debug: nodev already mounted or mount point busy.
/usr/bin/entry.sh: line 38: 31 Killed udevadm trigger &> /dev/null

12

Hi @pwn4 , I checked with the dev team and this issue was fixed with this PR: entryscript: check if sys/kernel mount point already exists by nghiant2710 · Pull Request #789 · balena-io-library/base-images · GitHub , and released.
Can you check again and if it doesn’t work, share here which base image you’re using?

15

Just to be clear, by “base image” I mean the first “FROM balenalib/?” on your Dockerfile.