Balena OS Jetson Nano Tensorflow

I have pushed my project into the application on the Balena cloud.
The docker file is in the following link:

Here, my Balena OS is balenalib/jetson-nano-ubuntu:bionic.

With this docker file, I completed pushing my project into the application successfully.
But after pushing, I have recognized that Cuda, Cudnn, and TensorFlow were installed, but I couldn’t import TensorFlow with the error of “Segmentation failure(Core dumped)”.

While analyzing the reason, I have found out that this Balena OS I used supports JetPack 2.2.1, but I installed tensorflow-gpu==1.13.1 in JetPack 4.2.
RUN pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 tensorflow-gpu==1.13.1+nv19.3 --user

So I guess the reason is based on the difference of the JetPack version between Balena OS and tensorflow-gpu, but I am not sure about it.

Is my analysis right?

And how can I solve this issue- not to import TensorFlow?

Did you check if tensorflow-gpu 1.13.1 is available precompiled for the aarch64 archtecture with CUDA version 10.0?

JetPack 4.2 components:

  • L4T R32.1 (K4.9)
  • Ubuntu 18.04 LTS aarch64
  • CUDA 10.0
  • cuDNN 7.3.1
  • TensorRT 5.0.6

You can find the latest TensoFlow version by nivida here:
https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html

We are using TF2.0 on the JetsonNano with Balena. If you are looking for a compiled wheel of TF2.0rc0 for the jetson nano, feel free to send me an email :slight_smile:

Sorry I did not click on the expand all button :see_no_evil:

If you want to install the latest version of TensorFlow supported by a particular version of JetPack, issue the following command:
sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v$JP_VERSION tensorflow

Furthermore i figured out that BouweCeunen has a nice guide on how to build your own Balena Images for computer vision with the jetson Nano. He posted some links on his GitHub packe with guides on how to download the latest JP, … code.

GitHub repo:

Just FYI:
NVIDA enables you to download these software packages with their proprietary licenses :frowning:

Thanks for your response.

I have installed Cuda, Cudnn, tensorflow as following:
Cuda: cuda-repo-l4t-10-0-local-10.0.326_1.0-1_arm64.deb
Cudnn: libcudnn7-dev_7.6.3.28-1+cuda10.0_arm64.deb
Tensorflow: tensorflow-gpu==1.13.1+nv19.3

I deployed Balena image successfully, but I can not import tensorflow inspite of setting the environment variables.
So I wonder whether these versions are matched or not.

I hope your help.

Best regards
Artem

Thanks for your response.

So you said you are using Tensorflow 2.0 on Jetson Nano board with Balena OS, then which version of Cuda and Cudnn do you install on Nano?
Could you please teach me more in detail?

When I run this app in terminal of the application in Balena Cloud, I have met an error that there is no such folder or file-libnvToolsExt.so.
So I verified whether this file existed or not in /usr/local/lib64, but I couldn’t find out it.
This means that I didn’t install Cuda or Cudnn correctly? Or unmatched version with Tensorflow?

I hope your help

Best regards
Artem

i installed the same version as showed in the repository by BouweCeunen.
JetPack 4.2:

  • L4T R32.1 (K4.9)
  • Ubuntu 18.04 LTS aarch64
  • CUDA 10.0
  • cuDNN 7.3.1
  • TensorRT 5.0.6

The main issue is that you can only install a custom compiled version of TF2.0, since there is no precompiled version of TF2.0 for aarch64 with CUDA support. But going this road will take a lot of time.
I would recommend you to use the latest version bellow 2.0. if you are not depending on features of TF2
pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 ‘tensorflow<2’

For a quick install try to use the dockerfiles provided BouweCeunen as reference:

  1. L4T
  2. CUDA
  3. CUDNN
  4. NvInfer
  5. TensorRT

Lastly try to run:
pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 ‘tensorflow<2’

But i do not know if this works, since i use a custom compiled version of TensorFlow 2.0 and only install a wheel with:
RUN pip3 install tensorflow-${TF_VERSION}-cp36-cp36m-linux_aarch64.whl.
Eventhough NVIDIA should provide a TF1.x wheel [here] (Index of /compute/redist/jp/v42/tensorflow-gpu)

Reference:

BouweCeunen/computer-vision-jetson-nano/blob/master/README.md

Hi, Langhalsdino

Thanks for your quick response

Reading your response, I realized that I needed the compiled wheel of TF2.0.
So could you please share it with me?
If you can do it, I will be thankful for your help.

Best regards
Artem

As mentioned above, my TF2.0 wheel is modified a lot and i would not recommend you to use it.

Did you try these wheels provided by NVIDIA?
JP 4.2.x and NV 19.11
TF1.15
TF2.0
JP 4.3.x and installation of NV 19.12
TF 1.15
TF 2.0

Sorry i should have read everything closely. This seems to be issue with your cuda, cudnn installation and not with Tensorflow.

If the lib64 file exists, make sure to add its env to the ldconfig and export CUDA path:

RUN echo "/usr/local/cuda/lib64" > /etc/ld.so.conf.d/nvidia-cuda && \
    echo "/usr/local/cuda/extras/CUPTI/lib64" > /etc/ld.so.conf.d/nvidia-cupti && \
    ldconfig

ENV CUDA_HOME=/usr/local/cuda
ENV PATH=$PATH:${CUDA_HOME}/bin

If it does not exist use these Dockerfile as reference to get a working installation of CUDA :slight_smile:

  1. L4T
  2. CUDA
  3. CUDNN

Thanks for your effort.

Then, as mentioned above, should I install Nvinfer and TensorRT surely?
I thought after only installing L4T, CUDA, and CUDNN, I intended to install Tensorflow
Am I right?

Best regards
Artem

NvInfere is needed by TensorRT, but both are not needed to run TensorFlow on the JetsonNano. If you want to optimize your NN to run fast on the JetsonNano these libraries might come in handy. Its up to you :slight_smile:

By the way, if you just want to have a working version of OpenCV, CUDA, … quickly, you can probably use this image as a baseimage and install TF from there as you described

FROM bouwe/jetson-nano-l4t-cuda-cudnn-nvinfer-tensorrt-opencv:latest

Thanks

Then to install CUDA and CUDNN, I made docker file considering all of three docker files: L4T, CUDA, CUDNN as following:

FROM balenalib/jetson-nano-ubuntu:bionic

ARG DRIVER_PACK=“Jetson-210_Linux_R32.2.1_aarch64.tbz2”
ARG POWER_MODE=0000

COPY packages/$DRIVER_PACK .

RUN apt-get update &&
apt-get install -y --no-install-recommends bzip2 ca-certificates curl lbzip2 sudo htop curl &&
apt-get install -y zip git python3 python3-pip python3-numpy cmake systemd &&
tar -xpj --overwrite -f ./{DRIVER_PACK} && \ sed -i '/.*tar -I lbzip2 -xpmf {LDK_NV_TEGRA_DIR}/config.tbz2./c\tar -I lbzip2 -xpm --overwrite -f {LDK_NV_TEGRA_DIR}\/config.tbz2' ./Linux_for_Tegra/apply_binaries.sh && \ ./Linux_for_Tegra/apply_binaries.sh -r / && \ rm -rf ./Linux_for_Tegra && \ rm ./{DRIVER_PACK} &&
apt-get clean &&
rm -rf /var/lib/apt/lists/
&&
pip3 install jetson-stats

ENV LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra-egl:${LD_LIBRARY_PATH}

RUN ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.32.1.0 /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so &&
ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.32.1.0 /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.1 &&
ln -sf /usr/lib/aarch64-linux-gnu/tegra/libGL.so /usr/lib/aarch64-linux-gnu/libGL.so &&
ln -s /usr/lib/aarch64-linux-gnu/libcuda.so /usr/lib/aarch64-linux-gnu/libcuda.so.1 &&
ln -sf /usr/lib/aarch64-linux-gnu/tegra-egl/libEGL.so /usr/lib/aarch64-linux-gnu/libEGL.so

RUN ln -s /etc/nvpmodel/nvpmodel_t210_jetson-nano.conf /etc/nvpmodel.conf &&
ln -s /etc/systemd/system/nvpmodel.service /etc/systemd/system/multi-user.target.wants/nvpmodel.service &&
mkdir /var/lib/nvpmodel &&
echo “/etc/nvpmodel.conf” > /var/lib/nvpmodel/conf_file_path &&
echo “pmode:${POWER_MODE} fmode:fanNull” > /var/lib/nvpmodel/status

ARG CUDA_TOOLKIT=“cuda-repo-l4t-10-0-local-10.0.326”
ARG CUDA_TOOLKIT_PKG="${CUDA_TOOLKIT}_1.0-1_arm64.deb"

COPY packages/$CUDA_TOOLKIT_PKG .

RUN apt-get update &&
apt-get install -y --no-install-recommends curl &&
dpkg --force-all -i {CUDA_TOOLKIT_PKG} && \ rm {CUDA_TOOLKIT_PKG} &&
apt-key add var/cuda-repo--local/.pub &&
apt-get update &&
apt-get install -y --allow-downgrades cuda-toolkit-10-0 libgomp1 libfreeimage-dev libopenmpi-dev openmpi-bin &&
dpkg --purge ${CUDA_TOOLKIT} &&
apt-get clean &&
rm -rf /var/lib/apt/lists/

ENV CUDA_HOME=/usr/local/cuda
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
ENV PATH=$PATH:$CUDA_HOME/bin

ENV CUDNN_VERSION 7.5.0.56
ENV CUDNN_PKG_VERSION={CUDNN_VERSION}-1 LABEL com.nvidia.cudnn.version="{CUDNN_VERSION}"

COPY packages/libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb .
COPY packages/libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb .
COPY packages/libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb .

RUN dpkg -i libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
dpkg -i libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
dpkg -i libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb

Here, I omitted the header and footer part from original three docker files.
That is to say:

FROM bouwe/jetson-nano-l4t:latest
ENTRYPOINT ["/bin/bash"]

Is this right?

Best regards
Artem

Looks about right to me. :slight_smile:
Did the build process work?

Yes, I have built the project to the application successfully.
Currently, it updates my device.
But I wonder whether it can be performed on terminal.

So do you think it is not a problem for me to omit the header part of CUDA and CUDNN docker file?

I wonder whether it is right or not.

Thanks for your help and I hope your response.

Best regards
Artem

That is right, since Dockerfiles only source from one base image e.g. FROM balenalib/ubuntu-example that might be hosted on hub.docker.com.
(Side note: There are exceptions such as multistage build, but designing multistage builds is a more advanced topic)

Sorry, I did not get your other questions, therefore my answer is a bit more general.

Maybe you can explain a bit more on what you would like to perform on the terminal?

What is docker

Maybe this might help you to understand Dockerfiles a bit better. Docker is a tool that uses containers to make creation, deployment and running of application a lot easier.

It envelopes the application along with all other elements like external libraries and other such dependencies which are required to run the application into a single package smoothly.

A Dockerfile captures the recipe for building a Docker image in writing. A Docker image typically consists of:

  • A base Docker image on top of which to build your own Docker image
  • A set of tools and applications to be installed in the Docker image
  • A set of files to be copied into the Docker image (e.g configuration files)
  • Possibly opened network ports
  • etc.

Dockerfile Structure

A Dockerfile consists of a set of instructions. Each instruction consists of a command followed by arguments to that command, similar to command line executables. Here is a simple example of a Dockerfile:

# The base image
FROM ubuntu:latest

# More instructions here that install software
RUN apt-get update && apt-get install git -y
# copy files into the image.
COPY /myapp    /src/myapp

# The command executed when running a Docker container
CMD echo "I am online"

Furthermore the documentation by balena might help you further:

Hi

I have successfully built docker file to the application two days ago.
Then balena application has been updating my device, but until now, it is still updating.
After all, it has been updating for 2 days, I wonder why this happens.

When I built before, it took a few hours at most to update device.
So I can’t test whether it is right or not.

I want your help

Best regards
Artem

What kind of internet speed do you have.
The resulting image might be around 10GB.

Can you share the balena logs {UUID} --system ?

Hi

I am always thankful for your help.
I have pushed my project into Balena application using the following Docker file which is very simple.

FROM bouwe/jetson-nano-l4t-cuda-cudnn-nvinfer-tensorrt-opencv-detectnet:latest

WORKDIR /usr/src/app

COPY ./source .
COPY ./utils .
COPY app.py .
COPY settings.py .

ENV UDEV=1

CMD [“sleep”, “infinity”]

, where the basic image is one of images you recommended me in BouweCeunen/computer-vision-jetson-nano/blob/master/README.md.

It has been successfully pushed into application and my device connected to the applicaiton has also been updated.
By the way, when I connect to my device through the terminal in the Balena cloud, it still happens errors as following.

Error response from daemon: Container 3430d36dfc99a3a957cb46ef2efae37c385f5f9e49fae3f917196e7355c45093 is restarting, wait until the container is running
Error response from daemon: Container 3430d36dfc99a3a957cb46ef2efae37c385f5f9e49fae3f917196e7355c45093 is restarting, wait until the container is running

I wonder why this happens.

Could you please continue to help me?

Best Regards
Artem