Balena OS Jetson Nano Tensorflow

artem-vasil · March 8, 2020, 7:49am

I have pushed my project into the application on the Balena cloud.
The docker file is in the following link:

artem-vasiltsov/JetsonAgeGender/blob/master/Dockerfile.template

FROM balenalib/jetson-nano-ubuntu:bionic

WORKDIR /usr/src/app

COPY ./deb/*.* ./

ENV DEBIAN_FRONTEND noninteractive

RUN \
    dpkg -i cuda-repo-l4t-10-0-local-10.0.326_1.0-1_arm64.deb \
    libcudnn7_7.6.3.28-1+cuda10.0_arm64.deb \
    libcudnn7-dev_7.6.3.28-1+cuda10.0_arm64.deb && \
    apt-key add /var/cuda-repo-10-0-local-10.0.326/*.pub && \
    apt-get update && \
    apt-get install cuda-compiler-10-0 cuda-samples-10-0 -y && \
    rm -rf *.deb && \
    dpkg --remove cuda-repo-l4t-10-0-local-10.0.326 && \
    dpkg -P cuda-repo-l4t-10-0-local-10.0.326 && \
    echo "/usr/lib/aarch64-linux-gnu/tegra" > /etc/ld.so.conf.d/nvidia-tegra.conf && \
    ldconfig

This file has been truncated. show original

Here, my Balena OS is balenalib/jetson-nano-ubuntu:bionic.

With this docker file, I completed pushing my project into the application successfully.
But after pushing, I have recognized that Cuda, Cudnn, and TensorFlow were installed, but I couldn’t import TensorFlow with the error of “Segmentation failure(Core dumped)”.

While analyzing the reason, I have found out that this Balena OS I used supports JetPack 2.2.1, but I installed tensorflow-gpu==1.13.1 in JetPack 4.2.
RUN pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 tensorflow-gpu==1.13.1+nv19.3 --user

So I guess the reason is based on the difference of the JetPack version between Balena OS and tensorflow-gpu, but I am not sure about it.

Is my analysis right?

And how can I solve this issue- not to import TensorFlow?

Langhalsdino · March 8, 2020, 2:51pm

Did you check if tensorflow-gpu 1.13.1 is available precompiled for the aarch64 archtecture with CUDA version 10.0?

JetPack 4.2 components:

L4T R32.1 (K4.9)
Ubuntu 18.04 LTS aarch64
CUDA 10.0
cuDNN 7.3.1
TensorRT 5.0.6

You can find the latest TensoFlow version by nivida here:
https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html

We are using TF2.0 on the JetsonNano with Balena. If you are looking for a compiled wheel of TF2.0rc0 for the jetson nano, feel free to send me an email

Langhalsdino · March 8, 2020, 2:53pm

Sorry I did not click on the expand all button

If you want to install the latest version of TensorFlow supported by a particular version of JetPack, issue the following command:
sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v$JP_VERSION tensorflow

Langhalsdino · March 8, 2020, 3:05pm

Furthermore i figured out that BouweCeunen has a nice guide on how to build your own Balena Images for computer vision with the jetson Nano. He posted some links on his GitHub packe with guides on how to download the latest JP, … code.

GitHub repo:

github.com

BouweCeunen/computer-vision-jetson-nano/blob/master/README.md

# Computer Vision with Jetson Nano - Object Detection
Object detection with SSD MobileNet v2 COCO model optimized with TensorRT on NVIDIA Jetson Nano built upon Jetson Inference of dusty-nv (https://github.com/dusty-nv/jetson-inference).

Feel free to read about this with some more details on [Medium](https://medium.com/technology-at-rombit/computer-vision-with-jetson-nano-bccf8e6c6256).

[![DockerHub Badge](https://dockeri.co/image/bouwe/computer-vision-jetson-nano)](https://hub.docker.com/r/bouwe/computer-vision-jetson-nano)

All docker files rely on standard deb/zip/tbz2 files which include all necessary libraries like CUDA, CUDNN, TensorRT etc. It is possible to download these directly from the nvidia developer console. I'm using a Logitech C920 HD Pro webcam attached to my Jetson Nano for doing object detection with. I've used the Jetson Inference of dusty-nv (https://github.com/dusty-nv/jetson-inference) to make this work. The pretrained SSD MobileNet v2 COCO model optimized for TensorRT is used. It will automatically serve the images with Bottle (https://github.com/bottlepy/bottle) on port 80 of your Jetson Nano which can then be viewed by going to the ip of that device. Resolution of the webcam is set to 1024x720. I'd advise to look at NVIDIA DeepStream for more performant detection.

[![jetson-nano](./images/detection.gif)](./images/detection.gif)

```bash
{'datetime': '2019-10-21 17:21:58.437843', 'objects': {0: {'width': 1162.0321044921875, 'height': 647.0380249023438, 'class': 'potted plant', 'confidence': 0.7641790533065796, 'center': (611.6964111328125, 379.59674072265625)}, 1: {'width': 1234.2661349701875, 'height': 829.8434554443438, 'class': 'vase', 'confidence': 0.7241782743065926, 'center': (608.7052469498425, 445.52664032255785)}}, 'objects_count': '2'}
```

## Run with Docker
There's a docker image of this application which can be run on your Jetson Nano device. This image has all of the Jetpack components already installed so it will work without NVIDIA JETPACK flashed onto your sd-card, this will come in handy when you would want to deploy this with Balena, as explained later. The object recognition will be available in your browser.

```bash
docker run --privileged -it bouwe/computer-vision-jetson-nano:latest

This file has been truncated. show original

Just FYI:
NVIDA enables you to download these software packages with their proprietary licenses

artem-vasil · March 8, 2020, 7:45pm

Thanks for your response.

I have installed Cuda, Cudnn, tensorflow as following:
Cuda: cuda-repo-l4t-10-0-local-10.0.326_1.0-1_arm64.deb
Cudnn: libcudnn7-dev_7.6.3.28-1+cuda10.0_arm64.deb
Tensorflow: tensorflow-gpu==1.13.1+nv19.3

I deployed Balena image successfully, but I can not import tensorflow inspite of setting the environment variables.
So I wonder whether these versions are matched or not.

I hope your help.

Best regards
Artem

artem-vasil · March 8, 2020, 7:49pm

Thanks for your response.

So you said you are using Tensorflow 2.0 on Jetson Nano board with Balena OS, then which version of Cuda and Cudnn do you install on Nano?
Could you please teach me more in detail?

When I run this app in terminal of the application in Balena Cloud, I have met an error that there is no such folder or file-libnvToolsExt.so.
So I verified whether this file existed or not in /usr/local/lib64, but I couldn’t find out it.
This means that I didn’t install Cuda or Cudnn correctly? Or unmatched version with Tensorflow?

I hope your help

Best regards
Artem

Langhalsdino · March 8, 2020, 8:20pm

i installed the same version as showed in the repository by BouweCeunen.
JetPack 4.2:

L4T R32.1 (K4.9)
Ubuntu 18.04 LTS aarch64
CUDA 10.0
cuDNN 7.3.1
TensorRT 5.0.6

The main issue is that you can only install a custom compiled version of TF2.0, since there is no precompiled version of TF2.0 for aarch64 with CUDA support. But going this road will take a lot of time.
I would recommend you to use the latest version bellow 2.0. if you are not depending on features of TF2
pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 ‘tensorflow<2’

For a quick install try to use the dockerfiles provided BouweCeunen as reference:

Lastly try to run:
pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 ‘tensorflow<2’

But i do not know if this works, since i use a custom compiled version of TensorFlow 2.0 and only install a wheel with:
RUN pip3 install tensorflow-${TF_VERSION}-cp36-cp36m-linux_aarch64.whl.
Eventhough NVIDIA should provide a TF1.x wheel [here] (Index of /compute/redist/jp/v42/tensorflow-gpu)

Reference:

BouweCeunen/computer-vision-jetson-nano/blob/master/README.md

artem-vasil · March 8, 2020, 9:56pm

Hi, Langhalsdino

Thanks for your quick response

Reading your response, I realized that I needed the compiled wheel of TF2.0.
So could you please share it with me?
If you can do it, I will be thankful for your help.

Best regards
Artem

Langhalsdino · March 8, 2020, 10:48pm

As mentioned above, my TF2.0 wheel is modified a lot and i would not recommend you to use it.

Did you try these wheels provided by NVIDIA?
JP 4.2.x and NV 19.11
TF1.15
TF2.0
JP 4.3.x and installation of NV 19.12
TF 1.15
TF 2.0

Langhalsdino · March 8, 2020, 10:59pm

Sorry i should have read everything closely. This seems to be issue with your cuda, cudnn installation and not with Tensorflow.

If the lib64 file exists, make sure to add its env to the ldconfig and export CUDA path:

RUN echo "/usr/local/cuda/lib64" > /etc/ld.so.conf.d/nvidia-cuda && \
    echo "/usr/local/cuda/extras/CUPTI/lib64" > /etc/ld.so.conf.d/nvidia-cupti && \
    ldconfig

ENV CUDA_HOME=/usr/local/cuda
ENV PATH=$PATH:${CUDA_HOME}/bin

If it does not exist use these Dockerfile as reference to get a working installation of CUDA

artem-vasil · March 8, 2020, 11:10pm

Thanks for your effort.

Then, as mentioned above, should I install Nvinfer and TensorRT surely?
I thought after only installing L4T, CUDA, and CUDNN, I intended to install Tensorflow
Am I right?

Best regards
Artem

Langhalsdino · March 8, 2020, 11:13pm

NvInfere is needed by TensorRT, but both are not needed to run TensorFlow on the JetsonNano. If you want to optimize your NN to run fast on the JetsonNano these libraries might come in handy. Its up to you

Langhalsdino · March 8, 2020, 11:17pm

By the way, if you just want to have a working version of OpenCV, CUDA, … quickly, you can probably use this image as a baseimage and install TF from there as you described

FROM bouwe/jetson-nano-l4t-cuda-cudnn-nvinfer-tensorrt-opencv:latest

artem-vasil · March 8, 2020, 11:42pm

Thanks

Then to install CUDA and CUDNN, I made docker file considering all of three docker files: L4T, CUDA, CUDNN as following:

FROM balenalib/jetson-nano-ubuntu:bionic

ARG DRIVER_PACK=“Jetson-210_Linux_R32.2.1_aarch64.tbz2”
ARG POWER_MODE=0000

COPY packages/$DRIVER_PACK .

RUN apt-get update &&
apt-get install -y --no-install-recommends bzip2 ca-certificates curl lbzip2 sudo htop curl &&
apt-get install -y zip git python3 python3-pip python3-numpy cmake systemd &&
tar -xpj --overwrite -f ./{DRIVER_PACK} && \ sed -i '/.*tar -I lbzip2 -xpmf {LDK_NV_TEGRA_DIR}/config.tbz2./c\tar -I lbzip2 -xpm --overwrite -f {LDK_NV_TEGRA_DIR}\/config.tbz2' ./Linux_for_Tegra/apply_binaries.sh && \ ./Linux_for_Tegra/apply_binaries.sh -r / && \ rm -rf ./Linux_for_Tegra && \ rm ./{DRIVER_PACK} &&
apt-get clean &&
rm -rf /var/lib/apt/lists/ &&
pip3 install jetson-stats

ENV LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra-egl:${LD_LIBRARY_PATH}

RUN ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.32.1.0 /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so &&
ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.32.1.0 /usr/lib/aarch64-linux-gnu/tegra/libnvidia-ptxjitcompiler.so.1 &&
ln -sf /usr/lib/aarch64-linux-gnu/tegra/libGL.so /usr/lib/aarch64-linux-gnu/libGL.so &&
ln -s /usr/lib/aarch64-linux-gnu/libcuda.so /usr/lib/aarch64-linux-gnu/libcuda.so.1 &&
ln -sf /usr/lib/aarch64-linux-gnu/tegra-egl/libEGL.so /usr/lib/aarch64-linux-gnu/libEGL.so

RUN ln -s /etc/nvpmodel/nvpmodel_t210_jetson-nano.conf /etc/nvpmodel.conf &&
ln -s /etc/systemd/system/nvpmodel.service /etc/systemd/system/multi-user.target.wants/nvpmodel.service &&
mkdir /var/lib/nvpmodel &&
echo “/etc/nvpmodel.conf” > /var/lib/nvpmodel/conf_file_path &&
echo “pmode:${POWER_MODE} fmode:fanNull” > /var/lib/nvpmodel/status

ARG CUDA_TOOLKIT=“cuda-repo-l4t-10-0-local-10.0.326”
ARG CUDA_TOOLKIT_PKG="${CUDA_TOOLKIT}_1.0-1_arm64.deb"

COPY packages/$CUDA_TOOLKIT_PKG .

RUN apt-get update &&
apt-get install -y --no-install-recommends curl &&
dpkg --force-all -i {CUDA_TOOLKIT_PKG} && \ rm {CUDA_TOOLKIT_PKG} &&
apt-key add var/cuda-repo--local/.pub &&
apt-get update &&
apt-get install -y --allow-downgrades cuda-toolkit-10-0 libgomp1 libfreeimage-dev libopenmpi-dev openmpi-bin &&
dpkg --purge ${CUDA_TOOLKIT} &&
apt-get clean &&
rm -rf /var/lib/apt/lists/

ENV CUDA_HOME=/usr/local/cuda
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
ENV PATH=$PATH:$CUDA_HOME/bin

ENV CUDNN_VERSION 7.5.0.56
ENV CUDNN_PKG_VERSION={CUDNN_VERSION}-1 LABEL com.nvidia.cudnn.version="{CUDNN_VERSION}"

COPY packages/libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb .
COPY packages/libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb .
COPY packages/libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb .

RUN dpkg -i libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
dpkg -i libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
dpkg -i libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7-dev_$CUDNN_VERSION-1+cuda10.0_arm64.deb &&
rm libcudnn7-doc_$CUDNN_VERSION-1+cuda10.0_arm64.deb

Here, I omitted the header and footer part from original three docker files.
That is to say:

FROM bouwe/jetson-nano-l4t:latest
ENTRYPOINT ["/bin/bash"]

Is this right?

Best regards
Artem

Langhalsdino · March 9, 2020, 9:05am

Looks about right to me.
Did the build process work?

artem-vasil · March 9, 2020, 1:32pm

Yes, I have built the project to the application successfully.
Currently, it updates my device.
But I wonder whether it can be performed on terminal.

So do you think it is not a problem for me to omit the header part of CUDA and CUDNN docker file?

I wonder whether it is right or not.

Thanks for your help and I hope your response.

Best regards
Artem

Langhalsdino · March 9, 2020, 10:38pm

That is right, since Dockerfiles only source from one base image e.g. FROM balenalib/ubuntu-example that might be hosted on hub.docker.com.
(Side note: There are exceptions such as multistage build, but designing multistage builds is a more advanced topic)

Sorry, I did not get your other questions, therefore my answer is a bit more general.

Maybe you can explain a bit more on what you would like to perform on the terminal?

What is docker

Maybe this might help you to understand Dockerfiles a bit better. Docker is a tool that uses containers to make creation, deployment and running of application a lot easier.

It envelopes the application along with all other elements like external libraries and other such dependencies which are required to run the application into a single package smoothly.

A Dockerfile captures the recipe for building a Docker image in writing. A Docker image typically consists of:

A base Docker image on top of which to build your own Docker image
A set of tools and applications to be installed in the Docker image
A set of files to be copied into the Docker image (e.g configuration files)
Possibly opened network ports
etc.

Dockerfile Structure

A Dockerfile consists of a set of instructions. Each instruction consists of a command followed by arguments to that command, similar to command line executables. Here is a simple example of a Dockerfile:

# The base image
FROM ubuntu:latest

# More instructions here that install software
RUN apt-get update && apt-get install git -y
# copy files into the image.
COPY /myapp    /src/myapp

# The command executed when running a Docker container
CMD echo "I am online"

Furthermore the documentation by balena might help you further:

artem-vasil · March 11, 2020, 8:24am

Hi

I have successfully built docker file to the application two days ago.
Then balena application has been updating my device, but until now, it is still updating.
After all, it has been updating for 2 days, I wonder why this happens.

When I built before, it took a few hours at most to update device.
So I can’t test whether it is right or not.

I want your help

Best regards
Artem

Langhalsdino · March 12, 2020, 9:13pm

What kind of internet speed do you have.
The resulting image might be around 10GB.

Can you share the balena logs {UUID} --system ?

artem-vasil · March 13, 2020, 11:54pm

Hi

I am always thankful for your help.
I have pushed my project into Balena application using the following Docker file which is very simple.

FROM bouwe/jetson-nano-l4t-cuda-cudnn-nvinfer-tensorrt-opencv-detectnet:latest

WORKDIR /usr/src/app

COPY ./source .
COPY ./utils .
COPY app.py .
COPY settings.py .

ENV UDEV=1

CMD [“sleep”, “infinity”]

, where the basic image is one of images you recommended me in BouweCeunen/computer-vision-jetson-nano/blob/master/README.md.

It has been successfully pushed into application and my device connected to the applicaiton has also been updated.
By the way, when I connect to my device through the terminal in the Balena cloud, it still happens errors as following.

Error response from daemon: Container 3430d36dfc99a3a957cb46ef2efae37c385f5f9e49fae3f917196e7355c45093 is restarting, wait until the container is running
Error response from daemon: Container 3430d36dfc99a3a957cb46ef2efae37c385f5f9e49fae3f917196e7355c45093 is restarting, wait until the container is running

I wonder why this happens.

Could you please continue to help me?

Best Regards
Artem

Topic		Replies	Views
docker jetson-nano balenaOS dockerfile , jetson , jetpack	15	1936	January 15, 2021
Jetson: Support Nvidia Docker Images balenaOS jetson	38	5074	September 11, 2022
nvidia gstreamer jetson balenaOS docker , jetson	23	6587	February 3, 2021
Jetson Nano image with TensorRT for Python projects balenaOS nano	2	1651	July 13, 2019
Installing CUDA on Jetson base images doesn't work as instructions say it should Product support docker , jetson	5	453	February 20, 2024

Balena OS Jetson Nano Tensorflow

BouweCeunen/computer-vision-jetson-nano/blob/master/README.md

What is docker

Dockerfile Structure

Related topics