Building a nanny bot

Show and Tell
, , , ,
33

JetBot II.

pytorch

Made a mistake last time and the pytorch wheel wasn’t working. The reason was that I forgot specify correct CUDA arch (missing TORCH_CUDA_ARCH_LIST=5.3). It’s fixed now and wheels are available for download at the jetson-nano-1.1 release page.

Don’t use the jetson-nano-1.0 release, it doesn’t work.

balena-nanny-bot/jetbot branch

Changes

  • Add missing GStreamer libraries (NVIDIA)
  • Download & install IMX219-160 Camera overrides
  • Install & properly configure Tegra EGL libraries
  • Utilize docker-compose.yml to be able to add SYS_RAWIO capability

All changes diff.

Deployment

  • Follow the README file instructions

NOTE: It’s still quite big image (not optimised yet).

[Info]     ┌─────────┬────────────┬────────────────────────┐
[Info]     │ Service │ Image Size │ Build Time             │
[Info]     ├─────────┼────────────┼────────────────────────┤
[Info]     │ jetbot  │ 8.57 GB    │ 29 minutes, 26 seconds │
[Info]     └─────────┴────────────┴────────────────────────┘
[Info]     Build finished in 36 minutes, 33 seconds

wander.py

JetBot repository contains wander.py application. Your JetBot just wanders and is trying to avoid collisions.

First terminal:

  • balena ssh $UUID --host
  • nvpmodel -m 1 (switch to 5W mode)
  • exit
  • balena ssh $UUID jetbot
  • nvargus-daemon (keep it running)
root@19c80a8:/usr/src/app# nvargus-daemon 
=== NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...

Second terminal:

  • balena ssh $UUID jetbot
  • Copy the best_model.pth from your computer (scp ...)
  • python3 jetbot/jetbot/apps/wander.py ./best_model.pth
root@19c80a8:/usr/src/app# python3 jetbot/jetbot/apps/wander.py ./best_model.pth 
Loading model...
Initializing camera...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: PowerService: requested_clock_Hz=37126320
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Running...

It’s not user friendly, I know, I know, … but at least it works properly now!

Will also expose Python notebooks, so, we can train the JetBot and do everything what’s explained at the examples page.