Unable to read from CSI camera on Raspberry Pi 4

Product support
1

I’ve got an Innomaker CSI camera attached to the camera connector on the Pi 4 but am unable to read frames from it.

On the face of it everything looks normal but trying to read frames blocks indefinitely. The following shows a minimal succession of commands to illustrate the issue along with some debugging information output.

I’ve double and triple checked that the cable is connected and attached properly and in the right slot. The camera is detected as can be seen in the output of vcgencmd and v4l2-ctl.

I’ve experimented with different gpu_mem settings to no avail. start_x is set.

The proper kernel modules are loaded. My container is running in privileged mode. The BalenaOS is balenaOS 2.51.1+rev1.

Please advice.

Summary 
% balena ssh 9e0d3b7f80216b3c1e1b9b852e037131
=============================================================
    Welcome to balenaOS
=============================================================
root@9e0d3b7:~# uname -a
Linux 9e0d3b7 4.19.75 #1 SMP PREEMPT Thu Jun 4 14:57:09 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
root@9e0d3b7:~# vcgencmd get_camera
supported=1 detected=1
root@9e0d3b7:~# cat /mnt/boot/config.txt
gpu_mem=192
start_x=1
arm_64bit=1
avoid_warnings=1
dtoverlay=vc4-fkms-v3d
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
gpu_mem_1024=256
gpu_mem_256=192
gpu_mem_512=192
root@9e0d3b7:~# lsmod
Module                  Size  Used by
ip6t_REJECT            16384  1
nf_reject_ipv6         16384  1 ip6t_REJECT
ipt_REJECT             16384  1
nf_reject_ipv4         16384  1 ipt_REJECT
xt_state               16384  0
ip6table_filter        16384  1
ip6_tables             28672  1 ip6table_filter
ipt_MASQUERADE         16384  3
nf_conntrack_netlink    49152  0
nfnetlink              16384  2 nf_conntrack_netlink
bnep                   20480  2
br_netfilter           24576  0
hci_uart               45056  1
btbcm                  16384  1 hci_uart
serdev                 20480  1 hci_uart
bluetooth             438272  24 hci_uart,btbcm,bnep
ecdh_generic           28672  1 bluetooth
xt_owner               16384  0
i2c_dev                20480  0
brcmfmac              319488  0
spidev                 20480  0
brcmutil               20480  1 brcmfmac
sha256_generic         20480  0
bcm2835_codec          40960  0
bcm2835_v4l2           49152  1
v4l2_mem2mem           28672  1 bcm2835_codec
v4l2_common            16384  1 bcm2835_v4l2
bcm2835_mmal_vchiq     32768  2 bcm2835_codec,bcm2835_v4l2
cfg80211              729088  1 brcmfmac
videobuf2_vmalloc      16384  1 bcm2835_v4l2
videobuf2_dma_contig    20480  1 bcm2835_codec
videobuf2_memops       16384  2 videobuf2_vmalloc,videobuf2_dma_contig
videobuf2_v4l2         28672  3 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem
vc4                   200704  0
videobuf2_common       49152  4 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem
rfkill                 32768  5 bluetooth,cfg80211
v3d                    73728  0
videodev              249856  7 bcm2835_codec,v4l2_common,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem
raspberrypi_hwmon      16384  0
snd_soc_core          217088  1 vc4
hwmon                  20480  1 raspberrypi_hwmon
media                  40960  3 videodev,bcm2835_codec,v4l2_mem2mem
gpu_sched              28672  1 v3d
snd_bcm2835            28672  0
snd_compress           20480  1 snd_soc_core
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_pcm               126976  4 vc4,snd_bcm2835,snd_soc_core,snd_pcm_dmaengine
snd_timer              40960  1 snd_pcm
snd                    86016  5 snd_bcm2835,snd_timer,snd_compress,snd_soc_core,snd_pcm
vc_sm_cma              36864  1 bcm2835_mmal_vchiq
spi_bcm2835            20480  0
rpivid_mem             16384  0
uio_pdrv_genirq        16384  0
uio                    20480  1 uio_pdrv_genirq
fixed                  16384  0
sch_fq_codel           20480  8
root@9e0d3b7:~# balena run --rm -it --privileged alpine
/ # apk add -U v4l-utils
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/aarch64/APKINDEX.tar.gz
(1/69) Installing libffi (3.2.1-r6)
(2/69) Installing libgcc (9.2.0-r4)
(3/69) Installing libstdc++ (9.2.0-r4)
(4/69) Installing xz-libs (5.2.4-r0)
(5/69) Installing libxml2 (2.9.10-r3)
(6/69) Installing llvm9-libs (9.0.0-r1)
(7/69) Installing libdrm (2.4.100-r0)
(8/69) Installing elfutils-libelf (0.168-r2)
(9/69) Installing expat (2.2.9-r1)
(10/69) Installing mesa-glapi (19.2.7-r0)
(11/69) Installing mesa (19.2.7-r0)
(12/69) Installing libxau (1.0.9-r0)
(13/69) Installing libbsd (0.10.0-r0)
(14/69) Installing libxdmcp (1.1.3-r0)
(15/69) Installing libxcb (1.13.1-r0)
(16/69) Installing libx11 (1.6.9-r0)
(17/69) Installing libxdamage (1.1.5-r0)
(18/69) Installing libxext (1.3.4-r0)
(19/69) Installing libxfixes (5.0.3-r2)
(20/69) Installing libxxf86vm (1.1.4-r2)
(21/69) Installing libxshmfence (1.3-r0)
(22/69) Installing mesa-gl (19.2.7-r0)
(23/69) Installing dbus-libs (1.12.16-r3)
(24/69) Installing libintl (0.20.1-r2)
(25/69) Installing libblkid (2.34-r1)
(26/69) Installing libmount (2.34-r1)
(27/69) Installing pcre (8.43-r0)
(28/69) Installing glib (2.62.6-r0)
(29/69) Installing icu-libs (64.2-r1)
(30/69) Installing libpcre2-16 (10.34-r1)
(31/69) Installing qt5-qtbase (5.12.5-r0)
(32/69) Installing hicolor-icon-theme (0.17-r1)
(33/69) Installing wayland-libs-server (1.17.0-r0)
(34/69) Installing mesa-gbm (19.2.7-r0)
(35/69) Installing wayland-libs-client (1.17.0-r0)
(36/69) Installing mesa-egl (19.2.7-r0)
(37/69) Installing libice (1.0.10-r0)
(38/69) Installing libuuid (2.34-r1)
(39/69) Installing libsm (1.2.3-r0)
(40/69) Installing libxrender (0.9.10-r3)
(41/69) Installing avahi-libs (0.7-r4)
(42/69) Installing gmp (6.1.2-r1)
(43/69) Installing nettle (3.5.1-r0)
(44/69) Installing p11-kit (0.23.18.1-r0)
(45/69) Installing libtasn1 (4.15.0-r0)
(46/69) Installing libunistring (0.9.10-r0)
(47/69) Installing gnutls (3.6.14-r0)
(48/69) Installing cups-libs (2.2.12-r1)
(49/69) Installing libbz2 (1.0.8-r1)
(50/69) Installing libpng (1.6.37-r1)
(51/69) Installing freetype (2.10.1-r0)
(52/69) Installing fontconfig (2.13.1-r2)
(53/69) Installing graphite2 (1.3.13-r1)
(54/69) Installing harfbuzz (2.6.4-r0)
(55/69) Installing libevdev (1.8.0-r0)
(56/69) Installing mtdev (1.1.5-r2)
(57/69) Installing eudev-libs (3.2.9-r1)
(58/69) Installing libinput-libs (1.14.3-r0)
(59/69) Installing libjpeg-turbo (2.0.4-r0)
(60/69) Installing xcb-util-wm (0.4.1-r1)
(61/69) Installing xcb-util (0.4.0-r1)
(62/69) Installing xcb-util-image (0.4.0-r1)
(63/69) Installing xcb-util-keysyms (0.4.0-r1)
(64/69) Installing xcb-util-renderutil (0.3.9-r1)
(65/69) Installing libxkbcommon (0.9.1-r0)
(66/69) Installing libxkbcommon-x11 (0.9.1-r0)
(67/69) Installing qt5-qtbase-x11 (5.12.5-r0)
(68/69) Installing v4l-utils-libs (1.18.0-r0)
(69/69) Installing v4l-utils (1.18.0-r0)
Executing busybox-1.31.1-r9.trigger
OK: 158 MiB in 83 packages
/ # v4l2-ctl --all
Driver Info:
    Driver name      : bm2835 mmal
    Card type        : mmal service 16.1
    Bus info         : platform:bcm2835-v4l2
    Driver version   : 4.19.75
    Capabilities     : 0x85200005
        Video Capture
        Video Overlay
        Read/Write
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x05200005
        Video Capture
        Video Overlay
        Read/Write
        Streaming
        Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
    Width/Height      : 640/480
    Pixel Format      : 'H264' (H.264)
    Field             : None
    Bytes per Line    : 0   
    Size Image        : 307200
    Colorspace        : SMPTE 170M
    Transfer Function : Default (maps to Rec. 709)
    YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    Quantization      : Default (maps to Full Range)
    Flags             :
Format Video Overlay:
    Left/Top    : 150/50
    Width/Height: 1024/768
    Field       : None
    Chroma Key  : 0x00000000
    Global Alpha: 0xff
    Clip Count  : 0
    Clip Bitmap : No
Framebuffer Format:
    Capability    : Extern Overlay
            Global Alpha
    Flags         : Overlay Matches Capture/Output Size
    Width         : 640
    Height        : 480
    Pixel Format  : 'YU12'
Streaming Parameters Video Capture:
    Capabilities     : timeperframe
    Frames per second: 10.000 (10/1)
    Read buffers     : 1
User Controls
                     brightness 0x00980900 (int)    : min=0 max=100 step=1 default=50 value=50 flags=slider
                       contrast 0x00980901 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                     saturation 0x00980902 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                    red_balance 0x0098090e (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                   blue_balance 0x0098090f (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=3 default=1 value=1
                0: Disabled
                1: 50 Hz
                2: 60 Hz
                3: Auto
                      sharpness 0x0098091b (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                  color_effects 0x0098091f (menu)   : min=0 max=15 default=0 value=0
                0: None
                1: Black & White
                2: Sepia
                3: Negative
                4: Emboss
                5: Sketch
                6: Sky Blue
                7: Grass Green
                8: Skin Whiten
                9: Vivid
                10: Aqua
                11: Art Freeze
                12: Silhouette
                13: Solarization
                14: Antique
                15: Set Cb/Cr
                         rotate 0x00980922 (int)    : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
             color_effects_cbcr 0x0098092a (int)    : min=0 max=65535 step=1 default=32896 value=32896
Codec Controls
             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=0 value=0 flags=update
                0: Variable Bitrate
                1: Constant Bitrate
                  video_bitrate 0x009909cf (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000
         repeat_sequence_header 0x009909e2 (bool)   : default=0 value=0
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level 0x00990a67 (menu)   : min=0 max=11 default=11 value=11
                0: 1
                1: 1b
                2: 1.1
                3: 1.2
                4: 1.3
                5: 2
                6: 2.1
                7: 2.2
                8: 3
                9: 3.1
                10: 3.2
                11: 4
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4
                0: Baseline
                1: Constrained Baseline
                2: Main
                4: High
Camera Controls
                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=0 value=0
                0: Auto Mode
                1: Manual Mode
         exposure_time_absolute 0x009a0902 (int)    : min=1 max=10000 step=1 default=1000 value=1000
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=0
             auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
                0: -4000 (0xfffffffffffff060)
                1: -3667 (0xfffffffffffff1ad)
                2: -3333 (0xfffffffffffff2fb)
                3: -3000 (0xfffffffffffff448)
                4: -2667 (0xfffffffffffff595)
                5: -2333 (0xfffffffffffff6e3)
                6: -2000 (0xfffffffffffff830)
                7: -1667 (0xfffffffffffff97d)
                8: -1333 (0xfffffffffffffacb)
                9: -1000 (0xfffffffffffffc18)
                10: -667 (0xfffffffffffffd65)
                11: -333 (0xfffffffffffffeb3)
                12: 0 (0x0)
                13: 333 (0x14d)
                14: 667 (0x29b)
                15: 1000 (0x3e8)
                16: 1333 (0x535)
                17: 1667 (0x683)
                18: 2000 (0x7d0)
                19: 2333 (0x91d)
                20: 2667 (0xa6b)
                21: 3000 (0xbb8)
                22: 3333 (0xd05)
                23: 3667 (0xe53)
                24: 4000 (0xfa0)
      white_balance_auto_preset 0x009a0914 (menu)   : min=0 max=10 default=1 value=1
                0: Manual
                1: Auto
                2: Incandescent
                3: Fluorescent
                4: Fluorescent H
                5: Horizon
                6: Daylight
                7: Flash
                8: Cloudy
                9: Shade
                10: Greyworld
            image_stabilization 0x009a0916 (bool)   : default=0 value=0
                iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
                0: 0 (0x0)
                1: 100000 (0x186a0)
                2: 200000 (0x30d40)
                3: 400000 (0x61a80)
                4: 800000 (0xc3500)
           iso_sensitivity_auto 0x009a0918 (menu)   : min=0 max=1 default=1 value=1
                0: Manual
                1: Auto
         exposure_metering_mode 0x009a0919 (menu)   : min=0 max=2 default=0 value=0
                0: Average
                1: Center Weighted
                2: Spot
                     scene_mode 0x009a091a (menu)   : min=0 max=13 default=0 value=0
                0: None
                8: Night
                11: Sports
JPEG Compression Controls
            compression_quality 0x009d0903 (int)    : min=1 max=100 step=1 default=30 value=30
/ # v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture
    [0]: 'YU12' (Planar YUV 4:2:0)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [1]: 'YUYV' (YUYV 4:2:2)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [2]: 'RGB3' (24-bit RGB 8-8-8)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [3]: 'JPEG' (JFIF JPEG, compressed)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [4]: 'H264' (H.264, compressed)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [5]: 'MJPG' (Motion-JPEG, compressed)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [6]: 'YVYU' (YVYU 4:2:2)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [7]: 'VYUY' (VYUY 4:2:2)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [8]: 'UYVY' (UYVY 4:2:2)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [9]: 'NV12' (Y/CbCr 4:2:0)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [10]: 'BGR3' (24-bit BGR 8-8-8)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [11]: 'YV12' (Planar YVU 4:2:0)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [12]: 'NV21' (Y/CrCb 4:2:0)
        Size: Stepwise 32x32 - 2592x1944 with step 2/2
    [13]: 'BGR4' (32-bit BGRA/X 8-8-8-8)
/ # v4l2-ctl -v width=640,height=480,pixelformat=H264
/ # v4l2-ctl --stream-mmap=3 --stream-count=2 --stream-to=/dev/null     # reading 2 frames (reads one then blocks)
<
4

Hey,
Thank you for contacting us! Have you tried to stream to another location other than /dev/null? This seems more related to that specific library than balena.

6

Hi @odyslam,

I’ve tried that, yes. It does not work. The other supported formats does not work either. Any other ideas? I’m happy to grant support access if that would help.

It’s not necessarily Balena specific but I don’t know how to peel off any more moving parts or layers. v4l2-ctl is making direct ioctl syscalls to the kernel driver through the V4L Linux subsystem. I would’ve thought this is what the picamera library does under the hood too but I might be wrong on that. I’m pretty sure it uses the same driver at least.

The Balena hello world of reading video seems to have open issues that might be related but I’m unsure on that too.

Can you provide example code that you would expect to work? Installing ffmpeg to read video it has the same behavior, blocking indefinitely, but then again it is using V4L under the hood.

Since Balena does provides the kernel and drivers I’m thinking that might be where the issue is.

It could also be the hardware. Unfortunately I only have access to one camera and the global supply chain is unable to provide more cameras at the moment because of COVID19.

7

Hi @hnnsgstfssn – thanks for the additional details. I’m curious about a couple of things:

  • Are you able to share your Dockerfile/docker-compose.yml? It would be good to see which capabilities it has.
  • Are you able to try this on a Raspberry Pi 3 with balenaOS? That might help figure out whether the issue you linked is related (though we’ll need to keep in mind differences in hardware).
  • Are you able to try this with plain Raspbian? That might help determine whether it’s a problem with your camera, or with balenaOS.

Thanks again, and let us know how you get on.

Hugh

9

Hi @saintaardvark,

I could put a few steps into a compose and Dockerfile and push that to an application. However the steps I’ve included previously are run directly on the device after having logged in with balena ssh. They run a container based on the alpine image and then manually run the steps that would be put in a potential Dockerfile. In my mind that should be equivalent to what you’re asking for. I’ve run the container with --privileged, does that not mean it has all the capabilities?

Are you able to try this on a Raspberry Pi 3 with balenaOS? That might help figure out whether the issue you linked is related (though we’ll need to keep in mind differences in hardware).

Unfortunately I only have two Raspberry Pi 4 boards but no 3 ones at the moment. There is a lot of (conflicting, confusing, and unclear) information out there that indicates the Pi 4 still has unresolved issues and where using the Pi 3 would be a temporary fix. I would struggle to compile an exhaustive list of this though, and frankly, I want the increased power of the Pi 4.

Are you able to try this with plain Raspbian? That might help determine whether it’s a problem with your camera, or with balenaOS.

I could try that, but what is it that is different between BalenaOS and Raspbian here? I’ll give it a quick go as I have a spare Pi 4. This could potentially give information about the working state of the camera hardware. However, would it help if I said I am able to read a single frame (H264 using the same command as above but with --stream-count=1) successfully. It hangs when reading 2 frames or more.

10

Hi @hnnsgstfssn – thanks for confirming how you’re running this. Running it with --privileged should give it all the capabilities, yes. For debugging and repeatability, I often find it easier to create a docker-compose.yml file, even if it is throwaway, and use it with balena push. Looking at balena-cam docker-compose.yml file may give you some ideas.

balenaOS and Raspbian are two different Linux distributions. There are some differences in kernel versions, firmware, and so on. It might help to see if it works on Raspbian, and at least see whether those differences are associated with a difference in behaviour.

Thanks for the additional detail about --stream-count=1 working, whereas reading 2 or more frames doesn’t. That makes me wonder if it’s a problem with the driver (or v4l2-ctl/ffmpeg) not doing the right thing between multiple frame reads, but I’m speculating. You may want to try seeing if the syscalls are different between a single frame read and multiple frame reads.

Let us know what you find out!

All the best,
Hugh

12

I’ve now tried this on a stock Raspberry Pi OS (32-bit) Lite. It has the same problem.

13

Hi @hnnsgstfssn,

Getting the same problem on the Raspbian suggests a driver issue. This device spec from Innomaker may not be for your exact camera, but it doesn’t explicitly confirm support for the RPi4. What camera model are you using?

John

15

Hi @jtonello,

I’ve ordered off this Amazon link. I would guess it is OV5647 Camera Module
 but I can honestly not tell with any certainty.

I have managed to get my hands on another camera meeting my requirements that works with zero changes so I’ve decided to give up on this completely, chalking it up to faulty hardware.

Given the dodgy Pi 4 support I’m experiencing overall I’ll probably be looking into other options for the device hardware as well, but that’s another rabbit hole for another day.

Anway, thanks for your input!

16

Thanks for letting us know!

John

1 Like
19

I’ve been recording videos from two MIPI-CSI OV5647 cameras using the libcamera library on a Raspberry Pi CM4. Some configurations I’ve done:

1) In /mnt/boot/config.txt, set the start_x variable.
The start_x variant is the full version of the firmware that includes hardware video decoding support and enables hardware-accelerated video decoding (H.264), which is essential for multimedia applications. You can achieve this by adding the following configuration to your fleet:

BALENA_HOST_CONFIG_start_x 1

2) In /mnt/boot/config.txt, add your MIPI-CSI camera device tree.
In my case, I have specified one device tree per MIPI-CSI channel. You can configure this in your fleet using Balena Cloud in the “Define DT overlays” configuration. In my case:

"ov5647,cam0","ov5647,cam1"

3) In the Docker Compose file, ensure the I2C devices are mounted inside your service container.

services:
  <your-service-name>:
    build: ./
    devices:
      - "/dev/i2c-0:/dev/i2c-0"
      - "/dev/i2c-1:/dev/i2c-1"
      - "/dev/i2c-10:/dev/i2c-10"
      - "/dev/i2c-22:/dev/i2c-22"
    privileged: true

4) Check if your cameras are detected. If you have libcamera installed, you can use the following command:

$ libcamera-hello --lists 
Available cameras:
-------------------
Available cameras
-----------------
0 : ov5647 [4608x2592] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [120.00 fps - (0, 0)/640x480 crop]
                             1280x720 [60.00 fps - (0, 0)/1280x720 crop]
                             1920x1080 [30.00 fps - (0, 0)/1920x1080 crop]
                             2592x1944 [15.00 fps - (0, 0)/2592x1944 crop]
                             4608x2592 [10.00 fps - (0, 0)/4608x2592 crop]
1 : ov5647 [4608x2592] (/base/soc/i2c0mux/i2c@0/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [120.00 fps - (0, 0)/640x480 crop]
                             1280x720 [60.00 fps - (0, 0)/1280x720 crop]
                             1920x1080 [30.00 fps - (0, 0)/1920x1080 crop]
                             2592x1944 [15.00 fps - (0, 0)/2592x1944 crop]
                             4608x2592 [10.00 fps - (0, 0)/4608x2592 crop]