camera performance super weird issue.

balenaCloud
#1

Device uuid is efbcb6487f441d485b4e95705f22d462 - granted support access in case you guys wanna take a look.

Alright I have this Raspberrypi4 where we connected 2 cameras via USB. these are the cameras: https://www.alibaba.com/product-detail/ELP-New-black-and-white-AR0144_62268036051.html

The cameras are identical. I am using a very stripped down python script, to grab images from both cameras.

Now i have the weirdest problem: if i connect both cameras - everything is fine. If i swap cameras, same cameras, same ports - 1 camera will work like a pig - grabbing images very slow. If i swap back - they will work fine. It’s a coin toss and i’d like to understand why, because to me it doesn’t make sense. first screenshot is when it’s all working fine, second screenshot - when i swap the cameras.

Does anyone have any ideas? It’s driving me insane!

Screenshot from 2020-04-03 18-39-54
Screenshot from 2020-04-03 18-39-541920×1080 419 KB

Screenshot from 2020-04-03 17-50-34
Screenshot from 2020-04-03 17-50-341920×1080 410 KB

#2

Hi, do both cameras work standalone in both ports?

#4

@katmai do you shutdown the device when you switch the cameras or do you just hotplug them. I would be interested to know if the same results occur if you power down the device and then switch the cameras. It could be that there is something weird going on with udev and the way it enumerates the devices or doesn’t remove others, but I’m just guessing.

#5

Additionally, if you have code you can share, I could try replicate this on my side as i have two usb cams to test with.

#6

The code is dependent on a bunch of env vars. if you want you can add your device in the app and that should make things easier:
https://dashboard.balena-cloud.com/apps/1576396

#7

Hi, here’s the code we’ve been using for the tests:

import cv2
import time

def run_capture_test():
    devices = [cv2.VideoCapture(0), cv2.VideoCapture(2)]
    for device in devices:
        device.set(cv2.CAP_PROP_EXPOSURE, 0.005)
        device.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
        device.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
        device.set(cv2.CAP_PROP_FPS, 2)

    t = time.time()
    while True:
        if time.time() - t > 1.0 / 2:
            t = time.time()

            for i, device in enumerate(devices):
                start_time = time.time()
                device.grab()
                grab_time = time.time() - start_time
                print('cam {} grab time = {}ms'.format(i, int(grab_time * 1000)))

            for i, device in enumerate(devices):
                start_time = time.time()
                device.retrieve()
                retrieve_time = time.time() - start_time
                print('cam {} retrieve time = {}ms'.format(i, int(retrieve_time * 1000)))


if __name__ == "__main__":
    run_capture_test()
#9

Hey @katmai, sorry if I repeat the question but it could be useful to know, do you shutdown the device when you switch the cameras or do you just hotplug them. We would be interested to know if the same results occur if you power down the device and then switch the cameras. It could be that there is something weird going on with udev and the way it enumerates the devices or doesn’t remove others.

#13

Hi @JSReds, we always do a reboot after replugging cameras in these tests.
I’ve also ran the test you suggested, with a complete shutdown before swapping cameras and the result is the same.

#17

@shaunmulligan Hi, any luck replicating the issue with the code i shared?

#18

Hi @katmai,

I’m not currently able to download an image to join your application, but I built a replica based on the opencv repo with your python script. I don’t have all your environment variables added, though. Meanwhile, swapping cameras in my set-up causes no change in retrieve times. Short answer: I’ve not been able to replicate your issue.

I did notice that you’re using Dropbox as a file/folder target. Is that correct? Could there be an issue of the Dropbox folder trying to refresh based on the newly captured images and the subsequent writes impacting the RPi4’s performance? Have you tried this with the Dropbox bits turned off?

John

#20

i see 2 things we can do:
1 - can you please share your dockerfile.template? maybe you set it up different than us.
2 - if you share your id, i can add you to the application as developer and then you’d be able to access the env vars as well as be able to download the image and test that way?

#21

Hi @katmai,

Let’s try #2. Go ahead and add me: g_john_tonello.

John

#22

hi @jtonello - i added you.
for curiosity sake, if possible please do share your dockerfile.template. just wanna peek.

#23

Hi @katmai,

My device is deploying to your app. Here’s the Dockerfile.template I used:

FROM balenalib/%%BALENA_MACHINE_NAME%%-ubuntu

WORKDIR /usr/src/app
RUN install_packages python3-opencv

COPY ./app .

CMD [ "python3", "./capture_test.py" ]

This is the base opencv build… the capture_test.py is your file.

#25

Hi @katmai,

I’ve successfully added my device. Unfortunately, the service won’t start properly. Perhaps there’s a missing environment variable or something that must be set at your end?

John

#27

umm… i am looking in the dashboard and i don’t see your device listed so i can take a look?

#28

about this piece. the reason it’s probably not working, is because by default, opencv version is 3.2. we’re using 3.4.8

#29

also when you add your device to the app - you have to override 2 variables:

MACHINE from UNDEFINED to TestMachine2
and
RUN_MODE from OFF to CAPTURE_TEST

#33

Hi there, sorry that we haven’t been able to identify what might be occurring here. Upon joining the App we still couldnt get this running. It may be worth starting with just a smaller container, as there seemed to be other non openCV issues with the App,
and simplifying the build / enabling features might be a solution path.

#35

hi, we actually managed to give it a huge performance boost by removing some dependencies and using ubuntu 18 aarm64 as container as well as disabling the dropbox uploads. we’re on our way to getting things nicely on track now. thanks for all your help!