Support x86_64. Video analyze services (Not IoT)

limpro · July 6, 2020, 10:27am

Hi Rahul,

Thanks for your comment. So which device can I choose from https://www.balena.io/docs/reference/base-images/devicetypes/ ?

Thanks, LimPro

rahul-thakoor · July 6, 2020, 10:28am

Also thanks for the feedback. Generic x86_64 is a new device type are we will be updating the docs and download links here(https://www.balena.io/os/#download) soon

rahul-thakoor · July 6, 2020, 10:31am

hey @limpro,

I don’t quite follow,

So which device can I choose from https://www.balena.io/docs/reference/base-images/devicetypes/ ?

For your use case, you should use the genericx86-64-ext device type which you mentioned you were able to download above. However, it is not (yet) listed here: https://www.balena.io/docs/reference/base-images/devicetypes/

limpro · July 14, 2020, 9:41am

Hello Balena Team,

Looking for two helps~

First one, Is there any guide to install CUDA on BalenaOS x86_64?
I referred https://www.balena.io/blog/getting-started-with-the-nvidia-jetson-nano-using-balena/
But failed with genericx86-64-ext devices.

Second one,
Regarding Docker-compose on Balena, do you support --shm-size and --gpus options?
I found that Balena seems took https://github.com/docker/compose/issues/6691 in recent few months.
However, these options are not working.

thundron · July 14, 2020, 9:39am

Hi @limpro, some questions to understand better:

At what stage of the provisioning process does the error happen?
What’s the error that you see when it fails?

For your second question, I’m not sure what you refer to with that issue, but you should be able to use the shm_size property as documented here along with the other listed properties. If they’re not listed, they’re most likely not available yet

limpro · July 14, 2020, 9:52am

Hi @thundron,

To verify whether balena can support GPU available container(nvidia/cuda:10.1-base), I ran below command in the device (OS Ver.: balenaOS 2.48.0+rev5 / Device Type: genericx86-64-ext)

balena-engine run --gpus all,capabilities=utility nvidia/cuda:10.1-base nvidia-smi

Please refer attached screenshot

srlowe · July 14, 2020, 10:28am

Hello, I don’t see the attachment in our system. Please could you try attaching again?

limpro · July 14, 2020, 10:30am

Hello @srlowe,

Can you see my screenshot?

srlowe · July 14, 2020, 10:54am

Yes, I have it now thank you. Looking into this.

limpro · July 14, 2020, 2:04pm

Hi Balena Team,

My project needs to deploy AI services on GPU machines(x86_64, Nvidia GPU). I am considering to use Balena for the project(GPU machines as balenaDevices). So it is critical that CUDA enabled docker can run on balenaDevice.

Could you help me figure out whether GPU inference(using Nvidia Docker) can run on balenaDevcie(x86_64), please?

mbalamat · July 14, 2020, 2:21pm

Just to clarify your device is a computer with a GPU. If so can you send us the GPU that you’ll use?

limpro · July 14, 2020, 2:34pm

=> YES. Support x86_64. Video analyze services (Not IoT)

Hi @mbalamat

If so can you send us the GPU that you’ll use?

=> Do you ask me to send GPU machine to you? Sorry it cannot be.

Balena team, could you help me find answers to below simple questions, please? Then my project can go forward

Is there any guide to install CUDA on BalenaOS x86_64?
Could you help me figure out whether GPU inference(using Nvidia Docker) can run on balenaDevcie( x86_64 ), please?

Thank you! LimPro

mbalamat · July 14, 2020, 2:46pm

Excuse me, sorry I meant to send us the name of the GPU you’ll use eg GTX1050, not the physical GPU. I have routed your questions to the devices and balena-engine teams, we’ll get back to you as soon as possible. In the meantime, if it’s ok with you, it would be great to tell us more about what hardware setup (GPU model) are you planning to use.

limpro · July 14, 2020, 2:56pm

Oh haha. sorry. Currently, my developing machine has two GTX 1080Ti but I will use GTX2080 also in near future.
The number of GPUs can be varied(depends on each project situation). Some machines have 3 GPUs or just one.

mbalamat · July 14, 2020, 3:07pm

the image you mentioned above nvidia/cuda:10.1-base is it able to handle multiple GPUs?

limpro · July 15, 2020, 8:11am

the image you mentioned above nvidia/cuda:10.1-base is it able to handle multiple GPUs?

Yes, it can access one and/or all GPUs

rahul-thakoor · July 15, 2020, 10:30am

Thanks for the feedback.
We will get back to you once we have more details regarding using --gpus with balena engine.

limpro · July 17, 2020, 3:34am

Hello Belena team,

I am looking for the answers

Thank you! LimPro

kaisoz · July 17, 2020, 8:18am

Hi!

Excuses for the delay, we have forwarded your question to the engine maintainer but he is not around at the moment, he should be able to have a look at your questions next week

Thanks!

nazrhom · July 20, 2020, 2:38pm

Hello, we recently added support for --gpus in the engine, set through the compose label io.balena.features.gpu. Unfortunately this still requires some work, as the CUDA drivers and some other libraries need to be installed on the hostOS to get it all working. We don’t have a guide for that yet, but I am trying to gather some info from the device team to pass on to you