# Rebooting device causes devices to disappear **URL:** \ **Category:** balenaOS\ **Created:** [July 13, 2020, 9:28pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285 "2020-07-13T21:28:58Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 13, 2020, 9:28pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/1 "2020-07-13T21:28:58Z")
## Context **Every morning my devices reboot in order to reestablish their cellular connection.** _(this may be an issue in itself. Perhaps I should try to solve the problem with the cell connection rather than rebooting, but I’ll still need to occasionally reboot when all other cell connection restoration steps have been tried, so this thread is still relevant.)_ The issue began when I upgraded from [balenaOS 2.31.5+rev1](https://github.com/balena-os/meta-balena/blob/master/CHANGELOG.md) to [balenaOS 2.45.1+rev1](https://github.com/balena-os/meta-balena/blob/master/CHANGELOG.md). I’m working on getting my application to work on higher OS versions so downgrading is not a great option. ## Problem **When the device reboots, the first container that it starts doesn’t have access to the `/dev/video0` device**. I thought this was because it didn’t have enough time, but even after adding an infinite wait loop that merely checks if `/dev/video0` is available it still hangs indefinitely. Restarting the container solves the problem. In other words, this is only an issue for the first container that’s started right after reboot. ## Replicating the Problem All you need to do to see the problem in action is **reboot this test device:** [https://dashboard.balena-cloud.com/devices/b9e8b35bb8569a31a4b225b41764f5fa/summary](https://dashboard.balena-cloud.com/devices/b9e8b35bb8569a31a4b225b41764f5fa/summary) I have granted support access to it. You should expect to see: ```nohighlight Device /dev/video0 not found Device /dev/video0 not found Device /dev/video0 not found ... Device /dev/video0 not found Device /dev/video0 not found ``` ## Non-solutions A potential solution is: ```php if "/dev/video0" is connected: run startup script else: kill and restart container ``` I don’t want to do this because: 1. It’s not the right way to solve the problem—this isn’t happening on older OSs and I shouldn’t need to restart a container to get it to connect to a device that’s already there. 2. I am trying to leave my `docker-compose.yml` with setting `restart: "no"` so that I can get emails when devices aren’t working and ssh into the container to debug it with its (broken) state maintained. ---
**Author:** ![jimsynz](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/jimsynz/32/4663_2.png) [@jimsynz](https://forums.balena.io/u/jimsynz)\ **Post date:** [July 13, 2020, 9:42pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/3 "2020-07-13T21:42:30Z")
Hi @cnr, I’ve just restarted the device to see if I can replicate it. From what I can tell the container is started before the USB bus has finished enumerating the devices. After reboot I was able to shell in and watch it, and the video devices aren’t present until at least 30 seconds after the container has started. I’ll check with our balenaOS team and see if I can get any insight into what’s happening. Cheers, James. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 13, 2020, 9:55pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/4 "2020-07-13T21:55:24Z")
Thank you! ---
**Author:** ![jimsynz](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/jimsynz/32/4663_2.png) [@jimsynz](https://forums.balena.io/u/jimsynz)\ **Post date:** [July 14, 2020, 9:00pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/7 "2020-07-14T21:00:55Z")
Hi again @cnr, I’ve had a response from the team and the suggestion is rather than delaying your service from starting before the USB device is available, to start the service with UDEV enabled so that you can detect the camera being “connected”. Here’s a thread discussing doing a very similar thing: [Docker container cannot access dynamically plugged USB devices](https://forums.balena.io/t/docker-container-cannot-access-dynamically-plugged-usb-devices/4277/13) ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 14, 2020, 9:39pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/9 "2020-07-14T21:39:47Z")
Hi, Thank you for this information. I do not use a `balenalib` base container, so the `UDEV=1` solution will be insufficient for me. Should I add [balenalib’s entry.sh](https://github.com/balena-io-library/base-images/blob/28844485a91b1408ffc550faa3b59e64809bc453/scripts/assets/entry.sh#L64-L68) to my own entry script to make my container dynamically plugged? ---
**Author:** ![samothx](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@samothx](https://forums.balena.io/u/samothx)\ **Post date:** [July 14, 2020, 10:07pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/10 "2020-07-14T22:07:09Z")
Hi Connor, that might be worth a try. You will also have to look into the dockerfile for the OS you are using, to install dependencies. I just looked into this one: [https://github.com/balena-io-library/base-images/blob/28844485a91b1408ffc550faa3b59e64809bc453/balena-base-images/amd64/ubuntu/bionic/build/Dockerfile](https://github.com/balena-io-library/base-images/blob/28844485a91b1408ffc550faa3b59e64809bc453/balena-base-images/amd64/ubuntu/bionic/build/Dockerfile) . I would think at lease udev would have to be installed in the container but you might need other packages. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 14, 2020, 10:11pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/11 "2020-07-14T22:11:33Z")
Gotcha, I’ll start iterating on that. In the meantime, can you give me the highlevel explanation on why this happens? Why is it that the OS doesn’t wait for devices to be loaded before starting the first container, but then by the time a second container comes up it has no trouble passing in the device? ---
**Author:** ![samothx](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@samothx](https://forums.balena.io/u/samothx)\ **Post date:** [July 14, 2020, 10:43pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/12 "2020-07-14T22:43:40Z")
For one thing as James mentioned above, the device shows up very lately and USB devices could show up at any time as they can be plugged in dynamically. I do not know if it makes sense or is even possible to wait for USB device enumeration to be complete before starting containers. Looking at a privileged container in ‘normal’ (non balena ) docker I can see that changes to the device file system (like plugging in a new USB disk) generally do not update the /dev folder in a running container. I guess you need UDEV running in the container for that to happen. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 20, 2020, 7:24pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/14 "2020-07-20T19:24:12Z")
This didn’t work. Do you have any hints for getting the hotplugging devices in my container? ---
**Author:** ![ntzovanis](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/ntzovanis/32/1150_2.png) [@ntzovanis](https://forums.balena.io/u/ntzovanis)\ **Post date:** [July 20, 2020, 8:41pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/16 "2020-07-20T20:41:32Z")
Hello @cnr , Did you try enabling UDEV and it didn’t work for you? Could you explain a bit more what you tried and which step failed? ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 20, 2020, 8:42pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/17 "2020-07-20T20:42:32Z")
Hi! As I said here, > [@cnr](#): > > I do not use a `balenalib` base container, so the `UDEV=1` solution will be insufficient for me. I have to implement manually ---
**Author:** ![ntzovanis](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/ntzovanis/32/1150_2.png) [@ntzovanis](https://forums.balena.io/u/ntzovanis)\ **Post date:** [July 20, 2020, 9:04pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/18 "2020-07-20T21:04:35Z")
@cnr have you tried running the container in privileged mode and adding `UDEV=1` ? That would be the first step to see if video0 gets mounted while the container is running. Cheers, Nico. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 20, 2020, 9:08pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/19 "2020-07-20T21:08:04Z")
Maybe I’m confused. The `UDEV=1` option is for docker containers based on the `balenalib` base images, right? ---
**Author:** ![jimsynz](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/jimsynz/32/4663_2.png) [@jimsynz](https://forums.balena.io/u/jimsynz)\ **Post date:** [July 20, 2020, 9:13pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/20 "2020-07-20T21:13:26Z")
Hi @cnr, Yes, setting `UDEV=1` only works for balenalib-based containers because of their configuration. Is there a reason you are unable to use a balenalib base image? You’re right that it is possible to replicate the udev behaviour of our base images. Cheers, James. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 20, 2020, 9:19pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/21 "2020-07-20T21:19:42Z")
Yeah I had to avoid the balenalib base images for my build because I needed to support some Nvidia specific stuff. I’ve now had the image for a while and upgrading to a balenalib base image won’t be the solution I employ for this fix. Could you please share any steps and resources you have for replicating the udev behavior of the balenalib base images? ---
**Author:** ![jimsynz](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/jimsynz/32/4663_2.png) [@jimsynz](https://forums.balena.io/u/jimsynz)\ **Post date:** [July 20, 2020, 9:39pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/22 "2020-07-20T21:39:07Z")
I’ll ask one of the base images team to jump in here and see if they can provide you some pointers. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 20, 2020, 9:57pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/24 "2020-07-20T21:57:04Z")
Great thank you! ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 21, 2020, 9:29pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/26 "2020-07-21T21:29:56Z")
Would it be helpful if I started a new thread so that 1. The question can be more relevant to the topic at hand 2. More people can find the thread more easily? ---
**Author:** ![zvin](https://avatars.discourse-cdn.com/v4/letter/z/a5b964/32.png) [@zvin](https://forums.balena.io/u/zvin)\ **Post date:** [July 22, 2020, 11:23am UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/28 "2020-07-22T11:23:44Z")
Hello @cnr You need to replicate this in your container entrypoint: [https://github.com/balena-io-library/base-images/blob/28844485a91b1408ffc550faa3b59e64809bc453/balena-base-images/amd64/ubuntu/bionic/build/entry.sh#L36-L42](https://github.com/balena-io-library/base-images/blob/28844485a91b1408ffc550faa3b59e64809bc453/balena-base-images/amd64/ubuntu/bionic/build/entry.sh#L36-L42) - start `udevd` or `systemd-udevd` depending on which one you have available; - run `udevadm trigger` You can check if `udevd` is running by running `udevadm monitor` and plugging or unplugging some usb devices. ---
**Author:** ![cnr](https://avatars.discourse-cdn.com/v4/letter/c/d78d45/32.png) [@cnr](https://forums.balena.io/u/cnr)\ **Post date:** [July 22, 2020, 5:38pm UTC](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285/30 "2020-07-22T17:38:48Z")
Great! This link also would have helped me > [@Doc correction: UDEV and PRIVILEGED in single container app](https://forums.balena.io/t/doc-correction-udev-and-privileged-in-single-container-app/22596): > > Hello, I had been reviewing the docs and I found a couple errors in the sample files as they relate to my implementation of a single container using debian buster (possibly stretch and others). Firstly, my single container should be privileged, but when evaluated by entry.sh [https://github.com/balena-io-library/base-images/blob/master/examples/INITSYSTEM/systemd/systemd.v230/entry.sh](https://github.com/balena-io-library/base-images/blob/master/examples/INITSYSTEM/systemd/systemd.v230/entry.sh) it does not correctly detect the PRIVILEGED status. I was able to hardcode this as a workaround. Second, the … [Next page](https://forums.balena.io/t/rebooting-device-causes-devices-to-disappear/154285.md?page=2)