BalenaOS in a docker container does not connect to balenaCloud

galoev · May 3, 2021, 3:54am

I’m trying to run balenaOS in a docker container on a RPI4 (Raspberry Pi OS (32-bit)) and then run a balena-sound. Due to the fact that my rpi4 has a 32-bit operating system, I run containers on the “resin/resinos:2.75.0_rev1-raspberrypi3” image, because “resin/resinos:2.75.0_rev1.dev-raspberrypi4-64” is not runs on my 32 bit operating system.

./balenaos-in-container.sh --image resin/resinos:2.75.0_rev1-raspberrypi3 --id balena-os -c "./sound.config.json"
INFO: Creating balena-boot-balena-os docker volume...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
INFO: Creating balena-state-balena-os docker volume...
INFO: Creating balena-data-balena-os docker volume...
INFO: Running balenaOS as container balena-container-balena-os ...
WARNING: Localhost DNS setting (--dns=127.0.0.2) may fail in containers.
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
No drop-in found. Most probably you don't run a development image.

Welcome to balenaOS 2.75.0+rev1!

/lib/systemd/system/balena-host.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/balena-host.sock → /run/balena-host.sock; please update the unit file accordingly.
/lib/systemd/system/balena-engine.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/balena-engine.sock → /run/balena-engine.sock; please update the unit file accordingly.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Audit Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Kernel Debug File System...
         Mounting Temporary Directory (/tmp)...
         Starting Create list of stati…ce nodes for the current kernel...
         Starting OS configuration device key service...
         Starting Rollback clear bootc… altboot crumb is still present...
         Mounting FUSE Control File System...
         Mounting Kernel Configuration File System...
         Starting Remount Root and Kernel File Systems...
         Starting Apply Kernel Variables...
         Starting udev Coldplug all Devices...
         Starting Setup Virtual Console...
         Starting Set the system clock from the build timestamp...
[  OK  ] Mounted Kernel Debug File System.
[  OK  ] Mounted Temporary Directory (/tmp).
[  OK  ] Started Create list of static…vice nodes for the current kernel.
[  OK  ] Started OS configuration device key service.
[  OK  ] Started Rollback clear bootco…if altboot crumb is still present.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Mounted Kernel Configuration File System.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Setup Virtual Console.
[  OK  ] Started Set the system clock from the build timestamp.
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Started Create Static Device Nodes in /dev.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Set up automount mnt-sysroot-inactive partition.
         Mounting resin-data bind mount...
         Mounting /var/volatile...
[  OK  ] Mounted resin-data bind mount.
[  OK  ] Mounted /var/volatile.
         Starting Resin reset OS state...
         Starting Bind mount volatile /var/cache...
         Starting Bind mount volatile /var/lib...
         Starting Bind mount volatile /var/spool...
         Starting Bind mount volatile /srv...
[  OK  ] Started Bind mount volatile /var/cache.
[  OK  ] Started Bind mount volatile /var/lib.
[  OK  ] Started Bind mount volatile /var/spool.
[  OK  ] Started Bind mount volatile /srv.
[  OK  ] Started Resin reset OS state.
         Mounting Bind mount for /etc/fake-hwclock...
         Mounting Docker data mountpoint...
[  OK  ] Mounted Bind mount for /etc/fake-hwclock.
[  OK  ] Mounted Docker data mountpoint.
[  OK  ] Reached target Local File Systems.
         Starting Restore/save the current clock...
[  OK  ] Started Restore/save the current clock.
         Starting Journal Service...
[  OK  ] Started udev Coldplug all Devices.
         Starting udev Wait for Complete Device Initialization...
[  OK  ] Started Journal Service.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
         Starting Bind mount for /etc/NetworkManager/conf.d...
         Starting Bind mount for /etc/…tworkManager/system-connections...
         Starting Bind mount for /etc/docker...
         Starting Bind mount for /etc/hostname...
         Starting Bind mount for /etc/openvpn...
         Starting Bind mount for /etc/resin-supervisor...
         Starting Bind mount for /etc/ssh/hostkeys...
         Starting Bind mount for /etc/udev/rules.d...
         Starting Bind mount for /home/root/.docker...
         Starting Bind mount for /home/root/.rnd...
         Starting Bind mount for /home/root/.ssh...
         Starting Bind mount for /usr/share/ca-certificates/balena...
         Starting Bind mount for /var/lib/NetworkManager...
         Starting Bind mount for /var/lib/bluetooth...
         Starting Bind mount for /var/lib/chrony...
         Starting Bind mount for /var/lib/systemd...
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Bind mount for /etc/NetworkManager/conf.d.
[  OK  ] Started Bind mount for /etc/NetworkManager/system-connections.
[  OK  ] Started Bind mount for /etc/docker.
[  OK  ] Started Bind mount for /etc/hostname.
[  OK  ] Started Bind mount for /etc/resin-supervisor.
[  OK  ] Started Bind mount for /etc/openvpn.
[  OK  ] Started Bind mount for /etc/ssh/hostkeys.
[  OK  ] Started Bind mount for /etc/udev/rules.d.
[  OK  ] Started Bind mount for /home/root/.docker.
[  OK  ] Started Bind mount for /home/root/.rnd.
[  OK  ] Started Bind mount for /home/root/.ssh.
[  OK  ] Started Bind mount for /usr/share/ca-certificates/balena.
[  OK  ] Started Bind mount for /var/lib/NetworkManager.
[  OK  ] Started Bind mount for /var/lib/bluetooth.
[  OK  ] Started Bind mount for /var/lib/chrony.
[  OK  ] Started Bind mount for /var/lib/systemd.
         Starting udev Kernel Device Manager...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started udev Wait for Complete Device Initialization.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Config.json path watch.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started Balena supervisor updater timer.
[  OK  ] Reached target Paths.
[  OK  ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
         Starting Docker Socket for the API.
         Starting Docker Socket for the API.
[  OK  ] Listening on D-Bus System Message Bus Socket.
         Starting sshd.socket.
[  OK  ] Reached target System Time Set.
[  OK  ] Listening on Docker Socket for the API.
[  OK  ] Listening on Docker Socket for the API.
[  OK  ] Listening on sshd.socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Modem Manager...
         Starting Balena device UUID...
[  OK  ] Started NTP client/server.
[  OK  ] Reached target System Time Synchronized.
[  OK  ] Started Periodically update the saved fake-hwclock..
[  OK  ] Started Periodic check for configuration changes.
[  OK  ] Started Periodic vacuum of journald logs.
[  OK  ] Reached target Timers.
[  OK  ] Started D-Bus System Message Bus.
         Starting Extract custom CA from config.json...
         Starting IPv6 Packet Filtering Framework...
         Starting IPv4 Packet Filtering Framework...
         Starting NetworkManager confi…tion fragments from config.json...
         Starting OS SSH keys...
         Starting udev rules configuration from config.json...
         Starting Resin network configure service...
         Starting Resin persistent logs...
         Starting OpenSSH Key Generation...
[  OK  ] Started IPv6 Packet Filtering Framework.
[  OK  ] Started IPv4 Packet Filtering Framework.
[  OK  ] Reached target Network (Pre).
[  OK  ] Started Modem Manager.
[  OK  ] Started Extract custom CA from config.json.
[  OK  ] Started Resin persistent logs.
[  OK  ] Started Resin network configure service.
         Starting Resin NTP server configure service...
[  OK  ] Started udev rules configuration from config.json.
[  OK  ] Started OS SSH keys.
[  OK  ] Started Resin NTP server configure service.
         Starting Config.json watcher service...
         Starting Resin NTP server configure service...
[  OK  ] Started Config.json watcher service.
[  OK  ] Started Balena device UUID.
         Starting Resin Hostname Configuration...
[  OK  ] Started NetworkManager configuration fragments from config.json.
         Starting Network Manager...
[  OK  ] Started Resin Hostname Configuration.
         Starting Avahi mDNS/DNS-SD Stack...
[  OK  ] Started Avahi mDNS/DNS-SD Stack.
[  OK  ] Started Resin NTP server configure service.
[  OK  ] Started Network Manager.
[  OK  ] Reached target Network.
         Starting DNS forwarder and DHCP server...
[  OK  ] Started OS configuration update service.
         Starting Resin init service...
[  OK  ] Started DNS forwarder and DHCP server.
[  OK  ] Started Resin init service.
         Starting Balena Application Container Engine...
         Starting Resin proxy configuration service...
         Starting Hostname Service...
         Starting Prepare OpenVPN...
[  OK  ] Started Hostname Service.
[  OK  ] Started Resin proxy configuration service.
         Starting Network Manager Script Dispatcher Service...
[  OK  ] Started Network Manager Script Dispatcher Service.
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Started OpenSSH Key Generation.
[  OK  ] Started Balena Application Container Engine.
         Starting Balena supervisor...
[  OK  ] Started Balena supervisor.
[  OK  ] Started Balena supervisor updater.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped Balena supervisor.
[  OK  ] Started libcontainer containe…3bb0ef3dcf90438fa6d253e37982757c4.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
         Starting Balena supervisor...
[  OK  ] Started Balena supervisor.
[  OK  ] Started libcontainer containe…10276f48084f412e463aeff5289ae5b3c.
[ TIME ] Timed out waiting for device /dev/serial1.
[DEPEND] Dependency failed for Configu…luetooth Modems connected by UART.
[ TIME ] Timed out waiting for device /dev/zram0.
[DEPEND] Dependency failed for Enable …pressed swap in memory using zram.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

Container with balenOS starts but does not connect to balenCloud.

In addition to “resin/resinos:2.75.0_rev1-raspberrypi3” I tried “resin/resinos:2021.01.0-raspberry-pi” the result is the same.
I connected to the container and checked if it picked up the config file. And yes the config file copied correctly to volume. (checked this with the command cat/mnt/boot/config.json)

I saw the topic: balenaOS in a docker container won´t connect to balenacloud. But there eventually @knaps97 changed the operating system to 64 bit, but I can not do it.

docker --version
Docker version 20.10.6, build 370c289


bash-5.0# balena-engine version
Client:
 Version:           19.03.18
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        840aacc77b6c600b3b929fe9e4d9356a322b9e5b
 Built:             Wed Apr 28 11:01:15 2021
 OS/Arch:           linux/arm
 Experimental:      false

Server:
 Engine:
  Version:          19.03.18
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       840aacc77b6c600b3b929fe9e4d9356a322b9e5b
  Built:            Wed Apr 28 11:01:15 2021
  OS/Arch:          linux/arm
  Experimental:     true
 containerd:
  Version:          1.2.0+unknown
  GitCommit:        
 runc:
  Version:          1.0.0-rc4+dev
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 balena-engine-init:
  Version:          0.13.0
  GitCommit:        949e6fa-dirty

Thanks for helping

knaps97 · May 3, 2021, 7:04am

Hi @galoev we managed to get Balena to run in a container on a 32 Bit OS (Debian).
I´m afraid the final fiddling with the software wasn´t done by me, but as I remember they had some issues with the Docker Software itself because that wasn´t supporting 32 bit anymore so they had to do some changes to that and the Balena App itself is self build.
Sorry that wasn´t too helpfull if you have any specific questions I can try to forward them to one of our developers.

galoev · May 3, 2021, 2:20pm

Everything worked for you @knaps97 when you installed Ubuntu Server for Pi4 as Base OS on RPI4, because it is a 64 bit operating system.
When I run a container from an image “resin/resinos:2.75.0_rev1-raspberrypi4-64” I get this output:

sudo ./balenaos-in-container.sh --image resin/resinos:2.75.0_rev1-raspberrypi4-64  -c "./sound.config.json"
INFO: Creating balena-boot-23193 docker volume...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
standard_init_linux.go:219: exec user process caused: exec format error

As I understand it this is happening because my RPI4 has a 32 bit Raspberry Pi OS. And so I run the container on the “resin/resinos:2.75.0_rev1-raspberrypi3” image. I’m right?

Sorry, I’m a docker noob. I just want to run balenaOs in docker on my RPI4 with Raspberry Pi OS (32-bit)

phil-d-wilson · May 4, 2021, 3:19pm

@galoev

As I understand it this is happening because my RPI4 has a 32 bit Raspberry Pi OS.

Correct. You won’t be able to run a Pi4 image (64-bit) in docker on a 32-bit PiOS device.

And so I run the container on the “resin/resinos:2.75.0_rev1-raspberrypi3” image. I’m right?

Correct again. I have just run the resin/resinos:2.75.0_rev1-raspberrypi3 image on my Pi4 running PiOS & docker and the “device” appeared in my application.
You will need to make sure your balenaCloud application (which you downloaded the config.json file for is also for the Raspberrypi3 device type.

Phil

galoev · May 4, 2021, 4:00pm

@phil-d-wilson

Yes, I have application for the RPI3 in balenaCloud.
To load the config.json file, I click on Add device in this application.

Then at the bottom I select “Download configuration file only” instead of “Download balenaOS (~ 148 MB)”
I click on this button and then the “sound.config.json” file is loaded.
Then I copy this file to the balenaos-in-container folder. And I execute this command:

sudo ./balenaos-in-container.sh --image resin/resinos:2.75.0_rev1-raspberrypi3 --id balena-os -c "$PWD/sound.config.json"

I get this output:

INFO: Creating balena-boot-balena-os docker volume...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
INFO: Creating balena-state-balena-os docker volume...
INFO: Creating balena-data-balena-os docker volume...
INFO: Running balenaOS as container balena-container-balena-os ...
WARNING: Localhost DNS setting (--dns=127.0.0.2) may fail in containers.
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
No drop-in found. Most probably you don't run a development image.

Welcome to balenaOS 2.75.0+rev1!

/lib/systemd/system/balena-host.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/balena-host.sock → /run/balena-host.sock; please update the unit file accordingly.
/lib/systemd/system/balena-engine.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/balena-engine.sock → /run/balena-engine.sock; please update the unit file accordingly.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Forward Password…ests to Wall Directory Watch.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Audit Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on udev Kernel Socket.
         Mounting Kernel Debug File System...
         Mounting Temporary Directory (/tmp)...
         Starting Create list of …des for the current kernel...
         Starting OS configuration device key service...
         Starting Rollback clear …oot crumb is still present...
         Mounting FUSE Control File System...
         Mounting Kernel Configuration File System...
         Starting Remount Root and Kernel File Systems...
         Starting Apply Kernel Variables...
         Starting udev Coldplug all Devices...
         Starting Setup Virtual Console...
         Starting Set the system …k from the build timestamp...
[  OK  ] Mounted Kernel Debug File System.
[  OK  ] Mounted Temporary Directory (/tmp).
[  OK  ] Started Create list of s…nodes for the current kernel.
[  OK  ] Started OS configuration device key service.
[  OK  ] Started Rollback clear b…tboot crumb is still present.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Mounted Kernel Configuration File System.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Setup Virtual Console.
[  OK  ] Started Set the system clock from the build timestamp.
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Started Create Static Device Nodes in /dev.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Set up automount mnt-sysroot-inactive partition.
         Mounting resin-data bind mount...
         Mounting /var/volatile...
[  OK  ] Mounted resin-data bind mount.
[  OK  ] Mounted /var/volatile.
         Starting Resin reset OS state...
         Starting Bind mount volatile /var/cache...
         Starting Bind mount volatile /var/lib...
         Starting Bind mount volatile /var/spool...
         Starting Bind mount volatile /srv...
[  OK  ] Started udev Coldplug all Devices.
[  OK  ] Started Bind mount volatile /var/cache.
[  OK  ] Started Bind mount volatile /var/lib.
[  OK  ] Started Bind mount volatile /var/spool.
[  OK  ] Started Bind mount volatile /srv.
[  OK  ] Started Resin reset OS state.
         Mounting Bind mount for /etc/fake-hwclock...
         Mounting Docker data mountpoint...
         Starting udev Wait for C…lete Device Initialization...
[  OK  ] Mounted Bind mount for /etc/fake-hwclock.
[  OK  ] Mounted Docker data mountpoint.
[  OK  ] Reached target Local File Systems.
         Starting Restore/save the current clock...
[  OK  ] Started Restore/save the current clock.
         Starting Journal Service...
[  OK  ] Started Journal Service.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
         Starting Bind mount for /etc/NetworkManager/conf.d...
         Starting Bind mount for …Manager/system-connections...
         Starting Bind mount for /etc/docker...
         Starting Bind mount for /etc/hostname...
         Starting Bind mount for /etc/openvpn...
         Starting Bind mount for /etc/resin-supervisor...
         Starting Bind mount for /etc/ssh/hostkeys...
         Starting Bind mount for /etc/udev/rules.d...
         Starting Bind mount for /home/root/.docker...
         Starting Bind mount for /home/root/.rnd...
         Starting Bind mount for /home/root/.ssh...
         Starting Bind mount for …are/ca-certificates/balena...
         Starting Bind mount for /var/lib/NetworkManager...
         Starting Bind mount for /var/lib/bluetooth...
         Starting Bind mount for /var/lib/chrony...
         Starting Bind mount for /var/lib/systemd...
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Started Bind mount for /etc/hostname.
[  OK  ] Started Bind mount for /etc/resin-supervisor.
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Started Bind mount for /etc/NetworkManager/conf.d.
[  OK  ] Started Bind mount for /…rkManager/system-connections.
[  OK  ] Started Bind mount for /etc/docker.
[  OK  ] Started Bind mount for /etc/openvpn.
[  OK  ] Started Bind mount for /etc/ssh/hostkeys.
[  OK  ] Started Bind mount for /etc/udev/rules.d.
[  OK  ] Started Bind mount for /home/root/.docker.
[  OK  ] Started Bind mount for /home/root/.rnd.
[  OK  ] Started Bind mount for /home/root/.ssh.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started Bind mount for /…share/ca-certificates/balena.
[  OK  ] Started Bind mount for /var/lib/NetworkManager.
[  OK  ] Started Bind mount for /var/lib/bluetooth.
[  OK  ] Started Bind mount for /var/lib/chrony.
[  OK  ] Started Bind mount for /var/lib/systemd.
[  OK  ] Started udev Wait for Complete Device Initialization.
[  OK  ] Started Dispatch Passwor…s to Console Directory Watch.
[  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Config.json path watch.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started Balena supervisor updater timer.
[  OK  ] Reached target Paths.
[  OK  ] Listening on Avahi mDNS/…S-SD Stack Activation Socket.
         Starting Docker Socket for the API.
         Starting Docker Socket for the API.
[  OK  ] Listening on D-Bus System Message Bus Socket.
         Starting sshd.socket.
[  OK  ] Reached target System Time Set.
[  OK  ] Listening on Docker Socket for the API.
[  OK  ] Listening on Docker Socket for the API.
[  OK  ] Listening on sshd.socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Modem Manager...
         Starting Balena device UUID...
[  OK  ] Started NTP client/server.
[  OK  ] Reached target System Time Synchronized.
[  OK  ] Started Periodically update the saved fake-hwclock..
[  OK  ] Started Periodic check for configuration changes.
[  OK  ] Started Periodic vacuum of journald logs.
[  OK  ] Reached target Timers.
[  OK  ] Started D-Bus System Message Bus.
         Starting Extract custom CA from config.json...
         Starting IPv6 Packet Filtering Framework...
         Starting IPv4 Packet Filtering Framework...
         Starting NetworkManager …fragments from config.json...
         Starting OS SSH keys...
         Starting udev rules configuration from config.json...
         Starting Resin network configure service...
         Starting Resin persistent logs...
         Starting OpenSSH Key Generation...
[  OK  ] Started IPv6 Packet Filtering Framework.
[  OK  ] Started IPv4 Packet Filtering Framework.
[  OK  ] Reached target Network (Pre).
[  OK  ] Started Modem Manager.
[  OK  ] Started Extract custom CA from config.json.
[  OK  ] Started Resin network configure service.
         Starting Resin NTP server configure service...
[  OK  ] Started Resin persistent logs.
[  OK  ] Started OS SSH keys.
[  OK  ] Started udev rules configuration from config.json.
[  OK  ] Started Resin NTP server configure service.
         Starting Config.json watcher service...
         Starting Resin NTP server configure service...
[  OK  ] Started Config.json watcher service.
[  OK  ] Started Balena device UUID.
         Starting Resin Hostname Configuration...
[  OK  ] Started Resin NTP server configure service.
[  OK  ] Started Resin Hostname Configuration.
         Starting Avahi mDNS/DNS-SD Stack...
[  OK  ] Started Avahi mDNS/DNS-SD Stack.
[  OK  ] Started NetworkManager c…n fragments from config.json.
         Starting Network Manager...
[  OK  ] Started Network Manager.
[  OK  ] Reached target Network.
         Starting DNS forwarder and DHCP server...
[  OK  ] Started OS configuration update service.
         Starting Resin init service...
[  OK  ] Started DNS forwarder and DHCP server.
         Starting Balena Application Container Engine...
         Starting Resin proxy configuration service...
         Starting Hostname Service...
[  OK  ] Started Resin init service.
         Starting Prepare OpenVPN...
[  OK  ] Started Hostname Service.
[  OK  ] Started Resin proxy configuration service.
         Starting Network Manager Script Dispatcher Service...
[  OK  ] Started Network Manager Script Dispatcher Service.
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Started OpenSSH Key Generation.
[  OK  ] Started Balena Application Container Engine.
         Starting Balena supervisor...
[  OK  ] Started Balena supervisor.
[  OK  ] Started Balena supervisor updater.
[  OK  ] Stopped Balena supervisor.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started libcontainer con…43881abdd10d21e6e97ffa0e1795.
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
         Starting Balena supervisor...
[  OK  ] Started Balena supervisor.
[  OK  ] Stopped OpenVPN.
         Starting Prepare OpenVPN...
[  OK  ] Started Prepare OpenVPN.
[  OK  ] Started OpenVPN.
[  OK  ] Started libcontainer con…5bfbe933cff8392e635c0a7427cd.
[ TIME ] Timed out waiting for device /dev/serial1.
[DEPEND] Dependency failed for Co…oth Modems connected by UART.
[ TIME ] Timed out waiting for device /dev/zram0.
[DEPEND] Dependency failed for En…ed swap in memory using zram.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

It took 10 minutes, but no new device appeared in balenaCloud. The last time I started the container, I left it running overnight, but it still didn’t connect to balenaCloud.

phil-d-wilson · May 5, 2021, 10:06am

Hi again Ilkin,

I have managed to recreate your issue. Yesterday I think I specified a local image and used an old version of the OS. I’ve just tried again after removing all my local images, and recreated the issue. By trial and error, I’ve found that the last working image was resin/resinos:2.60.1_rev1.dev-raspberrypi3 and since that version the device no longer appears on balenaCloud. Do note that you have to use a development image, which I don’t think you were - however, that’s not the issue here.

What I’m going to do now is talk to our OS team, and ask them if they can see from the changelog what might be causing the problem here, and if they know how we might fix it. However, please do bear in mind that running balenaOS in a container is still an experimental piece of work, and is not a high priority (at this time). I cannot guarantee if or when someone may have time to look at this.

In the meantime, you can try again but using the version of the OS I’ve pasted above.

Phil

galoev · May 5, 2021, 10:36am

Thanks a lot, Phil! The resin/resinos:2.60.1_rev1.dev-raspberrypi3 image worked for me and the container connected to balenaCloud

phil-d-wilson · May 5, 2021, 10:49am

Good to hear! I’ll keep pushing to get the latest image(s) to work and let you know if we make any progress.

Phil