Container access is only available for running containers

thibault · October 21, 2020, 8:26am

Hi,

I’m running a balena OS (balenaOS 2.58.6+rev1) in my Raspberry PI4. The built image has done well but when I ssh inside my container with the following command:

balena ssh RaspberryPI4 main

I get the following error:

Application container must be running for a terminal to be started.
Connection to ssh.balena-devices.com closed.

From my balena dashboard, when I start terminal session in the main, it directly says that “Container access is only available for running containers”

No idea how to correct that.

dfunckt · October 21, 2020, 8:30am

Is the “main” container actually running? What does the Dashboard show? Have you pushed a release? Try opening a terminal session into the host and checking the logs with journalctl.

thibault · October 21, 2020, 8:40am

When I open a terminal session into the host and run journalctl, it gives me:

– Logs begin at Thu 2020-08-06 07:25:58 UTC, end at Wed 2020-10-21 08:>
Aug 06 07:25:58 localhost kernel: Booting Linux on physical CPU 0x00000>
Aug 06 07:25:58 localhost kernel: Linux version 5.4.58 (oe-user@oe-host>
Aug 06 07:25:58 localhost kernel: Machine model: Raspberry Pi 4 Model B>
Aug 06 07:25:58 localhost kernel: efi: Getting EFI parameters from FDT:
Aug 06 07:25:58 localhost kernel: efi: UEFI not found.

I pushed a release before and I have a “Target Release” and the Dashboard show:

Starting service ‘main sha256:79f25a955cfd48b684205b08faede493363cd6ee9dab31722933fe24f4cb6d00’

I think my container is really not running but don’t know why

dfunckt · October 21, 2020, 8:43am

What is the output of balena ps -a? Also, can you try journalctl -fn100 -uresin-supervisor?

thibault · October 21, 2020, 8:46am

From balena ps -a, I get:

CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                 PORTS               NAMES
521fd32e3037        balena/aarch64-supervisor:v11.14.0   "/usr/src/app/entry.…"   4 hours ago         Up 4 hours (healthy)                       resin_supervisor
b01fd69e1080        79f25a955cfd                         "./start.sh python3 …"   16 hours ago        Created                                    main_2858636_1573311

From journalctl -fn100 -uresin-supervisor, I get:

-- Logs begin at Thu 2020-08-06 07:25:58 UTC. --
Oct 21 08:35:40 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:40 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:40 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:40 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:40 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:41 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:35:41 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 2000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:42 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:44 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:35:44 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 4000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:45 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 8000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:49 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:57 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]   LogBackend: server responded with status code: 502
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 16000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:35:58 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:36:14 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:36:14 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 32000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:36:15 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:36:47 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:36:47 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 64000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:36:48 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:37:30 45bfe66 resin-supervisor[1786]: [api]     GET /v1/healthy 200 - 5.671 ms
Oct 21 08:37:31 45bfe66 resin-supervisor[1786]: [debug]   Attempting container log timestamp flush...
Oct 21 08:37:31 45bfe66 resin-supervisor[1786]: [debug]   Container log timestamp flush complete
Oct 21 08:37:52 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:37:52 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 128000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:37:53 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 256000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:40:01 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:42:31 45bfe66 resin-supervisor[1786]: [api]     GET /v1/healthy 200 - 5.652 ms
Oct 21 08:44:17 45bfe66 resin-supervisor[1786]: [info]    Applying target state
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [event]   Event: Service start {"service":{"appId":1756818,"serviceId":745995,"serviceName":"main","releaseId":1573311}}
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]   Scheduling another update attempt in 512000ms due to failure:  Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]   Device state apply error Error: Failed to apply state transition steps. (HTTP code 400) unexpected - OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./start.sh\": permission denied": unknown  Steps:["start"]
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]         at /usr/src/app/dist/app.js:10:8684
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]       at runMicrotasks (<anonymous>)
Oct 21 08:44:18 45bfe66 resin-supervisor[1786]: [error]       at processTicksAndRejections (internal/process/task_queues.js:97:5)

I think is because of ENTRYPOINT ["./start.sh"] in my Dockerfile

dfunckt · October 21, 2020, 9:10am

Thank you. Can you lastly try systemctl kill balena from the host? This will force balenaEngine, the Supervisor and your application to restart and hopefully get things running again.

thibault · October 29, 2020, 10:57am

Sorry for my late response but I found what was my problem and corrected it. Thanks for the help!

georgiats · October 29, 2020, 11:00am

Glad to hear it worked @thibault!

dfunckt · October 29, 2020, 2:36pm

@thibault, I’m curious – what was it?

thibault · October 29, 2020, 3:49pm

Hi,

I saw in the result of journalctl -fn100 -uresin-supervisor command that the problem came from my ENTRYPOINT ["./start.sh"] in my Dockerfile. My start.sh was just containing the driver settings of MYRIAD and the build of python API wrapper of Openvino.

I just decided to switch my start.sh from ENTRYPOINT command to CMD as a final command of my Dockerfile. After that, I could correctly access my container. I’m not sure about the exact difference between ENTRYPOINT and CMD with Docker but the problem was not from balena I guess.

thgreasi · October 29, 2020, 7:17pm

Thanks for sharing what fixed your issue.
I’m guessing that one of the base images might be specifying an ENTRYPOINT of their own, which your dockerfile was overriding, since only the last ENTRYPOINT for the whole build image will be used.
That’s try for example for all balenalib base images which specify an ENTRYPOINT so that they can run some setup code before your CMD code runs.
Let me point you to the respective part in our documentation page:

Kind regards,
Thodoris

Topic		Replies	Views
Balena services downloaded but no running! Product support docker , network , balena-cli	25	849	June 29, 2022
Application container must be running for a terminal to be started. But the container is running! Product support	1	680	June 10, 2019
Trouble Getting balena run Command to Work Product support raspberrypi3	5	328	January 28, 2020
Debugging Containers General docker	4	739	February 10, 2021
Run commands in container using balena-cli ssh ? Project help	1	873	April 14, 2021

Container access is only available for running containers

Related topics