Application containers failing with error standard_init_linux.go:207: exec user process caused "exec format error"

Product support balenaOS
1

Hi,

I am getting an error after pushing to BalenaCloud ‘standard_init_linux.go:207: exec user process caused “exec format error”’ all my containers are failing with this error. I am in production mode balenaOS 2.48.0+rev1 (production) on Balena Fin (CM3). Same containers worked perfectly when I was using development OS and in local mode.

This is one of my Dockerfile,

FROM python:alpine3.7

WORKDIR ./

RUN mkdir ./app \
RUN mkdir ./app/logs \
    && mkdir ./app/config

COPY .env ./mercury-services/
COPY requirements.txt ./
COPY ./mercury ./mercury

RUN pip install -r requirements.txt

ENV PYTHONPATH "${PYTHONPATH}:./mercury"

COPY . .

CMD [ "python", "./service.py" ]

Also how do I do ssh into my device(production) from my Dev machine without using BalenaCloud.
I am getting this error

gh_cheryjose@ssh.balena-devices.com: Permission denied (publickey).
Warning: ssh process exited with non-zero code "255"

Chery

4

Hi,

Exec format error can happen due to multiple issues, most common being missing #! in the scripts, for ex. for sh files #!/bin/bash. Or if you are on windows it could happen due to CRLF so you can use --convert-eol option to convert them to LF.

To SSH into the device you can either use balena ssh or ssh -p 22222 root@<device_ip_address>, to see all the ways to SSH please visit https://www.balena.io/docs/learn/manage/ssh-access/#using-balena-ssh-from-the-cli.

7

@iamsolankiamit thanks for getting back. My application worked in local mode and this is happening when I switched to production. I am running a python file as given in the example.

Not sure whats different between local mode and production to get this error.

Same code worked fine also on Balena Fin (CM3) flashed with image_Raspbian-Fin-full (v0.1.0) and using docker-compose, so its not do with OS architecture or my code.

Need help with this!!

8

Hi Chery,

There isn’t much difference in dev and prod images in terms of how they work, we enable a few ports and extra services in dev images to enable faster development, and dev images are less secure. You can read more here https://www.balena.io/docs/reference/OS/overview/2.x/#development-vs-production-images. So there shouldn’t be much issue due to image change.

The Docker file looks good to me, if you can share the code(stripped down version would work as well) that can help in reproducing the issue, we can pin point the issue.

11

Hi @iamsolankiamit,

Its actually bit hard to share that code, I will need some extra time to do it, its one file calling an instance method and multiple dependencies used in it, coming from multiple files.

I have an easy one in the same project, all its doing is getting an image and its also failing with same error in Production environment and no python used in it.

 mosquitto  standard_init_linux.go:207: exec user process caused "exec format error"
 mosquitto  standard_init_linux.go:207: exec user process caused "exec format error"

Its got a Dockerfile and it is

FROM eclipse-mosquitto:latest

WORKDIR ./

And the docker-compose.yml used for this project is,

version: '2.1'
volumes:
    state-service:
    mercury-services:
    mosquitto:
services:
  mosquitto:
    build: ./mosquitto/
    volumes:
      - mosquitto:/certs
    ports:
      - "1883:1883"
      - "8883:8883"
    network_mode: "host"
  mercury_state_service:
    build: ./state-service/
    network_mode: "host"
    volumes:
       - state-service:/app
       - mercury-services:/mercury-services

Also attaching the log file from BalenaCloud,

dev-balena_holy-surf-25.05.20_20_43_33.log (100.0 KB)

12

Hello,

For sshing into a device without balena-cli, you’ll need to create an ssh configuration file. This is described in this forum post: Remote OS upgrade of custom host image possible?

For the exec format error it may be that you have the binaries for another architecture (x86 or x64 or arm64 instead of arm).
You should be able to check it with balena inspect --format='{{.Architecture}}' <IMAGE> from the host OS.

14

@zvin thanks, I will do an inspect, I did balena push from an Ubuntu machine but what’s interesting is it worked in local mode with out any errors.

15

Actually it seems like another user has already hit this issue: How to deploy official eclipse-mosquitto container on a raspberry pi as a service in a docker compose file? .
The fix is to specify the architecture in your Dockerfile: arm32v6/eclipse-mosquitto:latest

1 Like