fatal error: sys/cdefs.h: No such file or directory

Hi,
I am new to openBalena. Following guide here, I successfully deploy myApp for raspberry pi3(32 bit) on ubuntu 64 bit.

Then I tried to deploy a Kivy app using the Dockerfile below.

FROM andrewmk/rpi-kivy:stretch
RUN apt-get update && apt-get install -y build-essential python3-pip python3-setuptools python3-dev && apt-get clean && rm -rf /var/lib/apt/lists/
RUN pip3 install paho-mqtt && pip3 install wheel && pip3 install pgen && pip3 install -I Cython==0.28.2 && rm -Rf /root/.cache/

RUN git clone -b 1.10.1 --depth 1 https://github.com/kivy/kivy && cd kivy && python3 setup.py build && python3 setup.py install && cd …
RUN mkdir -p apps
COPY ./apps/ ./apps/
CMD [“python3”, “apps/pictures/main.py”]\

When compiling, it says below:

[Build] main running build_ext
[Build] main Build configuration is:
[Build] main * use_rpi = 1
[Build] main * use_mali = 0
[Build] main * use_egl = 0
[Build] main * use_opengl_es2 = 1
[Build] main * use_opengl_mock = 0
[Build] main * use_sdl2 = 0
[Build] main * use_ios = 0
[Build] main * use_mesagl = 0
[Build] main * use_x11 = 0
[Build] main * use_wayland = 0
[Build] main * use_gstreamer = 0
[Build] main * use_avfoundation = 0
[Build] main * use_osx_frameworks = 0
[Build] main * debug_gl = 0
[Build] main * debug = False
[Build] main Updated build/lib.linux-armv6l-3.5/kivy/include/config.h
[Build] main Updated kivy/include/config.h
[Build] main Updated build/lib.linux-armv6l-3.5/kivy/include/config.pxi
[Build] main Updated kivy/include/config.pxi
[Build] main Updated build/lib.linux-armv6l-3.5/kivy/setupconfig.py
[Build] main Updated kivy/setupconfig.py
[Build] main Detected compiler is unix
[Build] main cythoning kivy/graphics/vertex.pyx to kivy/graphics/vertex.c
[Build] main building ‘kivy.graphics.vertex’ extension
[Build] main creating build/temp.linux-armv6l-3.5
[Build] main creating build/temp.linux-armv6l-3.5/kivy
[Build] main creating build/temp.linux-armv6l-3.5/kivy/graphics
[Build] main arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6waWnr/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Ikivy/include -Ikivy/include -I/usr/include/python3.5m -c kivy/graphics/vertex.c -o build/temp.linux-armv6l-3.5/kivy/graphics/vertex.o
[Build] main In file included from /usr/include/limits.h:25:0,
[Build] main from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:168,
[Build] main from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/syslimits.h:7,
[Build] main from /usr/lib/gcc/arm-linux-gnueabihf/6/include-fixed/limits.h:34,
[Build] main from /usr/include/python3.5m/Python.h:11,
[Build] main from kivy/graphics/vertex.c:4:
[Build] main /usr/include/features.h:364:25: fatal error: sys/cdefs.h: No such file or directory
[Build] main # include <sys/cdefs.h>
[Build] main ^
[Build] main compilation terminated.
[Build] main /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: ‘extra_requires’
[Build] main warnings.warn(msg)
[Build] main /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: ‘install_requires’
[Build] main warnings.warn(msg)
[Build] main /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: ‘setup_requires’
[Build] main warnings.warn(msg)
[Build] main /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: ‘dependency_links’
[Build] main warnings.warn(msg)
[Build] main error: command ‘arm-linux-gnueabihf-gcc’ failed with exit status 1
[Build] Built 1 service in 8:01
[Error] Deploy failed\

After some google search fatal error: sys/cdefs.h: No such file or directory|

Tried install libc6-dev-i386 and g+±multilib, but still have no luck.

The Dockerfile was tested on a rpi3(32 bit) with docker and it did work.

So I have no idea now. Could you please help? Thank you.

Hi,

I can build your Dockerfile on emulated environment without any problems so it’s weird if it doesn’t work on your side.
Since you’re cross-building an ARM image on Ubuntu 64 bit host, can you please make sure QEMU is installed on your host and try again?

Hi,

Thank you for the reply. I didn’t install QEMU additionally, but when I run

balena deploy myApp --logs --source . --emulated

I noticed

main Step 2/8 : COPY [".balena/qemu-execve","/tmp/qemu-execve"]

So does this mean QEMU exist?

If run

file .balena/qemu-execve

get

.balena/qemu-execve: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked

And for the example sense-snake, I deployed successfully in the same way.

The balena-cli downloads and injects QEMU on demand and apparently it’s correctly used during the build. Pip however is a bit notorious and can’t always build native extensions/modules via an emulation layer. It does look to me this is the case here, which if I’m right, is a bit unfortunate as I don’t know of any workarounds. It won’t help with deployment, but you can use local mode to develop (which builds on the device instead of locally on your computer), then somehow copy the produced image and deploy it directly.

OK, thank you. I will try local mode.

Also, I’m thinking if creating a new docker image with all requirements installed will work.

Hi
I believe it would depend on where you are building this image. If you are building it on an Arm host natively, there shouldn’t be any issue with it building an image with the required packages installed. Do let us know if it works out for you using pip on a native build.

Hi,
I have tried deploying the example sense-snake app on a raspberry3, but stuck at

main standard_init_linux.go:211: exec user process caused “exec format error”

My client version is 11.21.5 on the rpi3, on the ubuntu it’s 11.20.1.

Hey there.

main standard_init_linux.go:211: exec user process caused “exec format error”

That sounds like trying to run a binary compiled for the wrong system/architecture. Do you get this at runtime? At which point?

Sorry for the late reply. I’m taking a vacation, the pc and env is at the office. I will post details as soon as I go to work.

Hello again, I still can’t find where the problem is.

After installing balena cli using npm on the rpi3, I ran

balena deploy myApp --logs --source . --emulated

for the example sense-snake, the client version is 11.21.5

balena --version
11.21.5

Rpi3 runs 32 bit system

getconf LONG_BIT
32

My created app is for 32bit rpi too.

balena apps
ID APP NAME DEVICE TYPE ONLINE DEVICES DEVICE COUNT
1 myApp raspberrypi3 0 0

Below is the whole log

balena deploy myApp --logs --source . --emulated
[Debug] Parsing input…
[Debug] Loading project…
[Debug] Resolving project…
[Debug] Failed to resolve project:
[Debug] AggregateError of:
[Debug] Error: ENOENT: no such file or directory, open ‘/home/dietpi/sense-snake/docker-compose.yml’
[Debug] Error: ENOENT: no such file or directory, open ‘/home/dietpi/sense-snake/docker-compose.yaml’
[Info] Creating default composition with source: /home/dietpi/sense-snake
[Debug] Creating project…
[Info] Building for armv7hf/raspberrypi3
[Build] Building services…
[Build] main Preparing…
[Info] Emulation is enabled
[Debug] Found build tasks:
[Debug] main: build [.]
[Debug] Resolving services with [raspberrypi3|armv7hf]
[Debug] Found project types:
[Debug] main: Dockerfile.template
[Debug] Prepared tasks; building…
[Build] main Step 1/13 : FROM balenalib/raspberrypi3-node:8-build AS base
[Build] main —> 5ae69787fd80
[Build] main Step 2/13 : COPY [".balena/qemu-execve","/tmp/qemu-execve"]
[Build] main —> Using cache
[Build] main —> 98413bd821f0
[Build] main Step 3/13 : WORKDIR /usr/src/app
[Build] main —> Using cache
[Build] main —> e1629190c88a
[Build] main Step 4/13 : COPY [“package.json”,"."]
[Build] main —> Using cache
[Build] main —> 0e1725014bd0
[Build] main Step 5/13 : RUN ["/tmp/qemu-execve","-execve","/bin/sh","-c",“JOBS=MAX npm install --unsafe-perm --production”]
[Build] main —> Running in 83084295147b
[Build] main standard_init_linux.go:211: exec user process caused “exec format error”
[Build] Built 1 service in 1:05
[Error] Deploy failed
The command ‘/tmp/qemu-execve -execve /bin/sh -c JOBS=MAX npm install --unsafe-perm --production’ returned a non-zero code: 1
Error: The command ‘/tmp/qemu-execve -execve /bin/sh -c JOBS=MAX npm install --unsafe-perm --production’ returned a non-zero code: 1
at Stream. (/usr/local/lib/node_modules/balena-cli/node_modules/resin-docker-build/build/builder.js:178:23)
at Stream.stream.write (/usr/local/lib/node_modules/balena-cli/node_modules/through/index.js:26:11)
at Stream.ondata (internal/streams/legacy.js:15:31)
at Stream.emit (events.js:189:13)
at Stream.EventEmitter.emit (domain.js:441:20)
at drain (/usr/local/lib/node_modules/balena-cli/node_modules/through/index.js:36:16)
at Stream.stream.queue.stream.push (/usr/local/lib/node_modules/balena-cli/node_modules/through/index.js:45:5)
at Parser.parser.onToken (/usr/local/lib/node_modules/balena-cli/node_modules/JSONStream/index.js:132:18)
at Parser.proto.write (/usr/local/lib/node_modules/balena-cli/node_modules/jsonparse/jsonparse.js:135:34)
at Stream. (/usr/local/lib/node_modules/balena-cli/node_modules/JSONStream/index.js:23:12)
at Stream.stream.write (/usr/local/lib/node_modules/balena-cli/node_modules/through/index.js:26:11)
at IncomingMessage.ondata (_stream_readable.js:693:20)
at IncomingMessage.emit (events.js:189:13)
at IncomingMessage.EventEmitter.emit (domain.js:441:20)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at IncomingMessage.Readable.push (_stream_readable.js:224:10)
at HTTPParser.parserOnBody (_http_common.js:126:22)
at Socket.socketOnData (_http_client.js:449:20)
at Socket.emit (events.js:189:13)
at Socket.EventEmitter.emit (domain.js:441:20)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at Pipe.onStreamRead [as onread] (internal/stream_base_commons.js:145:17)
at Bluebird.try.then (/usr/local/lib/node_modules/balena-cli/node_modules/resin-docker-build/build/builder.js:72:20)
at Builder.createBuildStream (/usr/local/lib/node_modules/balena-cli/node_modules/resin-docker-build/build/builder.js:71:97)
at Promise (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/build.js:82:17)
at Object.runBuildTask (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/build.js:59:12)
at /usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:157:34
at Generator.next ()
at /usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:8:71
at new Promise ()
at __awaiter (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:4:12)
at performSingleBuild (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:155:12)
at Bluebird.map (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:140:20)
at processImmediate (timers.js:632:19)
at process.topLevelDomainCallback (domain.js:120:23)
at Object. (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:139:39)
at Generator.next ()
at fulfilled (/usr/local/lib/node_modules/balena-cli/node_modules/resin-multibuild/build/index.js:5:58)
From previous event:
at runCommand (/usr/local/lib/node_modules/balena-cli/build/app-capitano.js:181:14)
at Object.exports.run (/usr/local/lib/node_modules/balena-cli/build/app-capitano.js:193:39)
at routeCliFramework (/usr/local/lib/node_modules/balena-cli/build/preparser.js:39:79)
at process._tickCallback (internal/process/next_tick.js:43:7)
at Function.Module.runMain (internal/modules/cjs/loader.js:777:11)
at executeUserCode (internal/bootstrap/node.js:342:17)
at startExecution (internal/bootstrap/node.js:276:5)
at startup (internal/bootstrap/node.js:227:5)
at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)

Did I install balena cli incorrectly?

Hi,

Are you deploying with local mode? If so then please remove the --emulatedoption and try again. I’m not sure where the build happens from your logs but there are only 2 places. If it’s running on your development machine (which is x86_64) then the --emulated option is needed since it will run an emulated build using qemu. On the other hand, the build can be run on your device as well so --emulated option must be removed.

Can you please try again and let us know how it goes?

Thank you for your kind remind.

I forgot to remove --emulated option when on rpi3…