Alpine image with custom musl libc settings

Product support
1

Recently, we switched from raspbian to alpine and are seeing problems which are likely due to the musl libc configuration. Specifically, musl seems to only allow 256 semaphores via sem_open(). This is configurable via limits.h in the musl tree, http://git.musl-libc.org/cgit/musl/tree/include/limits.h#n63

How would I go about changing this and producing a new raspberrypi base image for use with Resin?

4

Hi @toby, looking at that limits.h file, is that the one that is included in the musl-dev package? (link to the Alpine package search for musl-dev). It seems to be that one, when the package is installed at /usr/include/limits.h.

Are you seeing issues, when using code that you have compiled? If so, probably you would go about installing that package and modifying the file in a RUN step of your Dockerfile/Dockerfile.template, and do the code compilations after that. Can you give that a try?

If that’s not the cases when you see issues, could you give us more info?

5

The limits.h file is used when compiling musl, rather than my application. It looks like I need to rebuild the alpine musl package with my change.

I’ve now got a small patch to musl which I have successfully used with https://github.com/dnephin/docker-apk-build to generate a working x64 musl alpine package. Now, I need to rebuild the musl package for raspberry pi with my change, then persuade my Dockerfile to install this package at resin build time.

docker-apk-build appears to be native only, do you have a mechanism for cross compiling raspberry pi apks?

The patch is a one-liner, it’s:

--- a/include/limits.h
+++ b/include/limits.h
@@ -64,7 +64,7 @@
 #define PTHREAD_STACK_MIN 2048
 #define PTHREAD_DESTRUCTOR_ITERATIONS 4
 #define SEM_VALUE_MAX 0x7fffffff
-#define SEM_NSEMS_MAX 256
+#define SEM_NSEMS_MAX 4096
 #define DELAYTIMER_MAX 0x7fffffff
 #define MQ_PRIO_MAX 32768
 #define LOGIN_NAME_MAX 256
6

I see. Talked to our team, and would recommend a different path. The reason for the difference is that cross-compiling most likely would involve QEMU, and in our experience, QEMU is not really excelling in any compilation jobs, and likely you would end up with subtle issues.

Instead, we’d probably recommend a multi-stage build instead. The idea would be:

  • have a first stage, which would be based on the docker-apk-build that you mentioned, but explicitly building the package you need
  • in the next stage, it would be the image that you are planning to get onto the device, but would copy in the package from the earlier stage and install.

This would help you have the package and code stay together, and take advantage of our ARM builders, so it would be a native build.

See more in our blogpost regarding multi-stage builds:

What do you think?

7

Thanks, that definitely sounds like the right way to do it.

As a workaround, I’ve managed to rebuild the package in docker (via qemu). In case anyone hits the same problem, here are the steps I used:

git clone https://github.com/dnephin/docker-apk-build.git

Edit builder/Dockerfile and change the FROM line to read

FROM resin/raspberrypi3-alpine

Packages will now be built for this platform. Run,

make build
~/bin/setup.sh

Press enter to accept the default location for the key.

cd /work/main/musl

Create a file called 0007-increase-semaphores.patch containing:

--- a/include/limits.h
+++ b/include/limits.h
@@ -64,7 +64,7 @@
 #define PTHREAD_STACK_MIN 2048
 #define PTHREAD_DESTRUCTOR_ITERATIONS 4
 #define SEM_VALUE_MAX 0x7fffffff
-#define SEM_NSEMS_MAX 256
+#define SEM_NSEMS_MAX 4096
 #define DELAYTIMER_MAX 0x7fffffff
 #define MQ_PRIO_MAX 32768
 #define LOGIN_NAME_MAX 256

Edit APKBUILD and add 0007-increase-semaphores.patch to the list in the source variable. Update the SHA1 sums in the APKBUILD with,

abuild checksum

Build the updated package with:

abuild -c -r -P /target

Create an index file

cd /target/main/armfh
apk index -o APKINDEX.tar.gz *.apk
abuild-sign APKINDEX.tar.gz

Exit from the container and find the built package in target/main/armhf

I can then add the .apk binary to my resin git repo and use apk add --allow-untrusted foo.apk in my Dockerfile.

8

Glad it worked. I was checking out that project to cross-compile, but you got there first :slight_smile:
For one-off builds that is likely fine, just keeping the package up to date, a more repeatable setup is likely going to be helpful for you.

Cheers!