How can I increase the partition size of my containers so I can add to /lib/module?

[diydsp]

I’m trying to build ReSpeaker’s RPi driver under BalenaOS. The repo and instructions are at: https://github.com/respeaker/seeed-voicecard

I’ve been adapting their install.sh file. At one point, it downloads a number of headers to /lib/modules (see below) and the drive fills up.

Is there a way to resize the partitions? Or a way to tell docker to make them a certain size or to add extra space? Thank you.

Partitions are currently 312MB:
root@7a59785:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
mmcblk0 179:0 0 59.2G 0 disk
|-mmcblk0p1 179:1 0 40M 0 part /mnt/boot
|-mmcblk0p2 179:2 0 312M 0 part /mnt/sysroot/active
|-mmcblk0p3 179:3 0 312M 0 part

It fills up /lib/module after calling this command:
+ apt-get -y --force-yes install raspberrypi-kernel-headers raspberrypi-kernel
Reading package lists… Done
Building dependency tree
Reading state information… Done
raspberrypi-kernel-headers is already the newest version (1.20190819~stretch-1).
The following NEW packages will be installed:
raspberrypi-kernel
0 upgraded, 1 newly installed, 0 to remove and 7 not upgraded.
Need to get 35.4 MB of archives.
After this operation, 126 MB of additional disk space will be used.
Get:1 http://archive.raspberrypi.org/debian stretch/main armhf raspberrypi-kernel armhf 1.20190819~stretch-1 [35.4 MB]
Fetched 35.4 MB in 10s (3461 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database … 61127 files and directories currently installed.)
Preparing to unpack …/raspberrypi-kernel_1.20190819~stretch-1_armhf.deb …
Leaving ‘diversion of /boot/bcm2708-rpi-b-plus.dtb to /usr/share/rpikernelhack/bcm2708-rpi-b-plus.dtb by rpikernelhack’

Then, this “Leaving diversion” message is repeated a number of times with different files, until it gets to here:
Leaving ‘diversion of /boot/overlays/w1-gpio.dtbo to /usr/share/rpikernelhack/overlays/w1-gpio.dtbo by rpikernelhack’
Leaving ‘diversion of /boot/overlays/wittypi.dtbo to /usr/share/rpikernelhack/overlays/wittypi.dtbo by rpikernelhack’
Unpacking raspberrypi-kernel (1.20190819~stretch-1) …
dpkg: error processing archive /var/cache/apt/archives/raspberrypi-kernel_1.20190819~stretch-1_armhf.deb (–unpack):
cannot copy extracted data for ‘./lib/modules/4.19.66-v7+/kernel/fs/cifs/cifs.ko’ to ‘/lib/modules/4.19.66-v7+/kernel/fs/cifs/cifs.ko.dpkg-new’: failed to write (No space left on device)
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
/var/cache/apt/archives/raspberrypi-kernel_1.20190819~stretch-1_armhf.deb
W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Sub-process /usr/bin/dpkg returned an error code (1)

[saintaardvark]

Hi @diydsp - welcome to the forum! I replied to your other post on this topic; I suggested some approaches to building a custom image that contains your module. This is a better approach in the long run – BalenaOS is optimized for running applications in containers, and we don’t offer a way to customize the partition sizes.

You can look at our kernel-module-build repo, or other approaches people have taken to accomplish this. I’ve also suggested to the Balena Labs team that it might be helpful to publish a tutorial that uses a custom kernel module.

Thanks again,
Hugh

[diydsp]

Hi St. Aardvark, thank you for both of your replies. I’ll leave aside trying to change the file system and focus on building this out-of-tree kernel module.

I began by cloning that repo and pushing it to my balenafin. But I got an erro when it tries to run build.sh. Is this something I can fix easily? ty.

[main]     Step 6/7 : RUN ./build.sh fincm3 $VERSION example_module
[main]      ---> Running in 9bc973c0092b
[main]     /bin/sh: 1: ./build.sh: Permission denied
[main]
[main]     Removing intermediate container 9bc973c0092b
[main]     The command '/bin/sh -c ./build.sh fincm3 $VERSION example_module' returned a non-zero code: 126
[Info]     Uploading images
[Error]    An error occured: (HTTP code 404) no such image - no such image: 9abb5949e86f: No such image: 9abb5949e86f:latest
[Info]     Built on arm03
[Error]    Not deploying release.
[Info]     Still Working...
Remote build failed

[diydsp]

ah ok I fixed the permissions error by adding these two lines to Dockerfile.template:

RUN chmod +x build.sh
RUN chmod +x workarounds.sh

Will proceed to try out the rest now.

[diydsp]

Just a note, I added this, too:

RUN chmod +x run.sh

After that I was able to push my container to my balenafin. However, when it boots up, I got this message in the logs:

Supervisor starting
 main  OS Version is 2.46.1+rev3
 main  Loading module from example_module_fincm3_2.46.1+rev3*
 main  insmod: ERROR: could not load module example_module_fincm3_2.46.1+rev3*/hello.ko: No such file or directory
 main  rmmod: ERROR: Module hello is not currently loaded

I get the same message when I run ./run.sh in the main container…

Notice that the version numbers don’t match up. The directory is called 2.29.0+rev1, but the run.sh script comes up with 2.46.1+rev3:

root@7a59785:/usr/src/app# ls -l example_module_fincm3_2.29.0+rev1.prod
total 14
-rw-r--r-- 1 root root  245 Apr 30 14:19 Makefile
-rw-r--r-- 1 root root    0 Apr 30 14:19 Module.symvers
-rw-r--r-- 1 root root  389 Apr 30 14:19 hello.c
-rw-r--r-- 1 root root 3676 Apr 30 14:19 hello.ko
-rw-r--r-- 1 root root  865 Apr 30 14:19 hello.mod.c
-rw-r--r-- 1 root root 2184 Apr 30 14:19 hello.mod.o
-rw-r--r-- 1 root root 2168 Apr 30 14:19 hello.o
-rw-r--r-- 1 root root   68 Apr 30 14:19 modules.order

After that I attempted to hand-run the insmod command:
root@7a59785:/usr/src/app# insmod example_module_fincm3_2.29.0+rev1.prod/hello.ko
insmod: ERROR: could not insert module example_module_fincm3_2.29.0+rev1.prod/hello.ko: Invalid module format

Have you got any suggestions on how to proceed from here?

Thank you.

[ab77]

Hi there, just a quick check - are you setting the resin|balenaOS version in the build image to the same version as you expect to be using at runtime?

[diydsp]

Ah! thank you very much! For the benefit of others in the future, I believe the way to find out the version is to run this in the container:
set | grep RESIN_HOST_OS_VERSION
RESIN_HOST_OS_VERSION=‘balenaOS 2.46.1+rev3’

Or is it better to examine the variable BALENA_HOST_OS_VERSION ?

And then, I modified the Dockerfile.template to look like this:
ENV VERSION ‘2.46.1+rev3’

After these mods, I was able to build and run the ./run.sh file and it indicates the mod was successfully inserted!

root@7a59785:/usr/src/app# ./run.sh 
OS Version is 2.46.1+rev3
Loading module from example_module_fincm3_2.46.1+rev3.dev
hello                  16384  0
Loading module from example_module_fincm3_2.46.1+rev3.dev_from_src
hello                  16384  0
Loading module from example_module_fincm3_2.46.1+rev3.prod
hello                  16384  0
Loading module from example_module_fincm3_2.46.1+rev3.prod_from_src
hello                  16384  0

So thank you very much St. Aardvark and ab77! Now my next step is to try to build the ReSpeaker driver over at: https://github.com/respeaker/seeed-voicecard

[diydsp]

Hi! I need to follow up with some more questions:

I have followed the instructions in the kernel module build repo and I was able to build my driver. I get three lovely .ko files from it. yay!

However, when I run insmod on the modules, I get:
insmod: ERROR: could not insert module snd-soc-wm8960.ko: Unknown symbol in module

when I check dmesg, I see:
[May 5 15:06] snd_soc_wm8960: Unknown symbol snd_soc_component_read32 (err -2)
[ +0.007282] snd_soc_wm8960: Unknown symbol snd_soc_put_enum_double (err -2)
[ +0.007028] snd_soc_wm8960: Unknown symbol snd_soc_component_write (err -2)
[ +0.007242] snd_soc_wm8960: Unknown symbol snd_soc_get_enum_double (err -2)
[ +0.007150] snd_soc_wm8960: Unknown symbol snd_soc_info_volsw (err -2)
(many more symbols are there, too).

And just for verification, I run cat /proc/kallsyms | grep snd_soc_ I don’t see anything… of course there are some symbols for snd_ctl and similar.

Does this mean I am missing a kernel module that contains soc_soc_* symbols? How can I find out what it is? is there a package I can install? I have been installing as man alsa- packages as I can find, but none are helping out.

[garethtdavies]

Hi, there is a similar forum thread here Can not install in-tree kernel module, which has some additional advice. Can you check that first and let us know if you continue to have issues?

[dedline]

@diydsp I am now at the same juncture. What worries me is that the Makefile is supposed to copy the files to /lib/modules/$(uname_r)/kernel/*, but the they are not present.

In the thread that @garethtdavies lists, @ErikHH is able to load the generics via modprobe because they exist in /lib/modules/*.

When I go to load .ko files via modprobe, I experience:

modprobe: FATAL: Module snd-soc-ac108 not found in directory /lib/modules/4.19.75

I think it might be issues between the build.sh of kernel-module-build and seeed Makefile.

I’m going to explore deeper.

[dedline]

I can’t get past this

Any luck @diydsp ?

[gelbal]

Hello @dedline and @diydsp, we are looking into this topic and we hope to prepare some instructions on how to build the out-of-tree kernel modules for this microphone. We will ping back to this thread as we make progress. Stay tuned : )

[diydsp]

Hi @dedline, great to see someone else trying this. And @gelbal, we’d be so grateful to have your eyes on this.

Let me share my updated instructions - see below!
I am able to get quite far, but not quite use the microphone yet…
I am doing the same thing in parallel on my RPi where it works and it looks like the big difference is that on RPi, I end up with a /dev/controlC0 and controlC1. I think the C1 is the new device made by seeed driver, but it doesn’t show up on my balenafin…yet

Here is the current error I get:
root@7a59785:/usr/src/app# arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
arecord: main:788: audio open error: No such device

and I’m guessing(?) it’s about control C1, because of the message when I strace this… Here are some excerpts from that strace:

open("/dev/snd/controlC0", O_RDWR|O_CLOEXEC) = 3
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
ioctl(3, SNDRV_CTL_IOCTL_PVERSION, 0x7edb5eb4) = 0
ioctl(3, SNDRV_CTL_IOCTL_CARD_INFO, 0x7edb5f0c) = 0
close(3) = 0
open("/dev/snd/controlC1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/dev/aloadC1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[ … ]
open("/dev/snd/controlC31", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/dev/aloadC31", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, “ALSA lib pcm_hw.c:1713:(_snd_pcm”…, 42ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) ) = 42
write(2, “Invalid value for card”, 22Invalid value for card) = 22
write(2, “\n”, 1
) = 1

Steps to build respeaker drivers from boot-up
export PS1=’\w>’
cd ../..
cp -rv src/kernel-module-build2/* .
export VERSION=2.46.1+rev3
chmod +x *.sh
./build.sh fincm3 $VERSION seeed-voicecard

Place to look for stuff
ls /dev/snd
On RPi, you end up with pcmC0* AND pcmC1*
A.k.a. controlC0 and controlC1.
Probably C1 is respeaker HW…

From install.sh
cd seeed-voicecard_fincm3_2.46.1+rev3.dev
mkdir /boot/overlays
cp seeed-2mic-voicecard.dtbo /boot/overlays
cp seeed-4mic-voicecard.dtbo /boot/overlays
cp seeed-8mic-voicecard.dtbo /boot/overlays

Further installation commands
# from “seeed-voicecard/”
# need to turn on 12c_arm
echo dtparam=i2c_arm=on >> /boot/config.txt
echo dtoverlay=i2s-mmap >> /boot/config.txt
echo dtparam=i2s=on >> /boot/config.txt
mount -t configfs none /config

# /etc/voicecard
mkdir /etc/voicecard || true
cp *.conf /etc/voicecard
cp *.state /etc/voicecard
cp seeed-voicecard /usr/bin/
cp seeed-voicecard.service /lib/systemd/system/

Prototype script to insert modules
cd seeed-voicecard_fincm3_2.46.1+rev3.dev
modprobe snd_soc_core
insmod snd-soc-wm8960.ko
modprobe snd_soc_simple_card_utils
insmod snd-soc-seeed-voicecard.ko
modprobe snd_soc_simple_card_utils
insmod snd-soc-ac108.ko
lsmod | grep snd

Recording test command
(this works on RPi)
arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav

[saintaardvark]

Hi @diydsp – thanks for the update, and glad to see you’re making progress. There are a couple problems we’re discussing now, so I want to make sure I keep them straight.

The first problem is that (as @dedline encountered), there are problems running a simple modprobe of the drivers. We’ll have something to share shortly, but have managed to get around it by:

• running modprobe snd-soc-simple-card first – this is required by the seeed modules;
• and running insmod /full/path/to/snd-soc-ac108.ko (where /full/path/to is, of course, meant to be the full path to wherever the drivers live in the built containers).

With those changes, I’m able to load the modules on a Raspberry Pi 4. Keep in mind I do not have this hardware, so all I can tell you is that the modules load without any complaints. As I say, we should have code to share shortly. In the meantime – @diydsp, are you able to share the code that you have? It would be good to compare notes about how this was solved.

The second problem is whether or not these drivers work on a Balena Fin. @diydsp, you gave the example of arecord not working properly. It looks like arecord has options for setting the device it uses; can you try getting it to list the devices (--list-devices, --list-pcms) and see if you’re able to specify one of those with --device? If this doesn’t work, and further debugging is needed, I think it will be worth starting a new thread so that it’s not confused with the module issues discussed previously.

All the best,
Hugh

[diydsp]

Hi! Thank you for writing back. Yes, w/r/t the first problem, I am able to insert the modules similar to you. I don’t have any code, per se, because my application is simple enough that I can use aplay and arecord from python. But you can see the steps I used above to use the Balena out-of-tree code to build the Respeaker driver. That’s the line “./build.sh fincm3 etc”. If you’d like clarifications, I can go further. Once I am able to use the aplay and arecord commands, I will be in good shape.

The other steps I have above are extracted from the install.sh command which Seeed supplies in their github repo. Their stock install.sh does not work on BalenaFin/OS because it relies on many specific RPi things, such as in config.txt. So I am trying to make a new set of installation commands.

Here is my set of commands (from the above post) for inserting the kernel modules:
modprobe snd_soc_core
insmod snd-soc-wm8960.ko
modprobe snd_soc_simple_card_utils
insmod snd-soc-seeed-voicecard.ko
modprobe snd_soc_simple_card_utils
insmod snd-soc-ac108.ko
lsmod | grep snd

And for testing recording (this works on RPi):
arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav

W/r/t the second problem, yes, getting the drivers to do work is the next big challenge…
Below please see the output of aplay -L once my new installation steps are run. We can see that in the first few lines, new lines for the ReSpeaker mic array are visible. After that, the usual ones for BalenaFin appear. When I run the same command on the RPi, with working ReSepeaker mic array, I see the Seeed drivers appear just the same way. However, when I try to record with the codec, the strace shows some kind of error, possibly to do with not finding the proper /dev/snd/controlC* ?

I have been giving myself a crash-course in ASoC (alsa for System on a Chip). It seems to me that the codec drivers are working but there is no “card” appearing for this device. I’m still learning at a rapid pace… A new thread would be great, too. Thank you for your help.

Listing PCMS
root@7a59785:/usr/src/app/src/# aplay -L
null
Discard all samples (playback) or generate zero samples (capture)
default
ac108
dmixer
ac101
sysdefault:CARD=ALSA
bcm2835 ALSA, bcm2835 ALSA
Default Audio Device
dmix:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample mixing device
dmix:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample mixing device
dmix:CARD=ALSA,DEV=2
bcm2835 ALSA, bcm2835 IEC958/HDMI1
Direct sample mixing device
dsnoop:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample snooping device
dsnoop:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample snooping device
dsnoop:CARD=ALSA,DEV=2
bcm2835 ALSA, bcm2835 IEC958/HDMI1
Direct sample snooping device
hw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct hardware device without any conversions
hw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct hardware device without any conversions
hw:CARD=ALSA,DEV=2
bcm2835 ALSA, bcm2835 IEC958/HDMI1
Direct hardware device without any conversions
plughw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Hardware device with all software conversions
plughw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Hardware device with all software conversions
plughw:CARD=ALSA,DEV=2
bcm2835 ALSA, bcm2835 IEC958/HDMI1
Hardware device with all software conversions

Here’s the output of aplay --list-devices
root@7a59785:/usr/src/app/src/acoustic_sensors_acquire_sw# aplay --list-devices
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
Subdevices: 7/7
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 IEC958/HDMI [bcm2835 IEC958/HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: ALSA [bcm2835 ALSA], device 2: bcm2835 IEC958/HDMI1 [bcm2835 IEC958/HDMI1]
Subdevices: 1/1
Subdevice #0: subdevice #0

[MTolba]

Hi @diydsp Could you please check this repo of one of our Engineers https://github.com/richbayliss/resin-seeed-voicecard. It will help you go through your project.

The build kernel part is a little bit outdated other than that everything is fine.

Generally, you don’t have to create a custom image for such projects and you can only do it using containers.

Please don’t hesitate to let us know if you still need help

All the best with your tinkering
Mahmoud

[dedline]

Cannot get this script to work with updating kernel headers from https://files.balena-cloud.com/

[georgiats]

Could you please provide us some extra context about the issue you are facing with this script? Do you get any specific error messages?

[dedline]

I follow your instructions to the T.

I am able to finally insert the modules via the commands that you provided

Evidence:
1

I throw in alsa-utils to run arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav

I get the following error:
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM ac108
arecord: main:828: audio open error: No such file or directory

I am going to give it one more clean install / going through the process but I am growing discouraged.

[dedline]

ARG HEADERS_URL=“https://files.balena-cloud.com/esr-images/raspberrypi4-64/2020.04.0.dev/kernel_modules_headers.tar.gz”
ARG HEADERS_PATH=/tmp/kernel_modules_headers