"can't send spi message" error when running Matrix Creator examples on resinOS

Hello,

I am using a Raspberry Pi 3 with a Matrix Creator board. Using the Matrix documentation here I installed the necessary dependencies for the board through the dockerfile and tried to run the “mic_energy_direct” demo here.

However, I get the following output when running any of the matrix examples that involve the microphones:
“can’t send spi message”

Any help would be greatly appreciated. Thank you!

Also, here is my full dockerfile installing matrix dependencies, for reference. The forum does not seem to allow me to post it in text due to it having more than two links in it:

Hi @pcg108 - I can think of two things to try, judging from this: https://github.com/matrix-io/matrix-creator-quickstart/wiki/enable-spi and looking at how raspi-config works:

  1. Ensure that SPI is enabled. This is true by default in recent resinOS versions (what version are you using?). In the resin dashboard’s “Device configuration” page, check that “Define DT parameters / RESIN_HOST_CONFIG_dtparam” includes “spi=on” - default is "i2c_arm=on","spi=on","audio=on" and if that is set then it should be fine.

  2. Ensure that the SPI kernel module is loaded, by changing your CMD line in the Dockerfile to:

CMD modprobe spi-bcm2708 && python3 src/loop.py

(I’m not sure if it’s spi-bcm2708 or spi_bcm2708 so maybe try the second one if the first one doesn’t work)

Hi @pcarranzav thank you for your reply! I double checked the device configuration and spi is indeed on, and I did step 2 as well (I had to change it to spi-bcm2835 since apparently the kernel module has been updated from spi-bcm2708 according to this)

Hi @pcg108 - another idea. It seems like you need to program the FPGA on the Matrix on every boot, and this is handled by the matrixio-creator-init package https://github.com/cmetz/matrix-creator-init - but this works by starting a systemd service, and you don’t have systemd enabled in your container.

So I think what you could try is to start the script from that package manually:

CMD /usr/share/matrixlabs/matrixio-devices/matrix-init.bash && python3 src/loop.py

(Alternatively, you could enable systemd in your container by setting ENV INITSYSTEM on, and I think that would automatically start the service for you, but dunno if it would have other side effects in this case)

Interesting, I tried this and I get the following:

INFO: [/dev/spidev0.0] was opened
MATRIX device has not been detected

Then when I try running even the LED demos which used to work, they no longer work since the Matrix Creator is not detected. Also in my dockerfile there was an

ENV INITSYSTEM on

already, and when I removed it the FPGA was reprogrammed but the same error (can’t send spi message) appears to be there

@pcg108 that sounds odd.
Maybe try running it all after a reboot? Just to make sure it’s all starting from a clean state, I don’t know how the Creator works in that sense.

There’s also this: https://github.com/cmetz/matrix-creator-init/blob/master/matrix-mics.conf so another thing to try would be to modprobe those two modules too (though I can’t quite see how that would relate with an SPI message failing)

Hi there!
Did you get a chance to dig into the suggestions from Pablo, we’re keen to get to this worked out, but need some help from your end.

Hi was this ever solved? I am trying to also use matrix-hal with balena.

Hey, are you having the exact same issue as the one presented above? We can try to debug your issues too, but we’ll need some further information. If the issue is sufficiently different it’s probably worth starting a new forum thread.

This was the original issue, so yes I am having the exact same issue when trying to read microphone data. I’ve created a slightly modified version of the above dockerfile based on balenas official systemd based image, which allowed my to install matrixio-creator-init. I also have privileged mode enabled in my docker compose file.

These modifications allow me to control the led lights and read from the other sensors, but it doesn’t allow me to read any data from the microphones. It just says cannot send spi message.

Here is the exact setup I am using. https://github.com/airfield20/Matrix-Creator-BalenaOS
I would also like to be able to install the matrixio-kernel-modules package, but im not sure how exactly.

The repo for the kernel modules is here https://github.com/matrix-io/matrixio-kernel-modules . The repo states “This drivers only works with current stock raspbian kernel.”. How similar is this kernel to the stock raspberry kernel? If it is too dissimilar then is there any way to use the stock raspbian kernel or maybe even emulate it to get this device working?

Thank you for any help you may be able to provide.

Hi @airfield20, sounds like you need to install out-of-tree kernel modules to get the microphone working. We have an example project to help you on that front: GitHub - balena-os/kernel-module-build: Example project for building an OOT kernel module in balena

Then this topic of building out-of-tree kernel modules is coming up quite frequently lately. I asked my teammates to write a blog post about it.

"This drivers only works with current stock raspbian kernel.”. How similar is this kernel to the stock raspberry kernel?

We are using the Raspberry Pi Foundation kernel at GitHub - raspberrypi/linux: Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/

As far as we know, raspbian packages this kernel but it may lag behind.

So I suggest trying out building the kernel module yourself. Let us know how it goes and if you run into a specific issue.

Thanks, I have been working on this manual installation for some time. I will let you know how it goes. The original issue still remains though, issues using SPI. Even though lights and sensors are working, any attempt to read microphone data using matrix hal returns “can’t send spi message”. Do you have any advice for this issue?

@airfield20 have you enabled spi in the config.txt file like described here https://www.balena.io/docs/learn/develop/hardware/i2c-and-spi/#spi ?

That documentation states that it is enabled by default. I did check the device parameter through balenaos and it is at the default setting. Also in an earlier attempt i installed raspi-config in the container and enabled spi but to no avail. The issue was still present.

You’ll need to build https://github.com/matrix-io/matrixio-kernel-modules like in this example https://github.com/balena-os/kernel-module-build/ .
Once you have a kernel module built, you’ll need to insmod it from a privileged container.
The build should also create a matrixio.dtbo file that you will need to place in the overlays folder of the boot partition of your SD card.
You will also need to update the config.txt file in the boot partition of the sd card to add dtoverlay=matrixio.

I don’t think you need to run systemd in your container.

Sorry for the late response, it took me a minute to figure out the correct Balena_machine_name and version string to use. The image is showing the results of build.sh using
ENV VERSION '2.50.1+rev1.prod'
RUN ./build.sh raspberrypi4-64 $VERSION matrixio-kernel-modules/src
in the Dockerfile.

The list of errors are:

[main]     make: Entering directory '/tmp/tmp.d6J9bceBy8'
[main]     arch/arm64/Makefile:27: ld does not support --fix-cortex-a53-843419; kernel may be susceptible to erratum
[main]     
[main]     arch/arm64/Makefile:40: LSE atomics not supported by binutils
[main]     
[main]       CC [M]  /usr/src/app/matrixio-kernel-modules/src_raspberrypi4-64_2.50.1+rev1.prod/matrixio-core.o
[main]     gcc: error: unrecognized command line option '-mgeneral-regs-only'

I haven’t found much info on those errors so far. Could you point me in the right direction? Thanks

Hi @airfield20,

Boy, there’s a lot of history for me to read on this one. Specifically about the LSE atomics not supported by binutils error; is the build environment based on Alpine? That’s my first guess. My googling for this error seems to be related to building kernels for NVIDIA machines which I am not sure are relevant.

If it is alpine can you try building in a Debian-based container instead?

Cheers,
James.