"can't send spi message" error when running Matrix Creator examples on resinOS

saintaardvark · June 4, 2020, 9:12pm

Hi @airfield20 – thanks kindly for the additional info. You’re right, it definitely looks like the modules are being loaded. (It also looks like you may have run modprobe for the matrix modules as well as insmod; I had intended to suggest that for the other modules you were loading – snd_bcm2835, snd_dummy and so on.)

arecord can list the soundcards and devices it can see with the -l and -L (lower case and capital L) arguments. On my laptop, I get this output:

$ arecord -l 
**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC3271 Analog [ALC3271 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: J370 [Jabra Link 370], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Audio [CalDigit Thunderbolt 3 Audio], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
$ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
pulse
    PulseAudio Sound Server
default
    Default ALSA Output (currently PulseAudio Sound Server)
sysdefault:CARD=PCH
    HDA Intel PCH, ALC3271 Analog
    Default Audio Device
sysdefault:CARD=J370
    Jabra Link 370, USB Audio
    Default Audio Device
sysdefault:CARD=Audio
    CalDigit Thunderbolt 3 Audio, USB Audio
    Default Audio Device

Given that output, arecord -Dhw:1,0 test.wav works for me (first device, first subdevice). What do those commands show for you, and is it possible you need to adjust the arguments you use for arecord? If that works, it’s possible that the demo programs you tested need to be adjusted to try recording from different devices.

You may find the Arch Linux wiki page on ALSA troubleshooting helpful. I don’t know what troubleshooting resources are available for the Matrix Creator, but it would be worth checking there as well.

All the best,
Hugh

airfield20 · June 4, 2020, 9:18pm

Running those same commands gives me this:

Spawning shell...
root@5a54ff0:/usr/src/app# arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: MATRIXIOSOUND [MATRIXIO-SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
root@5a54ff0:/usr/src/app# arecord -Dhw:2,0 test.wav
arecord: main:828: audio open error: Invalid argument

As you can see I am using the correct device string hw:2,0 for the matrix microphone but alsa is not picking up anything. Since the modules are loading correctly then the issue must have something to do with alsa or alsa configuration.

airfield20 · June 4, 2020, 9:19pm

Here is the beginning of my asound.conf file

root@5a54ff0:/usr/src/app# cat /etc/asound.conf 
pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}

pcm.mic {
   type softvol
   slave {
     pcm "array"
   }
   control {
     name "MicArray Master"
        card "MATRIXIOSOUND"
   }
}

pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
  }
}

pcm.array {
  type plug
  slave {
    pcm "hw:MATRIXIOSOUND"
  }
}

pcm.channel_1 { ...

floion · June 4, 2020, 10:13pm

Hi, have you also added the matrixio dtoverlay to the rootfs and added it in config.txt through the balena dashboard?

airfield20 · June 4, 2020, 10:28pm

Yes I copied the dtoverlay file to the sd card directly by mounting in my desktop computer, and I have the set the /boot/config.txt file through the dashboard:

I also added this to my dockerfile ENV BALENA_HOST_CONFIG_dtoverlay matrixio to make sure its added.

floion · June 4, 2020, 10:54pm

Ok. Can you double check on the running system by using the dashboard webterminal for the hostOS that the overlay exists in /mnt/boot/overlays and also cat /mnt/boot/config.txt and paste its contents here?

airfield20 · June 4, 2020, 10:58pm

Yep sure thing.

root@5a54ff0:~# ls /mnt/boot/overlays | grep matrix
matrixio.dtbo

root@5a54ff0:~# cat /mnt/boot/config.txt
disable_splash=1
dtoverlay=matrixio
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
enable_uart=1
gpu_mem=16

Looks like those files are installed correctly

airfield20 · June 4, 2020, 11:24pm

After running build.sh from the kernel module build example here https://github.com/balena-os/kernel-module-build , where would the matrixio.dtbo file be stored at? It is not in the same directory as the .ko files.

robertgzr · June 8, 2020, 8:33am

Hi, this looks right. Your message earlier seems to be spot on. This is likely an ALSA issue now, so let’s focus on that. Can you elaborate on what you tried to far?

airfield20 · June 8, 2020, 7:28pm

Hi, with alsa so far I have just tried arecord commands:

root@5a54ff0:/usr/src/app/matrix-creator-hal/build/demos# arecord recording.wav -f S16_LE -r 16000 -d 5
arecord: main:828: audio open error: Invalid argument
root@5a54ff0:/usr/src/app/matrix-creator-hal/build/demos# aplay recording.wav
recording.wav: No such file or directory

I am asking about the dtbo file because it did not exist in the kernel modules directory so I used the dtbo file from my raspbian installation. I’m not sure if that would cause an issue.

airfield20 · June 8, 2020, 7:41pm

I’m not sure if this is relevant but maybe systemd is causing the issue.

root@5a54ff0:/usr/src/app# systemctl status systemd-modules-load
● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2020-06-08 19:27:31 UTC; 11min ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
  Process: 72 ExecStart=/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 72 (code=exited, status=1/FAILURE)

Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-everloop'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-gpio'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-imu'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-env'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-mic'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-playback'
Jun 08 19:27:31 5a54ff0 systemd-modules-load[72]: Failed to find module 'matrixio-codec'
Jun 08 19:27:31 5a54ff0 systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
Jun 08 19:27:31 5a54ff0 systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.
Jun 08 19:27:31 5a54ff0 systemd[1]: Failed to start Load Kernel Modules.
root@5a54ff0:/usr/src/app# lsmod
Module                  Size  Used by
ip6t_REJECT            16384  1
nf_reject_ipv6         16384  1 ip6t_REJECT
ipt_REJECT             16384  1
nf_reject_ipv4         16384  1 ipt_REJECT
xt_state               16384  0
ip6table_filter        16384  1
matrixio_regmap        61440  0
matrixio_imu           16384  0
matrixio_env           16384  0
matrixio_gpio          16384  0
matrixio_playback      49152  0
matrixio_mic           16384  0
matrixio_codec         16384  1
matrixio_everloop      16384  0
matrixio_uart          16384  0
matrixio_core          16384  7 matrixio_everloop,matrixio_regmap,matrixio_playback,matrixio_imu,matrixio_mic,matrixio_uart,matrixio_env
industrialio           73728  2 matrixio_imu,matrixio_env
snd_soc_core          192512  3 matrixio_playback,matrixio_codec,matrixio_mic
snd_compress           20480  1 snd_soc_core
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_dummy              24576  1
dummy                  16384  0
ip6_tables             24576  1 ip6table_filter
bnep                   20480  2
hci_uart               40960  1
btbcm                  16384  1 hci_uart
serdev                 20480  1 hci_uart
ipt_MASQUERADE         16384  3
nf_conntrack_netlink    40960  0
bluetooth             397312  24 hci_uart,bnep,btbcm
nfnetlink              16384  2 nf_conntrack_netlink
ecdh_generic           28672  1 bluetooth
br_netfilter           24576  0
xt_owner               16384  0
i2c_dev                20480  0
spidev                 20480  0
brcmfmac              319488  0
brcmutil               20480  1 brcmfmac
bcm2835_v4l2           49152  0
bcm2835_codec          36864  0
sha256_generic         20480  0
v4l2_mem2mem           24576  1 bcm2835_codec
bcm2835_mmal_vchiq     32768  2 bcm2835_codec,bcm2835_v4l2
raspberrypi_hwmon      16384  0
hwmon                  16384  1 raspberrypi_hwmon
v4l2_common            16384  1 bcm2835_v4l2
snd_bcm2835            24576  1
videobuf2_dma_contig    20480  1 bcm2835_codec
cfg80211              630784  1 brcmfmac
videobuf2_vmalloc      16384  1 bcm2835_v4l2
videobuf2_memops       16384  2 videobuf2_dma_contig,videobuf2_vmalloc
snd_pcm                98304  6 matrixio_playback,snd_pcm_dmaengine,matrixio_mic,snd_bcm2835,snd_soc_core,snd_dummy
videobuf2_v4l2         24576  3 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem
rfkill                 28672  6 bluetooth,cfg80211
videobuf2_common       45056  4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
lan78xx                49152  0
snd_timer              32768  1 snd_pcm
videodev              200704  6 bcm2835_codec,v4l2_common,videobuf2_common,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
snd                    73728  11 snd_compress,matrixio_playback,snd_timer,snd_bcm2835,snd_soc_core,snd_dummy,snd_pcm
media                  36864  3 bcm2835_codec,videodev,v4l2_mem2mem
i2c_bcm2835            16384  0
vc_sm_cma              36864  1 bcm2835_mmal_vchiq
spi_bcm2835            20480  0
uio_pdrv_genirq        16384  0
uio                    20480  1 uio_pdrv_genirq
fixed                  16384  0
sch_fq_codel           20480  4
root@5a54ff0:/usr/src/app#

Systemd says failed to find module however lsmod shows its loaded.

jimsynz · June 8, 2020, 9:54pm

Hi again @pcg108,

I’m not sure what systemd is going on about, to be honest. You’re right that the modules are loaded, so let’s put that to the side for now.

I did some googling about the ALSA configuration for this board, and found this old issue where the user was trying to use the board on resin (balena’s old name) and it looks like there may be some good pointers there to help us fix the issue.

I suspect the issue is going to be around the getting the asound.conf set up correctly.

As per this comment - can you open a shell and run cat /proc/asound/modules and paste the output here. I have a feeling that changing the default card may be enough to make it work.

Cheers,
James.

airfield20 · June 8, 2020, 10:17pm

here is my modules file

root@5a54ff0:/usr/src/app# cat /proc/asound/modules
 0 snd_bcm2835
 1 snd_dummy
 2 matrixio_codec

jimsynz · June 8, 2020, 10:24pm

Okay. According to this page you need to set the default card using card number listed in /proc/asound/cards. Can you give that a go?

airfield20 · June 8, 2020, 10:39pm

I tried setting the default to snd_dummy and recording, then rebooting through the dashboard and recording and got the same result. Then I set the default to matrixio_codec and got the same result each time:

root@5a54ff0:/usr/src/app# arecord recording.wav -f S16_LE -r 16000 -d 5
arecord: main:828: audio open error: Invalid argument

jimsynz · June 8, 2020, 10:48pm

I wonder if the config is even being loaded? What path are you using for the alsa.conf? Maybe we can try explicitly setting the device to record from - the arecord man page says you can use -D and select the device to use by name. Also, apparently adding some -v's to the command will give us more information. Is there anything in your dmesg output from the kernel?

airfield20 · June 8, 2020, 11:08pm

The path to alsa.conf is /usr/share/alsa/alsa.conf. I’ve tested specifying the device and the -v’s do not add any additional information unfortunately. however dmesg showed something interesting:

[   28.991986] matrixio_core: loading out-of-tree module taints kernel.
[   29.037972] matrixio-uart matrixio-uart: DMA mask not set
[   29.109141] matrixio-codec matrixio-codec: DMA mask not set
[   29.116069] matrixio-codec matrixio-codec: snd-soc-dummy-dai <-> matrixio-mic.0 mapping ok
[   29.125452] matrixio-codec matrixio-codec: snd-soc-dummy-dai <-> matrixio-pcm-out.0 mapping ok
[   29.169736] matrixio-mic matrixio-mic: DMA mask not set
[   29.175416] matrixio-mic matrixio-mic: MATRIXIO mic array audio driver loaded (IRQ=169)
[   29.203631] matrixio-playback matrixio-playback: DMA mask not set
[   29.229224] matrixio-gpio matrixio-gpio: DMA mask not set
[   29.282900] matrixio-env matrixio-env: DMA mask not set
[   29.316987] matrixio-imu matrixio-imu: DMA mask not set

richbayliss · June 10, 2020, 11:55am

Hey,

I was looking into this and it seems there may be a possibility that the modules are not compatible entirely with balenaOS… I have never come across this personally but perhaps it’s worth you following up with the vendor to see why it requires a raspbian kernel specifically. I think that since you have shown the DTBO to be loaded and the kernel modules also loaded, that it’s possible the modules are just not interfaced correctly, but not throwing any errors.

Here is the warning I found: https://github.com/matrix-io/matrixio-kernel-modules#warning-kernel-version

Please update us with your findings.

airfield20 · June 10, 2020, 1:32pm

Unfortunately the vendor has come to the same conclusion on their support forum. They say they have never used balena os and their must be limited hardware access.

I’m looking at trying this solution https://github.com/balena-os/balenaos-in-container
Running in raspbian lite host os. What features will i lose by using this instead of the yocto based host. Also do you know if there is a roadmap for this solution that’s publicly viewable?

richbayliss · June 10, 2020, 2:31pm

So this is still very much in active development and I am not aware of any internal roadmap or plan for this to be a 1st class supported way to run balenaCloud managed apps.

and their must be limited hardware access.

I am not aware of any restricted hardware access in the kernel. It would be good to understand why they mention the Raspbian kernel as the only compatible version too…

Topic		Replies	Views
Sending to resin.io dashboard logs Product support	12	2170	January 23, 2018
Serial Data (`/dev/ttyACMx`) works unreliably on ResinOS - No problems on Raspbian host Product support support , raspberrypi3 , serial	4	835	July 28, 2017
I2c Connection to Relais Board Product support	4	797	August 10, 2017
Compute Module CM3 & Resin Product support	7	2427	March 15, 2017
Sharing docker-compose between Pi2 and Pi3, and accessing serial port Product support serial	1	2345	June 28, 2018

"can't send spi message" error when running Matrix Creator examples on resinOS

Related topics