Troubleshooting Cellular - no connection found

eweimholt · April 12, 2021, 6:05pm

I am following the steps in the above tutorial. Here are the hardware components I’m using:

Raspberry Pi 4 Model B
SIM card activated on Twilio’s Super SIM (APN=super)
Quectel Modem EC25-A
4G Antenna
SixFab Raspberry Pi 3G-4G/LTE Base HAT

The cellular file found on the device in HostOS:

root@bbcf9ff:/mnt/boot/system-connections# cat cellular
[connection]
id=cellular  
type=gsm  
autoconnect=true

[gsm]
apn=super
auto-config=true

[serial]
baud=115200

[ipv4]
method=auto

[ipv6]
addr-gen-mode=stable-privacy  
method=auto

However, once I boot up the device, there’s no IP address for a cellular connection, just the ethernet.

root@bbcf9ff:/mnt/boot/system-connections# nmcli connection show
NAME                UUID                                  TYPE      DEVICE      
Wired connection 1  4e4af7a3-4e99-3068-b7ee-428c7014e1bb  ethernet  enp1s0u1u4  
supervisor0         6679a202-5ba0-4c2f-9329-b6e9c6862f26  bridge    supervisor0 
Wired connection 2  4b423ef9-80c2-3c53-9af6-5070abd949fe  ethernet  --

The antenna is connected to MAIN on the Quectel Modem.

Any other tips for troubleshooting cellular connections?

rosswesleyporter · April 13, 2021, 12:54am

Hello @eweimholt,

Welcome to the forums!

Thank you for clearly describing the problem. You asked for troubleshooting tips, so let’s start with the docs:

And some additional thoughts:

mmcli -L to see a list of available modems
mmcli -m 0 hopefully under Status you’ll see state:connected
nmcli device show and check GENERAL.STATE

I hope this is helpful. Please let us know what you find.

rosswesleyporter · April 13, 2021, 1:19am

Hello again @eweimholt,

One of my colleagues just mentioned that he has somewhat similar hardware (e.g. a Pi4, Twilio Super SIM) and offered to do a test tomorrow. So you may hear from him as well.

eweimholt · April 13, 2021, 1:20am

Thanks for the pointers! Here’s the output from those commands. Under Status, the state is registered, not connected… I am not sure how to change the state. nmcli device show shows only the ethernet connection.

root@bbcf9ff:~# mmcli -L
        /org/freedesktop/ModemManager1/Modem/0 [QUALCOMM INCORPORATED] QUECTEL Mobile Broadband Module
    root@bbcf9ff:~# mmcli -m 0
      -----------------------------------
      General  |               dbus path: /org/freedesktop/ModemManager1/Modem/0
               |               device id: df4e20c78d327216b59c6615d5aa5a0ca26f6f77
      -----------------------------------
      Hardware |            manufacturer: QUALCOMM INCORPORATED
               |                   model: QUECTEL Mobile Broadband Module
               |       firmware revision: EC25AFAR05A06M4G
               |          carrier config: ROW_Generic_3GPP
               | carrier config revision: 05010824
               |            h/w revision: 10000
               |               supported: gsm-umts, lte
               |                 current: gsm-umts, lte
               |            equipment id: 861641043007193
      -----------------------------------
      System   |                  device: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3
               |                 drivers: option1, qmi_wwan
               |                  plugin: quectel
               |            primary port: cdc-wdm0
               |                   ports: cdc-wdm0 (qmi), ttyUSB0 (qcdm), ttyUSB2 (at), 
               |                          ttyUSB1 (gps), wwp1s0u1u3i4 (net), ttyUSB3 (at)
      -----------------------------------
      Status   |                    lock: sim-pin2
               |          unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (10), sim-puk2 (10)
               |                   state: registered
               |             power state: on
               |             access tech: lte
               |          signal quality: 78% (recent)
      -----------------------------------
      Modes    |               supported: allowed: 3g; preferred: none
               |                          allowed: 4g; preferred: none
               |                          allowed: 3g, 4g; preferred: 4g
               |                          allowed: 3g, 4g; preferred: 3g
               |                 current: allowed: 3g, 4g; preferred: 4g
      -----------------------------------
      Bands    |               supported: utran-4, utran-5, utran-2, eutran-2, eutran-4, eutran-12
               |                 current: utran-4, utran-5, utran-2, eutran-2, eutran-4, eutran-12
      -----------------------------------
      IP       |               supported: ipv4, ipv6, ipv4v6
      -----------------------------------
      3GPP     |                    imei: 861641043007193
               |             operator id: 310410
               |           operator name: AT&T
               |            registration: home
      -----------------------------------
      3GPP EPS |    ue mode of operation: csps-2
      -----------------------------------
      SIM      |               dbus path: /org/freedesktop/ModemManager1/SIM/0
    root@bbcf9ff:~# nmcli device show
    GENERAL.DEVICE:                         enp1s0u1u4
    GENERAL.TYPE:                           ethernet
    GENERAL.HWADDR:                         XX:XX:XX:XX:XX:XX
    GENERAL.MTU:                            1500
    GENERAL.DEVICE:                         enp1s0u1u4
    GENERAL.TYPE:                           ethernet
    GENERAL.HWADDR:                         XX:XX:XX:XX:XX:XX
    GENERAL.MTU:                            1500
    GENERAL.STATE:                          100 (connected)
    GENERAL.CONNECTION:                     Wired connection 1
    GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/1
    WIRED-PROPERTIES.CARRIER:               on
    IP4.ADDRESS[1]:                         192.168.0.33/24
    IP4.GATEWAY:                            192.168.0.2
    IP4.ROUTE[1]:                           dst = 0.0.0.0/0, nh = 192.168.0.1, mt = 100
    IP4.ROUTE[2]:                           dst = 192.168.0.0/24, nh = 0.0.0.0, mt = 100
    IP4.DNS[1]:                             75.75.75.75
    IP4.DNS[2]:                             75.75.76.76
    IP4.DOMAIN[1]:                          hsd1.il.comcast.net
    IP6.ADDRESS[1]:                         xxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.ADDRESS[2]:                         xxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.GATEWAY:                            fxxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.ROUTE[1]:                           dst = 2601:241:300:afd0::/60, nh = fe80::b2b9:8aff:fe06:7e62, mt = 100
    IP6.ROUTE[2]:                           dst = 2601:241:300:afd0::/64, nh = ::, mt = 100
    IP6.ROUTE[3]:                           dst = ::/0, nh = fe80::b2b9:8aff:fe06:7e62, mt = 20100
    IP6.ROUTE[4]:                           dst = fe80::/64, nh = ::, mt = 100
    root@bbcf9ff:~# nmcli device show
    GENERAL.DEVICE:                         enp1s0u1u4
    GENERAL.TYPE:                           ethernet
    GENERAL.HWADDR:                         XX:XX:XX:XX:XX:XX
    GENERAL.MTU:                            1500
    GENERAL.STATE:                          100 (connected)
    GENERAL.CONNECTION:                     Wired connection 1
    GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/1
    WIRED-PROPERTIES.CARRIER:               on
    IP4.ADDRESS[1]:                         192.168.0.33/24
    IP4.GATEWAY:                            192.168.0.1
    IP4.ROUTE[1]:                           dst = 0.0.0.0/0, nh = 192.168.0.1, mt = 100
    IP4.ROUTE[2]:                           dst = 192.168.0.0/24, nh = 0.0.0.0, mt = 100
    IP4.DNS[1]:                             75.75.75.75
    IP4.DNS[2]:                             75.75.76.76
    IP4.DOMAIN[1]:                          hsd1.il.comcast.net
    IP6.ADDRESS[1]:                         xxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.ADDRESS[2]:                        xxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.GATEWAY:                            xxxx:xxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx/xx
    IP6.ROUTE[1]:                           dst = 2601:241:300:afd0::/60, nh = fe80::b2b9:8aff:fe06:7e62, mt = 100
    IP6.ROUTE[2]:                           dst = 2601:241:300:afd0::/64, nh = ::, mt = 100
    IP6.ROUTE[3]:                           dst = ::/0, nh = fe80::b2b9:8aff:fe06:7e62, mt = 20100
    IP6.ROUTE[4]:                           dst = fe80::/64, nh = ::, mt = 100

rosswesleyporter · April 13, 2021, 2:25am

Hi @eweimholt,

Thank you for running the commands. That is helpful diagnostic information. Given what you found, please also try ``mmcli -b 0```.

eweimholt · April 13, 2021, 2:43am

Here is the output:

root@bbcf9ff:~# mmcli -b 0
error: couldn't find bearer at '/org/freedesktop/ModemManager1/Bearer/0': 'not found in any modem'

eweimholt · April 13, 2021, 8:23pm

I am able to manually create a connection by doing the following:

First, I connected to the device using ethernet, and opened a HostOS terminal. looking at nmcli d, we see that the gsm connection is disconnected.

root@c210748:~# nmcli d
DEVICE           TYPE      STATE                   CONNECTION         
enp1s0u1u4       ethernet  connected               Wired connection 2 
supervisor0      bridge    connected (externally)  supervisor0        
cdc-wdm0         gsm       disconnected            --                 
wlan0            wifi      disconnected            --                 
eth0             ethernet  unavailable             --                 
balena0          bridge    unmanaged               --                 
br-ee348fe92228  bridge    unmanaged               --                 
resin-dns        bridge    unmanaged               --                 
lo               loopback  unmanaged               --                 
resin-vpn        tun       unmanaged               --

I added a connection, naming it “twilio-connection” with the apn for Twilio Super SIM:

root@c210748:~# nmcli con add type gsm ifname cdc-wdm0 con-name twilio-connection apn super
Connection 'twilio-connection' (31c3e9a8-27d4-4237-bf96-448cc045a62a) successfully added.

Now, I have a cellular connection. I unplugged the ethernet cable to confirm that cellular is working.

root@c210748:~# nmcli d
DEVICE           TYPE      STATE                   CONNECTION        
cdc-wdm0         gsm       connected               twilio-connection 
supervisor0      bridge    connected (externally)  supervisor0       
wlan0            wifi      disconnected            --                
eth0             ethernet  unavailable             --                
balena0          bridge    unmanaged               --                
br-ee348fe92228  bridge    unmanaged               --                
resin-dns        bridge    unmanaged               --                
lo               loopback  unmanaged               --                
resin-vpn        tun       unmanaged               --

My follow-up question is, how can I add this to the boot image so that I don’t have to manually create a cellular connection on each device?

Note: I followed this post: raspberry pi - Modem on Raspbian not connecting with `nmcli` - Stack Overflow, but I did not have to run nmcli con up <connection-name>, it connected automatically once I added it

dtischler · April 13, 2021, 8:31pm

Hi @eweimholt, I am starting to test this now… Just got my Twilio Super SIM installed and bringing up the board. I’ll report back with my findings shortly.

dtischler · April 13, 2021, 11:00pm

Hi @eweimholt, I think I have narrowed down the issue to some degree as a bug that has crept in to more recent OS releases. I don’t know exactly when it was introduced quite yet, but flashing an older version of the OS to an SD Card, setting the connection profile info exactly as you outlined above, and booting up resulted in an automatic connection as is expected. This was done using the Pi4 + SixFab HAT + Quectel EC25-A + Twilio Super SIM. We will start investigating precisely where things got off track, but in the meantime I am going to test more OS versions and we can at least get you on a path forward.

eweimholt · April 14, 2021, 12:52am

Sounds good! I am using balenaOS v2.73.1+rev1

rosswesleyporter · April 14, 2021, 7:35pm

Hi @eweimholt,

dtischler now has a working setup using the current version of balenaOS. He started as you did from the blog post. He pasted directly from the blog into the cellular file. He then removed all extra spaces at the end of lines, specifically including removing extra spaces at the end of method=auto (the last line). Please give this a try.

The hypothesis here is that NetworkManager is newly sensitive to extra spaces in some circumstances. That is, since the time of the blog post, balenaOS has been updated, which includes an updated NetworkManager.

For clarity, you did all the right things.

Please let us know how your test goes.

eweimholt · April 15, 2021, 2:04pm

I added a cellular file with removed spaces, and then rebooted the device. Still no luck.

vipulgupta2048 · April 15, 2021, 3:02pm

Thanks for reaching back, can you tell us a bit more about how you re-added the cellular file with the suggested changes. Did you follow instructions on the docs to change network at runtime or did you reflash the device?

eweimholt · April 15, 2021, 3:09pm

I am on macOS, I created the cellular file through terminal and saved it to the system-connections/ folder on resin-boot, and then ejected the SD card and reflashed the device.

Looking at it from HostOS, here’s the file.

root@bff648e:/mnt/boot/system-connections# cat cellular 
[connection]
id=cellular  
type=gsm  
autoconnect=true

[gsm]
apn=super

[serial]
baud=115200

[ipv4]
method=auto

[ipv6]
addr-gen-mode=stable-privacy  
method=auto
root@bff648e:/mnt/boot/system-connections#

rosswesleyporter · April 19, 2021, 4:46am

Hi @eweimholt,

Thank you the update. It sounds like you repeated the blog post process, starting from “Click the Add Device button”. Which is indeed a good way to do this. In other words, I’m assuming that you reflashed your uSD card from a balenaOS download, then used your Mac to add the cellular file (with all extra spaces removed), and then provisioned your Pi by booting from the just-updated uSD card. Thus from the perspective of that instance of the OS, that was the first boot (which is relevant to the process). Did I correctly understand what you did?

If the above is correct, then the next step is for me to confer with some colleagues and get back to you. I remain confident that together we’ll get this to work for you, as we have many customers that are happily using cellular at scale.

Thank you for all of the clear information and tests so far!

dtischler · April 19, 2021, 5:28pm

Hi @eweimholt, let me walk you through the process I took, just to see if there are any discrepancies with your setup…

First and foremost, I am on a Mac as well, and from balenaCloud I download my balenaOS image file, and flash it with Etcher like usual. Upon completion of the flashing process, I take the SD Card out of my laptop, then, re-insert it right back in. My Mac automatically mounts the ‘resin-boot’ partition. I then navigate to the ‘system-connections’ folder, and open up the ‘resin-sample.ignore’ demo configuration file that is already in the folder, with TextEdit. (on my machine, I have to right-click and “Open With…”)

Now I have the file open for editing, and I simply delete everything in there, and then copy/paste in the code block from the blog post containing the Twilio cellular connection info. Next, I have to go line-by-line and delete any extra spaces at the end of each line. On my machine, there were about 5 or 6 lines that had extra spaces at the end, that I had to trim off.

Once all of them are cleaned out, I save the file and close TextEdit. Next, in the Finder, I right-click and rename the file, changing it from ‘resin-sample.ignore’ to ‘cellular’.

Once that is complete, I eject the SD Card, put it in the Pi, and it boots up… After a few moments it shows up in my balenaCloud dashboard, once the Twilio connection is established, the device checks in to balena, etc.

Hope that helps a bit!