orange pi zero doesnt recover from wifi outaage

pi zero, 2, 3 and 4 in same cluster all recover automatically from wifi outage
but orange pi zero remains offline

can this be fixed by the balena image wifi stack ?

do you need a github bug ticket ? if so you can have full access to logs

may be related to comment

Awesome @shaunmulligan! It’s great to know someone is working on it. I don’t have any yocto knowledge but it sounds like I will be gaining some soon. Would be good to dust off the 2E. Looking forward to having it supported in the future. Regards, Michal

Hello what do you mean when you say same cluster. Do you mean you have one mixed device type application or something else?

1 Like

by cluster i mean a cluster of balena apps : one app has a zero, 2 & orange the other app has a 3 and a 4
both apps are connected to the same wifi access point and bizarrely the orange pi zero has come back on online today !

maybe the orange pi zero wifi driver only resyncs at midnight ?

Hi,

It should be something else since I don’t think the orange pi zero wifi only resyncs at midnight, can you please send us all the logs you can collect so we can help investigate the issue?

1 Like
14.04.20 19:34:16 (+0100) Service is already running 'main sha256:d30142851efc1dd8c622890b9abac976fd6e7ce517677e9f24b476de0192d206'
14.04.20 20:41:22 (+0100) Service is already running 'main sha256:d30142851efc1dd8c622890b9abac976fd6e7ce517677e9f24b476de0192d206'
15.04.20 02:36:43 (+0100) Service is already running 'main sha256:d30142851efc1dd8c622890b9abac976fd6e7ce517677e9f24b476de0192d206'
16.04.20 01:07:29 (+0100) Service is already running 'main sha256:d30142851efc1dd8c622890b9abac976fd6e7ce517677e9f24b476de0192d206'

dmesg | head

[32123.776980] xradio_wlan mmc1:0001:1: missed interrupt
[32128.047072] xradio_wlan mmc1:0001:1: missed interrupt
[32133.967108] xradio_wlan mmc1:0001:1: missed interrupt
[32134.007107] xradio_wlan mmc1:0001:1: missed interrupt
[32144.227256] xradio_wlan mmc1:0001:1: missed interrupt
[32144.267225] xradio_wlan mmc1:0001:1: missed interrupt
[32144.387200] xradio_wlan mmc1:0001:1: missed interrupt
[32144.427259] xradio_wlan mmc1:0001:1: missed interrupt
[32147.847266] xradio_wlan mmc1:0001:1: missed interrupt
[32154.927344] xradio_wlan mmc1:0001:1: missed interrupt
root@e19fbba:~#

# dmesg | tail
[33928.796542] xradio_wlan mmc1:0001:1: missed interrupt
[33928.836529] xradio_wlan mmc1:0001:1: missed interrupt
[33928.876538] xradio_wlan mmc1:0001:1: missed interrupt
[33928.956533] xradio_wlan mmc1:0001:1: missed interrupt
[33929.076531] xradio_wlan mmc1:0001:1: missed interrupt
[33929.166559] xradio_wlan mmc1:0001:1: missed interrupt
[33929.506541] xradio_wlan mmc1:0001:1: missed interrupt
[33929.546549] xradio_wlan mmc1:0001:1: missed interrupt
[33929.666538] xradio_wlan mmc1:0001:1: missed interrupt
[33929.706573] xradio_wlan mmc1:0001:1: missed interrupt
root@e19fbba:~# 

but let me know if youd like to login

# cat /etc/os-release 
ID="balena-os"
NAME="balenaOS"
VERSION="2.31.5+rev1"
VERSION_ID="2.31.5"
PRETTY_NAME="balenaOS 2.31.5+rev1"
MACHINE="orange-pi-zero"
VARIANT="Production"
VARIANT_ID="prod"
RESIN_BOARD_REV="18891db"
META_RESIN_REV="64859b0"
SLUG="orange-pi-zero"

Hey there! I found an interesting post here: https://forum.armbian.com/topic/3350-orange-pi-zero-ap-and-mainline-kernel-any-one-has-it-working/ that describes a very similar issue, also on the Orange Pi Zero. According to the post, the issue was the wpa_pairwise=CCMP NetworkManager option.

Do you have such option set? Can you paste your NetworkManager profile here?

i wonder where those files are on balena ?

root@e19fbba:~# cat /etc/hostapd/hostapd.conf
cat: /etc/hostapd/hostapd.conf: No such file or directory
root@e19fbba:~# cat /etc/dnsmasq.d/wlan0.conf
cat: /etc/dnsmasq.d/wlan0.conf: No such file or directory
root@e19fbba:~# cat /etc/network/interfaces
cat: /etc/network/interfaces: No such file or directory
root@e19fbba:~# more /var/log/syslog
more: stat of /var/log/syslog failed: No such file or directory

# cat !$
cat etc/wpa_supplicant.conf
ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=0
update_config=1

network={
        key_mgmt=NONE
}

Hi there, wpa_pairwise is a hostapd option, which is a Linux component that runs an access point as is not part of balenaOS by default. From reading this thread, you are having issues with establishing a Wi-Fi client connection to an external AP. balenaOS uses NM to manger network connections and we have a lot of examples how to customise its configuration here.

The message you are seeing in the logs has probably more specific to the kernel version/wifi module and possibly to do with a wider hardware issues, such as elevated device temperature and high CPU utilisation.

Have you checked dmesg output for other hints as well as run the device diagnostics from the dashboard?

1 Like

thanks i will take a look but my issue is that the orange connects fine but ONLY reconnects at midnight after wifi outage

We would need to see the logs from the device at the time the reconnect happens to help troubleshoot further. Are you able to provide these please? /var/log/messages on the host is likely where we will see something…

1 Like

sure you can login to the orange if you like but heres what i can see

but last wifi outage was yesterday so let me force another wifi outage tonight and send you new logs from tomorrow morning

Apr 16 16:26:43 localhost user.info kernel: [    1.326018] ehci-platform 1c1b000.usb: new USB bus registered, assigned bus number 3
Apr 16 16:26:43 localhost user.info kernel: [    1.334195] ehci-platform 1c1b000.usb: irq 29, io mem 0x01c1b000
Apr 16 16:26:43 localhost user.info kernel: [    1.367485] ehci-platform 1c1b000.usb: USB 2.0 started, EHCI 1.00
Apr 16 16:26:43 localhost user.info kernel: [    1.374319] hub 3-0:1.0: USB hub found
Apr 16 16:26:43 localhost user.info kernel: [    1.378154] hub 3-0:1.0: 1 port detected
Apr 16 16:26:43 localhost user.info kernel: [    1.382760] ohci-platform 1c1b400.usb: Generic Platform OHCI controller
Apr 16 16:26:43 localhost user.info kernel: [    1.389407] ohci-platform 1c1b400.usb: new USB bus registered, assigned bus number 4
Apr 16 16:26:43 localhost user.info kernel: [    1.397375] ohci-platform 1c1b400.usb: irq 30, io mem 0x01c1b400
Apr 16 16:26:43 localhost user.info kernel: [    1.472129] hub 4-0:1.0: USB hub found
Apr 16 16:26:43 localhost user.info kernel: [    1.475912] hub 4-0:1.0: 1 port detected
Apr 16 16:26:43 localhost user.warn kernel: [    1.480545] usb_phy_generic usb_phy_generic.0.auto: usb_phy_generic.0.auto supply vcc not found, using dummy regul
root@e19fbba:~# tail /var/log/messages
Apr 16 16:58:21 localhost user.warn kernel: [ 1884.757013] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:21 localhost user.warn kernel: [ 1884.796985] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:21 localhost user.warn kernel: [ 1884.837007] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:21 localhost user.warn kernel: [ 1884.877003] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:22 localhost user.warn kernel: [ 1885.036979] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:22 localhost user.warn kernel: [ 1885.126983] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:22 localhost user.warn kernel: [ 1885.726980] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:22 localhost user.warn kernel: [ 1885.767025] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:22 localhost user.warn kernel: [ 1885.847056] xradio_wlan mmc1:0001:1: missed interrupt
Apr 16 16:58:23 localhost user.warn kernel: [ 1886.046990] xradio_wlan mmc1:0001:1: missed interrupt

Sounds good . Please also grant support access to the device and share the device guid with us to we can take a closer look.

ok ive enabled support access for a week how do i DM you the guid ?

You can DM it to me (ab77) and I’ll record it privately in this ticket.

ok done you can connect anytime but i will cause a wifi outage in a couple of hours and if my theory is correct it will be down until midnight

sorry got bored just forced an outage at 6:35 UTC all other PIs and INUCs are back online apart from orange pi

so if you do get in (after midnight ?) look at logs from 6:35

No worries, just in case there is nothing evident in the logs, another suggestion would be to try and disable the on-board WLAN on the OPi, since there are reports of it being unstable and instead use an external USB adapter. Failing that, another workaround may be to wire up a watchdog type script to run periodically on the device, checking for network connectivity and manually reconnecting if detected offline.

1 Like

all good ideas but we’re talking about the IOT here and maybe i want to deploy a few 1000 of these into the wild i can just move on to another hw board vendor rather than mess around with having to buy EXTRA hw or rolling my own (unsupported) sw workarounds

so ideally it would be great if the balena drivers auto reconnected wifi for me and maybe we will discover a flaw in the drivers that can improve wifi for all balena targetted devices

FYI latest update its 7:25 and its still offline my midnight theory could be right

TBH if it went offline and stayed offline i would probably chuck it in the bin and buy banana pis instead its just strange that it CAN come back online