Wifi working but Ethernet isn't

bgold · October 15, 2018, 2:52pm

For some reason when I plug my resin into my companies network via ethernet, it doesn’t want to accept it and doesn’t even show up on any local networks on netstat for me to ping. However, if I run it on wifi, it shows up fine.

Does anyone know why this would be the case and what I can do to fix it?

Here are my logs from journalctl -fn 100 -u NetworkManager

Connecting to abc123...
Spawning shell...
=============================================================
    Welcome to ResinOS
=============================================================
root@f995498:~# journalctl -fn 100 -u NetworkManager
-- Logs begin at Mon 2018-10-15 14:19:10 UTC. --
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3053] dns-mgr[0x55a02509d120]: init: dns=default, rc-manager=resolvconf
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3057] rfkill1: found WiFi radio killswitch (at /sys/devices/pci0000:00/0000:00:1c.5/0000:3a:00.0/ieee80211/phy0/rfkill1)(driver iwlwifi)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3057] manager[0x55a025091040]: rfkill: WiFi hardware radio set enabled
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3057] manager[0x55a025091040]: rfkill: WWAN hardware radio set enabled
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3267] settings: Loaded settings plugin: NMSKeyfilePlugin (internal)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3276] keyfile: new connection /etc/NetworkManager/system-connections/resin-wifi-01 (a58e405d-fdc2-3649-9339-5bbaea740c76,"resin-wifi-01")
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3285] manager: rfkill: WiFi enabled by radio killswitch; enabled by state file
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3286] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3286] manager: Networking is enabled by state file
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3287] dhcp-init: Using DHCP client 'internal'
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3319] Loaded device plugin: NMWwanFactory (/usr/lib/NetworkManager/1.12.0/libnm-device-plugin-wwan.so)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3330] Loaded device plugin: NMWifiFactory (/usr/lib/NetworkManager/1.12.0/libnm-device-plugin-wifi.so)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3360] Loaded device plugin: NMBluezManager (/usr/lib/NetworkManager/1.12.0/libnm-device-plugin-bluetooth.so)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3365] Loaded device plugin: NMAtmManager (/usr/lib/NetworkManager/1.12.0/libnm-device-plugin-adsl.so)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3372] device (lo): carrier: link connected
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3374] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3380] manager: (eno1): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3388] keyfile: add connection in-memory (092abd07-02f5-30d1-a6ea-c0bb4c54ac3d,"Wired connection 1")
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3393] settings: (eno1): created default wired connection 'Wired connection 1'
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.3400] device (eno1): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5539] manager: (resin-dns): new Bridge device (/org/freedesktop/NetworkManager/Devices/3)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5605] keyfile: add connection in-memory (1c47377d-aca1-4d7d-8ba0-e22896c32e88,"resin-dns")
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5620] device (resin-dns): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5648] device (resin-dns): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5675] device (resin-dns): Activation: starting connection 'resin-dns' (1c47377d-aca1-4d7d-8ba0-e22896c32e88)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5686] manager: (sit0): new IPTunnel device (/org/freedesktop/NetworkManager/Devices/4)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5706] wifi-nl80211: (wlp58s0): using nl80211 for WiFi device control
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5708] device (wlp58s0): driver supports Access Point (AP) mode
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5718] manager: (wlp58s0): new 802.11 WiFi device (/org/freedesktop/NetworkManager/Devices/5)
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.5743] device (wlp58s0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7790] device (resin-dns): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7822] device (resin-dns): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7834] device (resin-dns): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7844] device (resin-dns): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7859] device (resin-dns): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7876] device (resin-dns): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7893] manager: NetworkManager state is now CONNECTED_LOCAL
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7940] device (resin-dns): Activation: successful, device activated.
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7956] bluez: use BlueZ version 5
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.7960] modem-manager: ModemManager available
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8053] bluez5: NAP: added interface 94:B8:6D:FD:EA:ED
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8093] supplicant: wpa_supplicant running
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8095] device (wlp58s0): supplicant interface state: init -> starting
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8318] sup-iface[0x55a02505c990,wlp58s0]: supports 5 scan SSIDs
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8330] device (wlp58s0): supplicant interface state: starting -> ready
Oct 15 14:19:12 f995498 NetworkManager[677]: <info>  [1539613152.8331] device (wlp58s0): state change: unavailable -> disconnected (reason 'supplicant-available', sys-iface-state: 'managed')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.4302] manager: (balena0): new Bridge device (/org/freedesktop/NetworkManager/Devices/6)
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5040] manager: (br-2a8f84ea7954): new Bridge device (/org/freedesktop/NetworkManager/Devices/7)
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5174] manager: (supervisor0): new Bridge device (/org/freedesktop/NetworkManager/Devices/8)
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5311] device (supervisor0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5329] keyfile: add connection in-memory (991ea01c-c636-4fdb-aa0e-9dda4237a828,"supervisor0")
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5332] device (supervisor0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5337] device (supervisor0): Activation: starting connection 'supervisor0' (991ea01c-c636-4fdb-aa0e-9dda4237a828)
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5338] device (supervisor0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5340] device (supervisor0): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5341] device (supervisor0): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5342] device (supervisor0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5345] device (supervisor0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5346] device (supervisor0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Oct 15 14:19:13 f995498 NetworkManager[677]: <info>  [1539613153.5364] device (supervisor0): Activation: successful, device activated.
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0457] policy: auto-activating connection 'resin-wifi-01' (a58e405d-fdc2-3649-9339-5bbaea740c76)
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0465] device (wlp58s0): Activation: starting connection 'resin-wifi-01' (a58e405d-fdc2-3649-9339-5bbaea740c76)
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0467] device (wlp58s0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0471] manager: NetworkManager state is now CONNECTING
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0476] device (wlp58s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0479] device (wlp58s0): Activation: (wifi) access point 'resin-wifi-01' has security, but secrets are required.
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0479] device (wlp58s0): state change: config -> need-auth (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0506] device (wlp58s0): state change: need-auth -> prepare (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0511] device (wlp58s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0514] device (wlp58s0): Activation: (wifi) connection 'resin-wifi-01' has security, and secrets exist.  No new secrets needed.
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0514] Config: added 'ssid' value 'WIFI NAME HERE'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0515] Config: added 'scan_ssid' value '1'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0515] Config: added 'bgscan' value 'simple:30:-80:86400'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0515] Config: added 'key_mgmt' value 'WPA-PSK'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0515] Config: added 'auth_alg' value 'OPEN'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0515] Config: added 'psk' value '<hidden>'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0798] device (wlp58s0): supplicant interface state: ready -> authenticating
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.0901] device (wlp58s0): supplicant interface state: authenticating -> 4-way handshake
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1028] device (wlp58s0): supplicant interface state: 4-way handshake -> completed
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1028] device (wlp58s0): Activation: (wifi) Stage 2 of 5 (Device Configure) successful.  Connected to wireless network 'WIFI NAME HERE'.
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1030] device (wlp58s0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1035] dhcp4 (wlp58s0): activation: beginning transaction (no timeout)
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1380] dhcp4 (wlp58s0):   address 192.168.3.164
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1380] dhcp4 (wlp58s0):   plen 24
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1380] dhcp4 (wlp58s0):   expires in 43200 seconds
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1381] dhcp4 (wlp58s0):   nameserver '192.168.3.1'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1381] dhcp4 (wlp58s0):   hostname 'f995498'
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1381] dhcp4 (wlp58s0):   gateway 192.168.3.1
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1385] dhcp4 (wlp58s0): state changed unknown -> bound
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1415] device (wlp58s0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1432] device (wlp58s0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1439] device (wlp58s0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1447] manager: NetworkManager state is now CONNECTED_LOCAL
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1527] manager: NetworkManager state is now CONNECTED_SITE
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1531] policy: set 'resin-wifi-01' (wlp58s0) as default for IPv4 routing and DNS
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1537] dns-mgr: Writing DNS information to /sbin/resolvconf
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1725] device (wlp58s0): Activation: successful, device activated.
Oct 15 14:19:16 f995498 NetworkManager[677]: <info>  [1539613156.1743] manager: NetworkManager state is now CONNECTED_GLOBAL
Oct 15 14:19:18 f995498 NetworkManager[677]: <info>  [1539613158.5509] manager: startup complete
Oct 15 14:19:24 f995498 NetworkManager[677]: <info>  [1539613164.5212] manager: (resin-vpn): new Tun device (/org/freedesktop/NetworkManager/Devices/9)

majorz · October 15, 2018, 3:13pm

The info logs from NetworkManager do not show what potentially the problem is. Can you please run the following script on your device and send me back the more verbose logs. You send them to by a private message:

bgold · October 15, 2018, 3:36pm

@majorz it seems like that script killed my device. When I ran the install, the moment it finished, it went offline and now when I restart the device via turning it off and on it doesn’t come online anymore.

majorz · October 15, 2018, 3:41pm

Can you please provision a device without adding any application to it – that is by creating a new application, but without pushing any code to it? You may setup WiFi credentials through the dashboard when you download the image. Then install the script and see what will happen. That will minimize the noise a bit.

Also what type of device are you using? And are you using some Ethernet dongle? Also please use the latest available OS version in production.

bgold · October 15, 2018, 4:14pm

@majorz So the application I installed this on has been forcing an update which may be the issue. I tried when the application was partially downloaded, refreshed and tried again with the same result. Kills the NUC.

I’m using the latest version, fresh download, Intel NUC, built in wifi, and just the ethernet port in the back.

The ship server I’m installing it on is going out to sea so I will have to try on a new application in a few days when it comes back and will private message you if I have any issues but at the moment I wasn’t able to get the logs because it crashed it.

Here is the issue I posted for the crash fyi

majorz · October 16, 2018, 9:24am

Did you install the script in a container, or in the host OS? It should be the host OS. I have not tried running it on NUC, so maybe it is related to the differences with resin OS for this device type.

bgold · October 16, 2018, 1:21pm

@majorz I installed it on the host OS. The other containers were not finished downloading.

Is there any other things I can try to get you the logs you need besides trying to install it on a blank application?
Should I try to use 2.12.7+rev2?

I know ethernet works on v2.13.6+rev1 but that was just plugged into the router. Would love to do anything I can to figure out why the network I’m plugging it into won’t work so we can get to the bottom of it.

majorz · October 16, 2018, 3:00pm

@bgold While connected through WiFi you may execute from a host OS terminal:

dbus-send --system --print-reply --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.SetLogging string:"debug" string:""

This will elevate the logging levels of NetworkManager.

Then plugin the Ethernet cable and let’s see what the NetworkManager logs will reveal.

bgold · October 16, 2018, 7:38pm

@majorz Because there isn’t any wifi available in the server room I will be installing these servers, this will be the steps I will take. Let me know if this will provide you what you need.

Plug in Intel NUC and connect with wifi.
Run Command
Shut down Intel NUC Device (because I won’t be in an area near the ethernet slot)
Turn it on with Ethernet, and wait for it to collect logs
Turn off machine and turn it back on with Wifi so I can read the logs

So I guess the question I am asking is - will the logs be persistent when I turn off the device?

majorz · October 16, 2018, 10:01pm

This is not so easy without the script.

For this to work you need to enable persistent logging for the device. This can be done by setting a "persistentLogging": true value in the config.json that resides on the boot partition.

The additionally the above dbus command will not work, since it has an effect only for the current session - rebooting the device will return the logging level to info. This can be done through the NetworkManager global config though.

The biggest issue is that the log entries could get lost since the number of entries when doing the normal persistent logging is limited by the size of the state partition and NetworkManager is really verbose and fills this up quickly.

I will have an Intel NUC here tomorrow which I may use to see what makes the script fails, and better to check this and fix it, since it will make the diagnostic process much easier. I will ping you tomorrow when I look into it.

majorz · October 24, 2018, 7:56am

@bgold I am updating the tool now and will ping you back once ready, so that you may give it another try. That should be in the following days.

majorz · January 2, 2019, 12:06pm

Hi @bgold, I was unable to complete the diagnostics tool I have been working on, but just to ask whether you were able to resolve the connectivity issue you faced on the Intel NUC device?