WiFi Connect - Community Maintainers

Product support
1

It has been proposed that WiFi Connect will benefit from additional community input for development and maintenance (see posts below from Balena staff). Here are a few thoughts after having been working with WiFi Connect for a while, and having gone through the issues on the GitHub repo. Hoping to get some input to evolve the ideas and then see who else may be able to contribute.

Note each number in the notes below relates to a numbered issue and associated GitHub Issue below.

1:

I think it’s safe to say that the issue where the list of available access points is empty is the biggest issue. There appears to have been quite a bit of investigation already (see some of the conversations in issues below), which point towards RequestScan as a potential solution. Would need someone with Rust background to explore this.

A related pull request for a refresh button also looks promising. It would require someone with Rust background to make a new endpoint available (/refresh for example) which would update the list. An issue has been raised that the AP cannot be running while this is done, which would subsequently kick the user off the device, and require reconnecting again to be able to see the refreshed list. This isn’t ideal, but would be a far better solution than telling someone to power cycle their device repeatedly until the list appears. It would be easy enough to add in a notification informing the user what is happening, and could also explore having the portal UI poll the access point and trigger a refresh of the page automatically once the user has reconnected to the device (shouldn’t be an issue as the web-UI is browser based).

A more complex alternative would either 1. exploring multiple channels so the user can stay connected while a second channel tries to make the connection, although my understanding is the amount of Wi-Fi chips on devices that can support his is limited (would also be wary about making the app any more complex than it already is, as it makes support and maintenance even harder); or 2, if someone can find a way to rescan while the access point is still running (Add a refresh networks button by sradevski · Pull Request #354 · balena-os/wifi-connect · GitHub).

A combination of exploring the RequestScan fix, along with a refresh button (that requires the user to reconnect to wi-fi afterwards) would be a big step forward.

2, 3 & 4:

Being able to forget the network currently connected to is a real pain point (At the moment, once you successfully connect to a network, you then get reconnected to that network every time the device boots). There are a bunch of script solutions posed, Python and Shell, but they can be temperamental and generate a lot of support questions. If the app allows you to connect, it seems fair to have a feature built in that allows you to disconnect again.

Unfortunately, it would require a bit of a rethink of how the app works. At the moment the app starts and creates an access point, then when the connection is established successfully the app exits. To be able to have a disconnect option, the app would have to stay running and listening for the request. I think this however, may well be a better workflow. The application, thanks to Rust, is very small and consumes very few resources, especially since someone did some work on the very clever Balena Blocks version that shares the firmware from the device (GitHub - balenablocks/wifi-connect: Easy WiFi setup for Linux devices from your mobile phone or laptop).

Building towards the Balena Blocks version being the default seems to make sense to me, it reduces the footprint and dependencies, and also resolves issue 2. which is a request for Alpine container support (at the moment it only runs on Debian image, which for Alpine users means having an extra image). Not having to deal with additional libraries (musl) should be advantageous for maintainace and support going forward. The app then running as a listening portal would allow the expansion of an API to add in some of the basic features, such as forgetting the wifi-connection (our apps could call the API via the other containers).

Would suggest then adding in a few endpoints:

/forget-connections

/start-access-point (as a POST, able to include the current config strings such as --SSID --PORT etc.)

/stop-access-point

/status

/shutdown

Hopefully self explanatory, but by having the option to stop and start the access point, we remove the need to have a large Debian container wrapper around the app. Without these endpoints, it is very hard to build more controls of the access point (i.e. enabling and disabling it through a settings panel), without having the Debian container and a custom listening port (I use a Flask instance) that listens just for stopping and starting the app. Which seems a little crazy considering that wifi-connect already has an API built into it listening for requests.

I would suggest also adding in an ‘—api’ flag for starting the app in the listening state. That way it is backwards compatible with how people were using it before, and those that want to leave it running and listening for commands can simply add the flag.

Being able to prevent the retry of connections when the wrong password is entered (number 4.) could also be resolved though an API, by checking the returned /status and using the /start and /stop endpoints as necessary. In addition to a —no-retry flag that would also help the workflow ( https://github.com/balena-os/wifi-connect/issues/283#issuecomment-485362893 ).

5:

Multi-language support I think would be better thought of as an easy way to configure the UI. This should be simple enough, we could have all the text strings used in the UI stored in a config file and users can then adjust that config file so the messages display as they wish (whether it be in a different language or different wording). The config file could also control some basic colour themes and logos. This isn’t as significant a change as incorporating multi-language support, but would still massively improve the ability to customise. I can submit a pull request for this once there is more of an idea of what is going on with the UI from the other pending pull request (Add a refresh networks button by sradevski · Pull Request #354 · balena-os/wifi-connect · GitHub).

6.

I think number 6 is probably already implemented, but the issue wasn’t closed?

7, 8 and 9

These have far fewer requests. I can see the value they add, but getting to them would all come down to how much time people are able to commit.

—

At one point I wondered if this was just going to become part of the Balena OS, but it often entails asking devices to do things they were not explicitly designed to do, so it is likely there will be bugs squashed and improvements made along the way. If it was integrated into the OS, it would require a significant update to devices, whereas keeping it as a small container like in the Balena Blocks example allows for more regular updates.

One of the biggest barriers from getting community support is the Rust language. A bunch of posts mention how they were going to submit pull requests but are not familiar enough with Rust. One user started a pull request, but couldn’t get around the Rust framework and ended up rebuilding the entire app in Python (wifi-connect python version · Issue #289 · balena-os/wifi-connect · GitHub). It would be a shame to not keep using Rust though, as it has so many benefits being self contained and using such a small footprint.

Hoping to hear ideas and thoughts.

Bugs

  1. Sometimes list of available wifi-networks are unavailable

https://github.com/balena-os/wifi-connect/issues/383

https://github.com/balena-os/wifi-connect/issues/327

https://github.com/balena-os/wifi-connect/issues/206

Related pull request

Feature requests

  1. Allow forgetting of connected network connection
  1. Support Alpine containers
  1. Prevent retry of correction when password is wrong
  1. Multi-language support
  1. Use only WPA2 for IOS compatibility (may already be implemented)
  1. Allow connections to hidden networks
  1. Support SSIDs with special characters
  1. Return signal strength (not a high priority, and not sure if NM returns this sort of info)

@dtischler @rbaynes

2 Likes
3

@majorz may be able to narrow a few of these down off the bat:

Do you know if this has been implemented already, or whether it’s still pending? Only WPA2 as security protocol for the portal AP · Issue #242 · balena-os/wifi-connect · GitHub

Did your investigation here still leave some hope that the RequestScan option as something we should persevere with or did something else come to light which means this needs a rethink? list of detected wifi access points is empty · Issue #206 · balena-os/wifi-connect · GitHub (also a more recent investigation here: Delay needed before wifi_device.get_access_points (on Raspberry Pi 3) · Issue #383 · balena-os/wifi-connect · GitHub

And of course any other thoughts.

5

Hi @maggie0002,

Thanks for making this detailed list with issues, that will be very helpful.

Good news is that I resumed development on WiFi Connect very recently and I am making good progress with it. The first step was to make it compatible with newer Rust versions. Currently I am working on making it work on top of libnm instead of using D-Bus directly. This will make it much more easier to develop and maintain further. Once we release a new version that uses libnm we can start looking at those known issues you are mentioning here.

After that once we start with the fixes we can indeed start with looking at RequestScan. With libnm and the newer async architecture we will introduce in WiFi Connect it should be much easier to implement this properly.

The rescan button is indeed a non-trivial problem because NetworkManager does not allow scanning while the AP is running. We need to explore the reasons of why this is not allowed and also compare this behavior with setup based on hostapd or iwd. As an alternative I have been thinking about acquiring the scan information from the Linux kernel directly. If iw dev wlan0 scan is able to return results while NetworkManager has put the interface in AP mode then we can find a solution for this I think.

Let me follow up with the rest of your items next week so that we continue arranging the list of issues you prepared.

Thanks,
Zahari

1 Like
8

This is great news, thanks for the update.

If a pull request containing the UI updates I mentioned would be of interest then let me know. I will wait to hear to avoid any unnecessary work or overlap.

15

@majorz Are you stilling working on WiFi Connect? I’m hoping to use it in an upcoming project.

Thanks,

16

Just to be clear, it is still working, the changes I mentioned were only suggested improvements and a few bug fixes.

I took a look at the Rust code and considered contributing, but it seemed like it was going to be difficult to get continued support. I ended up recreating it in Python instead: Alternative Wi-Fi Connect looking for testers

17

Hey there, curious whether there is still an intention to maintain and extend the original Wifi-Connect or the Python fork?

For the Mycroft Mark II we’re using the Rust version at the moment. At least for the majority of cases it generally works well, but as people are no doubt aware there are some limitations and bugs. A lot of these have been wonderfully summarized above.

So we’re about to start looking at our options going forward which broadly land in the buckets of:

  • Actively contribute to one of these projects
  • Look for an alternative project with the same goal
  • Explore different approaches for end-users to get devices connected to wifi
  • Write our own thing
    This is generally my least favourite option, would we call it YAWC - Yet Another WiFi Connect :roll_eyes: :laughing:

So thought I’d just drop a line and see if there was any newer information that would help us make a decision down any of those paths…

Thanks

18

Hi Gez,

I am not aware of any bugs on the current Python WiFi one, do let me know if there are. I realise though your question is more about long-term support intention, and you’re right, the answer to that question is still a little fuzzy.

Could you highlight which wifi-connect bugs in particular are creating issues? You mention they are summarised above, although there are quite a lot mentioned there. Which is it that is causing the bulk of the trouble?

1 Like
19

Yeah I haven’t tried the Python version at all, only discovered it yesterday.

In terms of other bugs, we’ve also seen more than the above. A few examples:

  • After entering an incorrect password, the AP sometimes does not get restarted. Again, it works most of the time, but occasionally just hangs and we haven’t delved into why.
  • Some users have reported that their SSID isn’t showing up in the list presented. Possibly this only occurs in locations with a lot of networks, and may be simply that only so many networks are presented.
  • It seems to be on specific hardware (but all Pi 4’s), the AP doesn’t get generated at all. Might be some Pi firmware issue, or EM interference based on that users environment, or something else entirely. Needs more investigation.

Then there are a range of UX issues that we’re looking at, like:

  • How long should we attempt to connect to an existing network before triggering WiFi Connect setup again? Eg is the router just rebooting after a power outage, or is the network actually gone.
  • Hidden SSID support
  • Better UX for passwordless networks - you can submit an empty password but that’s not obvious to users
  • Support to connect to networks with their own captive portals eg hotels.
  • Better status messages based on different behaviour - eg invalid credentials
  • Sort presented SSID’s by signal strength

Some of which are outside of this package but probably similar to the 2, 3, 4 group above - intimately related and can be solved by external scripts but it makes it more fragile. Though on the flip side, I think it’s always good to consider the scope of a project - it is after all called WiFi Connect, not “WiFi Connect and Management”…

20

This is all really helpful, thanks.

I think a lot of this would be out of the scope of wifi-connect, as you suggest because it is designed for basic connection of devices rather than management. If the missing SSID on first boot issue is a blocker, I would be interested to hear if this PR makes any difference though: Use IW to scan for networks before launch by maggie0002 · Pull Request #450 · balena-os/wifi-connect · GitHub

Considering the number of features you are looking for, and some of the particular UX considerations, it sounds like it may make sense to use something like the more exploratory Python version (GitHub - balena-labs-research/python-wifi-connect: An API for controlling Wi-Fi connections on balena devices.) which is an API that a frontend can be built on top off. That doesn’t mean we know for sure it will be long-term supported or whether there will be YAWC, but may be the best option at this point? I think/hope it is close enough to consider it a viable option for RPI4 already, it has been working fine for my use cases and we haven’t had any bugs reported. But I wouldn’t want to bank on what changes RPI5 might bring, or whether that would change the intention to continue supporting it or not.

I think there is a good chance that some of the issues you mentioned could be hardware related or networkmanager related, so overcoming them all could be difficult. For example, Python Wifi-Connect has to use a bit of hackery/trickery to get around the SSIDs not showing on first boot by bypassing networkmanger, and some device WiFi chips are not able to support scanning for devices while connected to a network, which can make for a less than convenient user experience and little we can do about it.

On the UX side, there are some components in the starter-interface (GitHub - balena-labs-research/starter-interface: A UI for balena devices.) that interact with Python-WiFi Connect that would be a good starting point, and potentially a way to try it out quickly. There is more available on the API than the UI enables right now, but it is a great starting point. It could be stripped back quite easily to be just the captive portal, and we would appreciate any contributions and testing, your insights on user experience are already really helpful for us to keep moving that project along.

1 Like