On-screen keyboard for an Electron-based application

Product support balenaOS
1

Hi,

I’m experimenting with running WebThings Shell (a web app runtime for smart displays) on top of balenaOS.

I’ve built a prototype based on the balena-electron example and have managed to get the application to render full-screen on a touch screen attached to a Raspberry Pi, but I’m in need of an on-screen keyboard.

Is there an existing on-screen keyboard which is known to work with the xserver block, and if so how would I go about installing it?

Alternatively, since I note that the xserver block repo has been archived and the balenalib Docker image that the balena-electron example is built on has been deprecated, is there a more up-to-date approach for running an Electron-based application full screen on balenaOS?

Many thanks

Ben

2

Hi @benfrancis, the balena-electron repository you’ve linked is a bootstrap example project to get you up & running with connecting an electron app with the X11 Window Manager.

My intuition tells me there is a way of adding touchscreen keyboard support to your device, via the X11 Window Manager, and a suitable touchscreen interface driver, but this is not something we have a pre-made example for.

We do not offer a more up-to-date example of using an electron-based application on BalenaOS.

3

Just reporting back on this, we haven’t had any luck getting an on-screen keyboard to automatically appear and disappear on focus of text boxes in an Electron app.

So far we’ve tried matchbox-keyboard, KioskBoard, electron-virtual-keyboard, squeakboard, onboard, svkbd and xvkbd.

I note this is a solved problem on Ubuntu Core with ubuntu-frame and ubuntu-frame-osk, but in trying to replicate their approach with squeekboard it is very challenging to try and get Wayland/compositor/Electron to play nicely in containers.

The JavaScript-based keyboards require injecting code into every single text box which is too brittle and impractical for the webviews inside our application which can load arbitrary web pages.

This seems to be quite a tricky issue with Chromium and GTK in general, and getting it all to work inside containers makes it even more challenging.

It would be great if there was an example solution for on-screen keyboards because this currently rules out quite a lot of kiosk-like use cases for balenaOS.

See this issue for more details on what we have tried so far.