Keyboard does not open when accessing a VM console on Android (Chrome & Firefox)

[JeffWDH]

cockpit: 311.2
cockpit-machines: 308.3
Phone: Pixel 8 Pro
OS: Android 14

When accessing Cockpit from an Android device I am unable to interact with VMs as the keyboard does not open when tapping on the console session. This is the case for both the small and expanded consoles. Tested with both Firefox and Chrome.

[garrett]

This would be tricky, if not impossible to fully implement.

The onscreen virtual keyboard is active when an input has focus. However, the VNC widget is not a text area or a contenteditable and it wants mouse focus, not just keyboard focus. We could add a button for popping up the keyboard in some circumstances, but there isn't an API to do so.

There is an experimental virtual keyboard API we might be able to use, but it's only supported in Chrome (and some other Chromium derived), not Firefox or Safari:

• https://w3c.github.io/virtual-keyboard/#dom-navigator-virtualkeyboard
• https://caniuse.com/?search=virtualkeyboard
• https://developer.mozilla.org/en-US/docs/Web/API/VirtualKeyboard_API

Specifically, navigator.virtualKeyboard.show(), which can be checked by typeof navigator.virtualKeyboard?.show === 'function'. We'd probably want to only show it on devices that need a virtual keyboard and a check like this.

[JeffWDH]

Presumably at some point the virtual keyboard API will become a fully fleshed w3c standard and all major browsers will eventually pick it up. For now, Chrome/Chrome-derived support would be a great help.

Coming from VMware (which has a native Android app), having the ability to interact with a VM console from a mobile device is sorely missed.

[garrett]

I agree with you, it would be nice, and I hope browsers all add it soon. It's a shame it's a Chromium-only thing for now, as it's sorely needed on all browsers on mobile devices (both in phone and tablet form), and would even be useful on desktops/laptops with touchscreens that have on-screen keyboards too.

The WebKit and Mozilla organizations both have standards tracking for this API:

• WebKit: VirtualKeyboard API WebKit/standards-positions#16
• Mozilla: Virtual Keyboard Control mozilla/standards-positions#531

We do have designs for changing the console part of Cockpit Machines. When someone eventually picks up the implementation, then it would be nice to include support for this API, even if it's Chromium-only for now. Hopefully the other browser engines will eventually pick it up and it'll "just work" to spec.