Qt for WebAssembly on mobile devices

Qt for WebAssembly on mobile devices, specifically phones, has lacked an essential feature - support for the native keyboard. It may or may not have worked. If it worked, it did not work very well. The tricky issue is opening the keyboard when needed and closing when it wasn't. There is no simple API for doing that on any platform we target - iOS, Android and Windows.

Although it feels a bit hacky to me to open the keyboard using javascript , we use Emscripten's C++ interface to create a hidden javascript text input element and set focus to that, which opens the platform keyboard.

Emscripten on Android had one other issue - the usual Emscripten input event API was not working. Nothing being typed on the native virtual keyboard was being handled like on other platforms. I found I could utilize the hidden input element that is used to pop up the keyboard, to listen for input characters and then send them to Qt. 

As it is, this patch adds support for native mobile keyboard for iOS, Android and Windows on Qt for WebAssembly. (now merged into dev as 66a76a5def46d0e4a330f7130ad440c639b87cf7), too late to make it into 6.3.

Other issues on mobile are memory (as usual). Up until recently, Safari limits browser memory to much less than the other browsers. 

Other areas that Qt covers for mobile devices that do not work on Qt for WebAssembly yet are sensors and bluetooth connectivity. Although there is a patch for some sensors support, it has not been merged and probably needs updating. It may not work at all, either.

Bluetooth connectivity for javascript is currently only supported on the Chrome browser and is currently experimental. While I could probably add this support to Qt WebAssembly, it would not get merged and the API might be too changey.

Any areas you find Qt WebAssembly is lacking on mobile, please report to https://bugreports.qt.io/

Qt WebAssembly clipboard

Clipboard use on the desktop platforms is ubiquitous. Most people use it without thinking. Copy, Paste and Cut keyboard strokes are in-grained into muscle memory. 

On the web it can present security issues as someone could read or write to your clipboard without you knowing.

Up until now, Qt for WebAssembly's clipboard was text only and only within the app itself. Qt 6.3 will have better clipboard support between host and app but also adds copy/pasting of images.

WebAssembly is a sandboxed platform like javascript. There are some extra security hurdles in doing some common things such as copy and paste of binary data such as images. One issue is clipboard use between the host platform and the browser sandbox. Allowing the web app to have access to the clipboard in which it could send arbitrary data without the user knowing could be dangerous for the user.

Browsers generally allow clipboard during user generated events such as when a user makes common key sequences such as [ctrl | command] c - the copy keys.

Qt itself has support for programmatically copying text and binary data such as images and works great on the desktop, but which presents issues for web browsers. There are work-arounds, like using a hidden javascript element and the javascript function execCommand to "copy". However, this function has been depreciated. 

By using the asynchronous Clipboard API and making use of javascript clipboard events where possible, we can bring image clipboard support to Qt WebAssembly. The Clipboard API requires a secure context (https) for full feature use. Among other things, the Clipboard API allows image and arbitrary data to be copied and pasted. Whereas before, only text mime types were supported. 

This API is of course implemented in the different browsers in different ways. On Firefox, read() and write() are only partially implemented and is hidden behind about:config settings. As well, copy/paste of arbitrary binary data does not seem to be supported and mostly silently fails.

Here are the ways browsers support Clipboard API:

Firefox 

• write() is available without permission in secure contexts and browser extensions, but only from user-initiated event callbacks.
• Clipboard API
• secure context (https or localhost)
• dom.events.asyncClipboard
• dom.events.asyncClipboard.clipboardItem
• dom.events.asyncClipboard.read
• dom.events.testing.asyncClipboard

Safari

• Clipboard API
• secure context    

Chrome

• Clipboard API
• secure context
• user permissions
• read() only supports
• text/html
• text/png

Also included in the now merged commit f0be152896471aa392bb1b2b649b66feb31480cc  is a clipboard manual test app that can be used on both desktop and webassembly to test clipbpoard use.

You can use the clipboard without a secure https context, but you won't get interaction between host and web app.