Wednesday, January 8, 2020

Qt WebAssembly faster builds

Admittedly, building Qt WebAssembly apps takes what seems like forever. Technically, it is the linking part that takes a huge amount of time, as that is where the magic happens.

You may have missed my first blog regarding faster build times for Qt WebAssembly apps:
http://qtandeverything.blogspot.com/2019/06/faster-link-time-for-qt-webassembly.html

Here is how to get set up to use this now integrated feature.

1. You need emscripten with upstream clang, which has support for transpiling directly to wasm, instead of taking the intermediary step of building javascript, then outputting wasm binary with that. Starting with version 1.39.0, upstream clang is the default. You can use "latest" to get 1.39.x or something like "sdk-upstream-1.38.43-64bit" to get an earlier version.

./emsdk install latest
./emsdk activate --embedded latest
source ./emsdk_env.sh

You can then configure and rebuild Qt with  -device-option WASM_OBJECT_FILES=1

configure -xplatform wasm-emscripten -developer-build -nomake tests -nomake examples -opensource -confirm-license -verbose -compile-examples -no-warnings-are-errors -release -device-option WASM_OBJECT_FILES=1

Then you can build your Qt app as normal using the qmake from that build, and watch the build time speed up!

*NOTE* Using  WASM_OBJECT_FILES with a multi-threaded build is still experimental, as both threads and using wasm object files in emscriptem/wasm and Qt are experimental. I have not seen any real issues regarding the use of both of these at the same time.

You can read more about Qt for WebAssembly, mobile and embedded development in the book Hands-On Mobile and Embedded Development with Qt 5

Tuesday, January 7, 2020

Aussie firestorms

As some may know, I currently reside in Australia, which is undergoing an unprecedented bush fire season. Total burnt area is about the size of Demark, with no end in sight. Whole forests and towns are being incinerated. Millions of animals, both wild and farm are being killed and injured by the swiftly moving crown fire being whipped up by high dry winds.

There are people that need help, and people that rescue animal need help as well.
https://www.news.com.au/technology/environment/bushfire-relief-how-you-can-help-those-in-need/news-story/a0476ac3538b8c373f281ea6be204421

Some of you may have seen that I authored a book that was published by Packt this last year
Hands on Mobile and Embedded Development with Qt 5

I have decided the donate all my last quarters royalties to the WIRES Emergency Fund for Wildlife. It's not much, but I wanted to help out creatures that cannot help themselves or cannot seek out help. If there is any interest, I will donate the next quarters royalties as well.


Wednesday, August 28, 2019

AsteroidOS - the open source watch

So I get a message regarding changes needed to Sensor Framework and QtSensors to add heart rate sensor support for AsteroidOS. AsteroidOS is an open source operating system for watches. It uses OpenEmbedded and Yocto to build the system image and kernel that you can flash to a small group of Android watches. The UI is based on Qt and Qml.

I thought about it for approximately 30 seconds and I bought a used LG Urbane watch. Which is listed on the Install documentation for the OS, as one of the watches that can run AsteroidOS. I choose LG, as I am partial to their Smart TV's, which happen to run Qt and QML.

First issue I ran into was that adb did not see the watch on WearOS, even in developer mode, but once I changed the usb cable to a good, solid cable (actually is an old Nokia cable that still works like new!), I was then able to flash the pre-built AsteroidOS 'nightly' build image on it. You might want to backup the image on the watch before you do, unlike what I did. First flash resulted in a boot loop, so I flashed the image again, and this one booted!


In order to test new sensors code, I need to build the thing. Essentially, it was simply to git clone the repo, run a script, and build the image. Too easy! Luckily I have a fast development machine and it did not take too long to complete.
Again I had to flash it two times. As long as I can get into the fastboot screen, everything is ok.

WearOS has a wrist gesture sensor, which in WearOS, turns on the display when the user flicks their wrist. iOS does this as well. AsteroidOS does not yet have support for utilizing this, so that is one of the first things I would like to fix. Neither QtSensors or Sensor Framework has support for this sensor yet. Since it is open source and I am familiar with sensor API's that run on it, I can fix this on my own!

You can read more about Qt development for Mobile and Embedded devices in the book Hands-On Mobile and Embedded Development with Qt 5
















































You can learn more about Yocto and embedded programming with Qt in my book Hands-on Mobile and Embedded Programming with Qt 5

Thursday, August 15, 2019

QMqtt & QWebAssembly

Qt for WebAssembly is an exciting new platform for Qt. It means that Qt applications can run in a web browser. This means that deploying a Qt application is as easy as copying the files to your properly configured web server and handing out the url, for your users to run in their web browser.

Mqtt is a machine to machine communication protocol, which is used for messaging in IoT. Websocket is a two way communication protocol between a web browser and a web server.

QtMqtt has been ported to run on the WebAssembly platform, using QtWebSockets as it's transport. There is a catch to getting it working, however. That catch is that you need to use the WebSocketIODevice class that is in the QtMqtt websocketsubscription example app. WebSocketIODevice depends on the QtWebSocket module.

You need to copy websocketiodevice.h and websocketiodevice.cpp into your project, using the WebSocketIODevice as QMqttClient's transport.

In a nutshell, at a minimum,  code such as this:

QMqttClient m_client
WebSocketIODevice m_device;
m_device.setUrl(m_url);
connect(&m_device, &WebSocketIODevice::socketConnected, this, [this]() {
    m_client.setTransport(&m_device, QMqttClient::IODevice);
}

Then you can use the QMqttClient as normal.

You can read more about Qt for WebAssembly, QtMqtt and QtWebSockets in the book Hands-On Mobile and Embedded Development with Qt 5

Tuesday, July 2, 2019

[Update] Faster link time for Qt WebAssembly

The guys working on Emscripten have begun integrating the new llvm wasm backend into mainstream emscripten. Although it is still not the default, it is now heaps easier to install and use, as you no longer need to build llvm yourself.

https://v8.dev/blog/emscripten-llvm-wasm

The jist of it is:

emsdk install latest-upstream
emsdk activate latest-upstream

According to the linked blog, there are code size benefits as well as link time speed-ups.

The faster link time mostly affects application builds.

Of course, you will need to recompile Qt for WebAssembly to use this, and need to configure it with the Qt WebAssembly specific option:

-device-option WASM_OBJECT_FILES=1

After that, you just need to run qmake as normal.

One note: You will need to remove the line
 -s \"BINARYEN_TRAP_MODE=\'clamp\'\"

from the mkspecs/wasm-emscripten/qmake.conf, as upstream llvm webassembly backend does it's own overflow clamping and does not support BINARYEN_TRAP_MODE argument.

Friday, June 7, 2019

Faster link time for Qt WebAssembly

I have built Qt and various apps using emscripten so many times over the last couple of years, it isn't even funny.

One detractor with building Qt applications for the web using Qt for WebAssembly, is the time it takes to build the client application. Especially during linking, it takes a huge amount of time to produce a binary for Qt WebAssembly.
This is caused by all the magic that the linker in emscripten is doing.

Luckily, llvm can now produce wasm binaries directly so emscripten does not have to go through any additional steps to output wasm. Emscripten and Qt can utilize this feature to reduce the length of time it takes to link and emit a .wasm binary.

YAY! \0/

I did some very non scientific measurements on build time. Using my development machine with make -j8:

Qt (not including configure)textedit
default configure Qt for WebAssemblyuser 20m3.706suser 2m46.678s
using -device-option WASM_OBJECT_FILES=1user 23m30.230suser 0m29.435s

The result is that building Qt takes a tad longer, but building the client application is significantly faster. Which means development iterations will be much faster. In this instance, I used the textedit example in Qt. The build time went from 2 minutes and 46 seconds for a default non object files build, to 29 seconds! Zip! Zoom! Zang! Dang! that's fast (for wasm)

Unfortunately, this is not currently done by default by either Qt, emscripten or llvm. There is a way to do this which involves building llvm from upstream and telling emscripten to use that.

For the emscripten and binaryen version, I tested the current 1.38.32

From this bug report https://bugreports.qt.io/browse/QTBUG-72537
Morten has fleshed out a method for doing this.

------------------------------------------------------------

Getting started:

(Note: this is "how-I-did-it", not necessarily "how-it-should-be-done".)
1. Clone the following repositories:

2. Check out the version you want to use 
  • emscripten and binaryen have matching emsdk version tags, for example "1.38.23".
  • llvm has its own version numbers, and must be synced up manually. Currently:
    • emscripten <= 1.38.23 : llvm 8.0 (8.0.0-rc2 branch)
    • emscripten > 1.38.23 : llvm 9.0 (master branch)
      (em++/emcc will complain if you have the incorrect llvm version)

3. Build
  • emscripten is all python, no build needed.
  • binaryen: "cmake -GNinja && ninja"
  • llvm:
    cmake -GNinja
    ../path/to/llvmcheckout
    -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi;lld"
    -DCMAKE_BUILD_TYPE=Release
    -DLLVM_TARGETS_TO_BUILD=WebAssembly
    -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly
    
    (one of the WebAssembly targets may be redundant)

4. Configure Environment
I use the following:
export EMSDK="/Users/msorvig/dev/emsdks/emscripten-1.38.23"
export PATH="$EMSDK:$PATH"
export LLVM="/Users/msorvig/dev/emsdks/llvm-8.0.0-build/bin"
export BINARYEN="/Users/msorvig/dev/emsdks/binaryen-1.38.23"
export EM_CONFIG="/Users/msorvig/dev/emsdks/.emscripten-vanillallvm-1.38.23"
export EM_CACHE="/Users/msorvig/dev/emsdks/.emscripten-vanillallvm-cache-1.38.23"
Where .emscripten-vanillallvm-1.38.23 is a copy of the .emscripten config file that emsdk generates.

------------------------------------------------------------

You will need to adjust the various paths to your system, of course. 
Then to configure and build Qt, add -device-option WASM_OBJECT_FILES=1 to the normal Qt for WebAssembly configure line.

The one I normally use is:

configure -xplatform wasm-emscripten -developer-build -nomake tests -nomake examples -opensource -confirm-license -verbose -compile-examples -no-warnings-are-errors -release  -device-option WASM_OBJECT_FILES=1


Works like a charm for the Qt 5.13 and the 5.13.0 branches of the Qt source code repo. I tried this with the 5.13.0 beta4 WebAssembly binaries, but got:

wasm-ld: warning: function signature mismatch:
so a complete rebuild is required.

There is a chapter regarding Qt for WebAssembly in the book, Hands-On Mobile and Embedded Development with Qt 5


Wednesday, May 22, 2019

exec on Qt WebAssembly

When porting applications to Qt for WebAssembly, there are a few issues to keep in mind.

One big issue is that you do not have a full operating system underneath for you to utilize.
What you get is the same sandbox in which javascript lives. In some aspects, it is somewhat similar to a single-threaded microcontroller where there is just one thread in which to operate, limited filesystem access and limited resources.

One of the unusual and tricky bits about using Qt for WebAssembly, is that the exec loop is not typical. The Emscripten main loop is co-operative, after each event has a turn to run, control is then returned to the browser. Any blocking of the event loop will mean the web page in which your application runs will become unresponsive.

This is all fine and dandy for simple applications that use one main loop, but gets complicated when you try to exec a secondary loop. To stop the execution of an event loop, emscripten throws an exception, which leads to all kinds of fun. It also means it never returns to the same place that you expect it. So any modal dialog that uses exec() will not return values. Less than ideal.

Take for instance QColorDialog. Typical use is as such:


    QColorDialog dlg(parent);
    dlg.exec();
    QColor color = dlg.selectedColor();






Which is basically what QColorDialog::getColor does.

... and does not work with Qt WebAssembly, because exec() does not return to the same place as you expect! The call to selectedColor will never get called.

What you can do is use a non-modal dialog with the show() or in the case of QColorDialog open() function and use Qt's signal/slot API to retrieve the selected QColor.

 QColorDialog *dlg = new QColorDialog(this);

    connect(
        dlg, &QColorDialog::colorSelected,
        [=](const QColor &selectedColor) {
        qDebug() << Q_FUNC_INFO << selectedColor;
    });
    dlg->open();

You can read more about the Emscripten execution environment at
https://emscripten.org/docs/api_reference/emscripten.h.html#browser-execution-environment

You can also learn more about Qt for WebAssembly and other things in the book I wrote:
Hands on Mobile and Embedded Development with Qt 5