Wednesday, May 22, 2019

exec on Qt WebAssembly

When porting applications to Qt for WebAssembly, there are a few issues to keep in mind.

One big issue is that you do not have a full operating system underneath for you to utilize.
What you get is the same sandbox in which javascript lives. In some aspects, it is somewhat similar to a single-threaded microcontroller where there is just one thread in which to operate, limited filesystem access and limited resources.

One of the unusual and tricky bits about using Qt for WebAssembly, is that the exec loop is not typical. The Emscripten main loop is co-operative, after each event has a turn to run, control is then returned to the browser. Any blocking of the event loop will mean the web page in which your application runs will become unresponsive.

This is all fine and dandy for simple applications that use one main loop, but gets complicated when you try to exec a secondary loop. To stop the execution of an event loop, emscripten throws an exception, which leads to all kinds of fun. It also means it never returns to the same place that you expect it. So any modal dialog that uses exec() will not return values. Less than ideal.

Take for instance QColorDialog. Typical use is as such:


    QColorDialog dlg(parent);
    dlg.exec();
    QColor color = dlg.selectedColor();






Which is basically what QColorDialog::getColor does.

... and does not work with Qt WebAssembly, because exec() does not return to the same place as you expect! The call to selectedColor will never get called.

What you can do is use a non-modal dialog with the show() or in the case of QColorDialog open() function and use Qt's signal/slot API to retrieve the selected QColor.

 QColorDialog *dlg = new QColorDialog(this);

    connect(
        dlg, &QColorDialog::colorSelected,
        [=](const QColor &selectedColor) {
        qDebug() << Q_FUNC_INFO << selectedColor;
    });
    dlg->open();

You can read more about the Emscripten execution environment at
https://emscripten.org/docs/api_reference/emscripten.h.html#browser-execution-environment

You can also learn more about Qt for WebAssembly and other things in the book I wrote:
Hands on Mobile and Embedded Development with Qt 5

Friday, May 3, 2019

Mobile and Embedded Development with Qt

At times it was a bit painful juggling writing a book, doing my day job and running around doing the things that life throws. It's done and dusted now, you too can buy my book titled Hands-On Mobile and Embedded Development with Qt 5! It has a nice image of glacier ice on the cover, which I thought was appropriate for a technology founded in Norway and then continued in Finland.



https://www.packtpub.com/application-development/hands-mobile-and-embedded-development-qt-5

A big thanks to the co-founder of Trolltech, Eirik Chambe-Eng, who was gracious enough to write the forward at the last second. Tons of thanks to all the editors who also worked on this book.

One of the things I learned writing this book is that Qt is big. I already knew that, but now it's plainly apparent just how big it has grown. Not only are there major companies developing products with Qt, but it has a lot of different functionality and is not just about desktop widgets. There are a huge number of classes to cover.

You can check out the table of contents if you want to see what is covered and included. One area that I did not include is OpenGL ES. This is a huge topic and easily a book on it's own. It's something I would like to know more about, which is why I did not feel qualified to cover it. You need to know OpenGL Shader Language (GLSL), and I did not have the time to discover that to any real depth.

I hope that I covered topics that are relevant for mobile, embedded and IoT developers. From QtWidgets, QtQuick, QtSensors (of course) to In-app purchasing and building an embedded system with Qt Company's Boot To Qt and Yocto. I also explore Qt's newest platform - Qt for WebAssembly, which allows you to serve Qt applications from a web server to run in a web browser.


Enjoy!

Wednesday, November 22, 2017

Qt for WebAssembly update

The project Qt 5 emscripten that Intopalo (who are doing tons of amazing stuff with some awesome talent) started as research for a client, has moved git repos. The new main git repo has been moved to a branch in Qt proper! and a new name - Qt for WebAssembly.

git://code.qt.io/qt/qtbase.git in the 'wip/webassembly' branch, and can be downloaded here:
git clone -b wip/webassembly git://code.qt.io/qt/qtbase.git

which is also here at github:
https://github.com/qt/qtbase/tree/wip/webassembly

Along with the move comes expanded requirements, like targeting non QtWidgets based apps, including QtDeclarative.

As well, it now builds only for wasm and does not bother with the asmjs fallback, This means your web browser will need to specifically support wasm. Pure wasm builds in emscripten have performance optimizations that do not happen when also building the asmjs fallback.

That said....

*NOTE* Qt for WebAssembly is still very alpha and buggy! Some of which I will cover in the next blog post.

I have managed to conjole Qt Creator to use the emscripten built Qt for apps. Here are the steps to do so, if you are so inclined.

Build Qt for web assembly.

1) Download and build emsdk (you may also try to pre built binary version)

git the emsdk source repo:
git clone https://github.com/juj/emsdk.git
(to update: git pull; ./emsdk update-tags )

Follow these instructions to build emsdk:
http://kripken.github.io/emscripten-site/docs/getting_started/downloads.html#installation-instructions

I usually use 'incoming' instead of 'latest', as it has more recent features.

/path/to/emsdk install sdk-master-64bit binaryen-master-64bit

2) Download Qt for emsdk

git clone -b wip/webassembly https://code.qt.io/qt/qtbase.git

3) Build Qt

source /path/to/emsdk/emsdk_env.sh

$QT_DIR/configure -xplatform emscripten -confirm-license -opensource -nomake tests -nomake examples -developer-build -no-dbus -no-headersclean -release -no-thread  -no-feature-networkinterface

(non developer builds should work too)

4) Add emscripten compiler to Creator

  • Tools->Options->Build & Run -> Compilers->Add->Custom
  • add emcc as C, and emc++ as C++
  • 'emscripten' as Qt mkspecs.
  • ABI: x86, linux (or whatever platform you build on), unknown, elf, 64bit

5) Add emscripten built Qt

  • Tools->Options->Build & Run ->Qt Versions->Add
    • add the qmake from your emscripten Qt build.
6) Add emscripten kit

  • Tools->Options->Build & Run ->Kits->Add
    • add emcc and emc++ as Compilers
    • If you use cmake, you can also add to CMake configuration:
      • CMAKE_TOOLCHAIN_FILE=/path/to/emsdk/emscripten/incoming/cmake/Modules/Platform/Emscripten.cmake
7) Build your app
  • Use 'release' mode build, as 'debug' will result in a runtime error: "failed to asynchronously prepare wasm: CompileError: wasm validation error: at offset 14762: too many functions"
8) Run your app

  • add custom executable: Projects->(qt5-wasm) Run
    • specify /path/to/emsdk/emscripten/incoming/emrun
      • arguments: .html
        • I also use --browser firefox (I use nightly)
Hopefully some things will actually work! This is all still in the development phase. Your kilometerage will vary. 

Good luck! You can join the fun, we are in #qt-webassembly at https://qtmob.slack.com and on freenode.


Monday, June 5, 2017

Qt for web (assembly)

So ya, I have been doing work for the up and coming Finnish company Intopalo getting Qt5 for WebAssembly using emscripten up and running. This is the 3rd Finnish company I have done work for, and the 4th Nordic company (Trolltech, Nokia, Jolla, Intopalo)

This is very much a work in progress.

Background:

There was a Qt4 version, emscripten-qt and a related Qt5 for Native Client which do similar things, and we have taken inspiration from both these projects. Thanks!

The gist of it:

We use emscripten to cross compile Qt5 into javascript and/or webassembly. We use 'incoming' from emscripten's git repo. Details how to build emsdk are here.

My working Qt5 repo is here, (the wm branch contains multi window and window decorations) while there is also an outdated WIP gerrit MR  which will get updated at some point.

This is how I configure Qt for emscripten:
 . ~/emsdk/emsdk_env.sh (to get em compiler in path)

~/depot/qt/qt5/qtbase/configure -xplatform emscripten -confirm-license -opensource -nomake tests  -nomake examples -developer-build -no-dbus -no-thread

 (no-thread is a new feature I added for this, as threading in javascript is basically non existent)

and then, use that qmake to compile a Qt app!

To run the app, in the firefox browser
emrun --browser firefox path/to/app.html

This is all bleeding edge from the emscripten compiler to the web assembly support in the browser, so your mileage may vary.

With the 'wm' branch of my github repo, multi windows and window decorations are kind of working. As well as compiling into webassembly and not just javascript asmjs.
Compiling into wasm brings much smaller download sizes, but you need to have a browser that supports it.

There is still heaps of work to do to bring it up to release state. So for now, it is still a research and WIP project, and a lot of things may or may not work. But it can run some Qt widget based apps in your (firefox) browser!

The future:

Currently there is no support for qml/qt quick. That just means we haven't tried it as our focus is currently on widgets and opengl.

Personally, I would like to see location and some sensor events, but that is for the future.

Friday, May 5, 2017

new QtSensor IoT plugins!

I've said it before and I will say it again... one thing I am tired of hearing about, is that people think Qt is only for UI applications.
WRONG!
Qt is great for IoT as well! even without a UI. (IoT is today's buzzword for 'embedded', which really isn't embedded, just small) and you cannot have IoT without sensors!

I just submitted a MR to QtSensors to add support for the SenseHAT sensor board for raspberry pi:

https://codereview.qt-project.org/#/c/193456
For raspberry pi, you will need RTIMULib installed on your rpi (and dev package to compile QtSensors). It's in the default repo, so you don't actually need to compile RTIMULib yourself.

Even though this is submitted for dev branch and should be available in Qt Sensors 5.10, this will compile and work for the Qt 5.3 that comes with the raspbian distribution.

As well, if my spare time allows, I will also get my Matrix Creator QtSensor plugin into shape to submit as well.

I also have planned getting the TI SensorTag code I wrote shoved into a QtSensor plugin. What's special about the SensorTag is that it has (buzzword alert) wireless sensors! This is a bit more tricky as it will use and be dependent on Qt's QLowEnergyService but will work nonetheless. This plugin doesn't run on the device, but requires a host machine which has bluetooth.

Saturday, January 7, 2017

Movin' on...

A year has gone by since I started work with Canonical. As it turns out, I must be on my way. Where to? Not real sure at this moment, there seems plenty of companies using Qt & QML these days. \0/

But saying that, I am open to suggestions. LinkedIn
 
Plenty of IoT and devices using sensors around. Heck, even Moto Z phone has some great uses for sensor gestures similar to what I wrote for QtSensors while I was at Nokia.

But a lack of companies that allow freelance or remote work. The last few years I have worked remotely doing work for Jolla and Canonical. Both fantastic companies to work for, which really have it together for working remotely.

I am still surprised that only a handful of companies regularly allow remote work. I do not miss the stuffy non window opening offices and the long daily commute, which sometimes means riding a motorcycle through hail! (I do not suggest this for anyone)

Of course, I am still maintainer for QtSensors, QtSystemInfo for the Qt Project, and Sensor Framework for Mer, and always dreaming up new ways to use sensors. Still keeping tabs on QtNetwork bearer classes.

Although I had to send back the Canonical devices, I still have Ubuntu on my Nexus 4. I still have my Jolla phones and tablet.

That said, I still have this blog here, and besides spending my time looking for a new programming gig, I am (always) preparing to release a new album. http://llornkcor.com
and always willing to work with anyone needing music/audio/soundtrack work.

Friday, December 9, 2016

sensorfw & Qt is not just UI framework! or Without sensors, there's no IoT

Like it says on the Intel IoT developer site, "Without sensors, there's no IoT".

Because I am the maintainer of QtSensors, I like to inquire about  people's use of sensors and if they use QtSensors. Over the years, I have heard quite often something like, 'Qt is thought of as a UI framework'. *sigh*
 
But Qt is more than just a UI framework and it's use is not dependent on widgets or declarative wizardry. It is used in quite a few middleware components without UI elements. One of those middleware frameworks is Sensor Framework.

Sensor framework is a daemon that uses a plugin system written using Qt for reading various sensors such as accelerometer or light sensors. It was originally developed by Nokia for Harmattan and ran on the N9. It was also used in MeeGo and later included in the Mer Project and on Jolla phones and the ill fated tablet. So it has been released onto a few commercial products.

We looked at it when I was working at Nokia on the project that I still cannot name, but we had decided we would come up with our own solution. Looking back, this was the wrong decision, we should have taken the already proven sensor framework and ran with that. Why? Because it existed and works.

I started maintaining it when I was a privateer (contractor) developer for Jolla. No one else had touched it for some time so I grabbed the few not yet merged bug fixes and added support for libhybris/android libhardware adaptors.

Sensor Framework has support for multiple clients with down sampling for different data rates. It uses dbus for control lines (to start and stop, etc) but sends data through a socket. It also has a working backend in QtSensors.

I noticed that Ubuntu's Unity does nothing to respond when I put this into "tablet mode". I have to manually open the virtual keyboard among other things.

So I thought I could use sensorfw on my Dell 2 in 1. It's one of those converged laptop/tablet devices. It has a few sensors - accelerometer, gyroscope, magnetometer, and lid sensors. One problem... sensorfw does not support lid sensors, or a few other sensors that are around today in IoT (which I will add a bit later). Lid "sensor" might be a bit of a misnomer, as they could be switches but I'd like to think it is more like a hal effect sensor that uses magnets. In any case there are event nodes to use.

First one I chose is to add the lid sensor - to detect when this machine is put into tablet mode, so the UI can better deal with it.

I also noticed that this kernel has support for iio sensor interface for the accel and gyro. Sensorfw only supports sysfs, evdev and hybris interfaces, so I also wanted to add support for that.

I worked on adding iio support first. Well... really just wrote a sensor adaptor plugin. My plugin supports accelerometer, gyroscope and magnetometer, which this device seems to have. I will expand this to support other sensors later, as well as clean it up a bit.

Thanks to QtSensors sensor framework backend, I can make a UI app change with the orientation and lid changes. Better yet, I can create a game that uses accelerometer data like a marble maze game. Or I can upload the data to one of those Node.js data visualization web apps.

And since sensor framework is opensource, others can as well.