Qt6 Webassembly: QtMultimedia or, How to play a video in a web browser using Qt

QtMultimedia in WebAssembly

Since Qt 6.5.0, QtMultimedia has support for playing video in a QGraphicsVideoItem and QGraphicsScene, as well as recording from a camera. You can use Qt to play video, access the camera from a web browser, thus simplifying deployment.

With anything WebAssembly, there are a few things that work differently from the desktop applications. The javascript function that is used to gather the list of devices,  getUserMedia is asyncronous, returning a javascript Promise, so the list is not readily available. This function can take some amount of time, depending on device and browser. I have found that sometimes, it is inexplicitly long. 

You can connect to the QMediaDevices::audioInputsChanged, QMediaDevices::videoInputsChanged and QMediaDevices::audioOutputsChanged signals to discover when they are available.

QPermission

Browser permissions are required for accessing the microphone and camera, and a secure https context is a prerequisite for granting them. If this is not run or permission is not granted by the user, the list of available devices will be missing the label, or name, of any device not given permission and it will be unavailable for use.

Using the new QPermission API which will be introduced in Qt 6.5 will make sure your app is permissions ready. The example QtMultimedia apps have been changed in Qt 6.6 to use this API.

You do not need to use QPermissions in wasm, because it uses the same function - getUserMedia which causes the browser to ask for permissions.

See https://developer.mozilla.org/en-US/docs/Web/API/Permissions_API

Camera

The camera (and microphone) need user permissions. Something like this will work for granting permissions to use the camera:

    QCameraPermission cameraPermission;

    switch (qApp->checkPermission(cameraPermission)) {

    case Qt::PermissionStatus::Undetermined:

        qApp->requestPermission(cameraPermission, this, &MainWindow::init);

        return;

    case Qt::PermissionStatus::Denied:

        qWarning("Camera permission is not granted!");

        return;

    case Qt::PermissionStatus::Granted:

        break;

    }

What this can look like in the browser:

When you connect to the signal QMediaDevices::videoInputChanged, it will be emitted when the list of camera devices are available. 

   connect(m_mediaDevices, &QMediaDevices::videoInputsChanged, [this]() { doCamera();});

Here is a screenshot of the webcam showing in a test app I used for developing this feature. The sources for this are in qtmultimedia/tests/manual/wasm/camera

You can try it out for yourself! camera-test

(Chrome 114 seems to have issue with VideoFrame, working with Chrome 117) 

Video

Using Javascript's WebCodecs API, we are be able to leverage Javascript's VideoFrame API, which is currently available on Chrome and Safari based browsers.

Permissions are not needed to play video.

Video currently only works with QWidget based apps. Playing video in quick apps will be fixed in an upcoming version of Qt. Here's a screenshot of the test app I used to develop it:

Video recording also works and will save the file to the browsers local filesystem, which has a limited capacity. It can then be played as normal or downloaded to the system filesystem via QFileDialog:SaveFileContent

Known Issues

• List of media devices is not readily available for application.
• Video does not show in QtQuick declarative apps