Qt 5

Starting on August 1st, Google Play will no longer accept new applications or application updates without a 64-bit version (unless of course there is no native code at all). For Qt users, this means you have to build an additional APK that contains ... [More] the 64-bit binaries. Qt has shipped 64-bit binaries for Android since Qt 5.12.0, so complying with the new requirement is technically no big deal. But after discussing with users, I see that it is not clear to everyone exactly how to set up an app in Google Play that supports multiple architectures at once. This call for help, combined with the fact that I am currently setting up a fresh Windows work station, made for a golden opportunity to look at Qt for Android app development in general. In this blog, I will start with a clean slate and show how to get started on Android, as well as how to publish an app that complies with the Google Play requirements. I will guide you through the installation steps needed to get a working environment, describe the process of building an application for multiple architectures, and show you how to upload your binaries to Google Play. The first few parts might be familiar to many of you, so if you get bored and want to hear about the main topic, feel free to skip right to Step 4. A note about SDK versions The Android SDK is itself under heavy development, and quite often it isn’t backwards compatible, causing problems with our integration in Qt. We react as quickly as we can to issues that arise from changes or regressions in the SDK, but a general rule of thumb is to wait before you upgrade to the latest and greatest versions of the Android tools until we have had a chance to adapt to incompatibilities in Qt. While there have been some issues on other platforms as well, the majority of the problems we have seen have been on Windows. So if you are on this host system, be extra aware to check for known good versions before setting up your environment. We are currently recommending the use of the following tools together with Qt 5.13.0: Android build tools version 28 Android NDK r19 Java Development Kit 8 If you do bump into some problems, please make sure to check our known issues page to see if there is any updated information. Now for the details on where and how to get the right versions of everything. Step 1: Installing the JDK Android is primarily a Java-based platform, and while you can write your Qt applications entirely in C++ and/or QML, you will need the Java Development Kit in order to compile the files that make the integration possible. However, there is an incompatibility between Android’s SDK Manager tool and the later versions of Oracle’s JDK, making the latest JDK versions unusable together with the Android environment. To work around this, we recommend that you download JDK version 8 for use with Android. Download and run the installer from Oracle (you will have to register an Oracle account if you haven’t already) and install it in the default location. Step 2: Setting up the Android environment The second step is getting the actual Android development environment. Start by downloading and installing Android Studio. Scroll past the different “beta” and “canary” releases, and you will find the latest stable release. Once the Android Studio has been installed, you can use this to install the “SDK Platform” for you. This is the actual collection of Java classes for a particular Android distribution. When you start Android Studio the first time, it should prompt you to install the SDK Platform. You can safely use the latest version of the SDK, platform 29, which is the suggested default. In addition to the SDK, we also need to install the NDK. This is the development kit used for cross-compiling your C++ code to run on Android. As mentioned above, we will use Android NDK r19c and not the latest release, since there are issues with Android NDK r20 causing compilation errors. The issue will be been addressed in Qt 5.13.1 and Qt 5.12.5, so when you start using those, then upgrading to Android NDK r20 is possible. And as a final step, we need to make sure that we are using version 28.0.3 of the Android build tools rather than the latest version. Note that this is only an issue on Windows hosts. From the starting dialog box of Android Studio, click on Configure and then select SDK Manager. Go to the SDK Tools tab and make sure Show Package Details is checked. Under the Android build tools, make sure you deselect 29.0.0 and select 28.0.3 instead. This will uninstall the non-functioning version of the build tools and install the older one. Click Apply to start the process, and when it is done you will have installed a functioning Android environment. Step 3: Install Qt For this guide, we will be using Qt 5.13.0. If you haven’t already, start by downloading the online installer tool from your Qt Account. When you run the installer, make sure you select the arm64-v8a and armv7a target architectures. These are technical names for, respectively, the 64-bit and 32-bit versions of the ARM family of processors, which is the most commonly used processors on Android devices. Note: For this example in particular, we will also need Qt Purchasing, since it contains the application I am planning to use as demonstration. This can also be selected from the same list. When Qt is finished installing, start Qt Creator and open the Options. Under Devices, select the Android tab and select the directories where you installed the different packages in the previous steps. If everything is set up correctly, Qt Creator will show a green check mark, and you will be ready to do Android development with Qt. Step 4: Setting up project in Qt Creator For this example, I will use the Qt Hangman example. This is a small example we made to show how to implement in-app purchases in a cross-platform way. First we open the example in Qt Creator, which can be done from the Welcome screen. Once it has been opened, Qt Creator will ask us to select which Qt versions we want to use for building it. Select both the 64-bit and 32-bit versions of Qt and click Configure Project. In order to comply with the additional requirements in Google Play, we want to create two APK packages: One for 32-bit devices and one for 64-bit devices. We need to configure each of these separately. This screenshot shows an example setup for the 32-bit build. Important things to notice here: Use a different shadow build directory for each of the builds. Make sure you select the Release configuration. You should also tick the Sign package checkbox to sign your package, otherwise the Google Play store will reject it. With the exception of the build directory, the setup for the 64-bit build should be the same. Select the 64-bit kit on the left-hand side and make the equivalent adjustments there. Step 5: Preparing the manifest In addition, the two packages will need identical AndroidManifest.xml files, except for one detail: The version code of the two has to differ. The version code can be pretty much anything you choose, as long as you keep in mind that when an APK is installed on a device from the store, it will select the APK with the highest version code. As Qt user Fabien Chéreau poined out in a comment to a bug report, you therefore typically want to set the version code of the 64-bit version to be higher than for the 32-bit version, so that a device which supports both will prefer the 64-bit one. As Felix Barz pointed out in the same thread, this can be automated in the .pro file of the project. Here is my slightly modified version of his code: defineReplace(droidVersionCode) { segments = $$split(1, ".") for (segment, segments): vCode = "$$first(vCode)$$format_number($$segment, width=3 zeropad)" contains(ANDROID_TARGET_ARCH, arm64-v8a): \ suffix = 1 else:contains(ANDROID_TARGET_ARCH, armeabi-v7a): \ suffix = 0 # add more cases as needed return($$first(vCode)$$first(suffix)) } VERSION = 1.2.3 ANDROID_VERSION_NAME = $$VERSION ANDROID_VERSION_CODE = $$droidVersionCode($$ANDROID_VERSION_NAME) This neat trick (thanks, Felix!) will convert the application’s VERSION to an integer and append a new digit, on the least significant end, to signify the architecture. So for version 1.2.3 for instance, the version code will be 0010020030 for the 32-bit package and 0010020031 for the 64-bit one. When you generate an AndroidManifest.xml using the button under Build APK in the project settings, this will automatically pick up this version code from the project. Once you have done that and edited the manifest to have your application’s package name and title, the final step is to build the package: First you do a build with one of the two kits, and then you must activate the other kit and do the build again. When you are done, you will have two releasable APK packages, one in each of the build directories you set up earlier. Relative to the build directory, the package will be in android-build\build\outputs\apk\release. Note that for a more efficient setup, you will probably want to automate this process. This is also quite possible, since all the tools used by Qt Creator can be run from the command line. Take a look at the androiddeployqt documentation for more information. Step 6: Publish the application in Google Play The Google Play publishing page is quite self-documenting, and there are many good guides out there on how to do this, so I won’t go through all the steps for filling out the form. In general, just fill out all the information it asks for, provide the images it needs, and make sure all the checkmarks in the left side bar are green. You can add all kinds of content here, so take your time with it. In the end, it will have an impact on how popular your app becomes. Once that has been done, you can create a new release under App Releases and upload your APKs to it. One thing to note is that the first time you do this, you will be asked if you want to allow Google Play to manage your app signing key. For now, you will have to select to Opt Out of this. In order to use this feature, the application has to be in the new “Android App Bundle” format. This is not yet supported by Qt, but we are working to support this as well. In fact, Bogdan Vatra from KDAB (who is also the maintainer of the Android port of Qt) has already posted a patch which addresses the biggest challenge in getting such support in place. When we do get support for it, it will make the release process a little bit more convenient. With the AAB format, Google Play will generate the optimized APKs for different architectures for us, but for now we have to do this manually by setting up multiple kits and building multiple APKs, as I have described in this tutorial. When the two APKs have been uploaded to a release, you should see a listing such as this: Two separate APK packages, each covering a single native platform. By expanding each of the entries, you can see what the “Differentiating APK details” are. These are the criteria used for selecting one over the other when a device is downloading the APK from the Google Play Store. In this case, the differentiating detail should be the native platform. And that is all there is to it: Creating and releasing a Qt application in Google Play with both 32-bit and a 64-bit binaries. When the APKs have been uploaded, you can hit Publish and wait for Google Play to do its automated magic. And if you do have existing 32-bit apps in the store at the moment, make sure you update them with a 64-bit version well before August 2021, as that is when non-compliant apps will no longer be served to 64-bit devices, even if they also support 32-bit binaries. Until then, happy hacking and follow me on Twitter for irregular updates and fun curiosities. The post How to comply with the upcoming requirements in Google Play appeared first on Qt Blog. [Less]

Starting on August 1st, Google Play will no longer accept new applications or application updates without a 64-bit version (unless of course there is no native code at all). For Qt users, this means you have to build an additional APK that contains ... [More] the 64-bit binaries. Qt has shipped 64-bit binaries for Android since Qt 5.12.0, so complying with the new requirement is technically no big deal. But after discussing with users, I see that it is not clear to everyone exactly how to set up an app in Google Play that supports multiple architectures at once. This call for help, combined with the fact that I am currently setting up a fresh Windows work station, made for a golden opportunity to look at Qt for Android app development in general. In this blog, I will start with a clean slate and show how to get started on Android, as well as how to publish an app that complies with the Google Play requirements. I will guide you through the installation steps needed to get a working environment, describe the process of building an application for multiple architectures, and show you how to upload your binaries to Google Play. The first few parts might be familiar to many of you, so if you get bored and want to hear about the main topic, feel free to skip right to Step 4. A note about SDK versions The Android SDK is itself under heavy development, and quite often it isn’t backwards compatible, causing problems with our integration in Qt. We react as quickly as we can to issues that arise from changes or regressions in the SDK, but a general rule of thumb is to wait before you upgrade to the latest and greatest versions of the Android tools until we have had a chance to adapt to incompatibilities in Qt. While there have been some issues on other platforms as well, the majority of the problems we have seen have been on Windows. So if you are on this host system, be extra aware to check for known good versions before setting up your environment. We are currently recommending the use of the following tools together with Qt 5.13.0: Android build tools version 28 Android NDK r19 Java Development Kit 8 If you do bump into some problems, please make sure to check our known issues page to see if there is any updated information. Now for the details on where and how to get the right versions of everything. Step 1: Installing the JDK Android is primarily a Java-based platform, and while you can write your Qt applications entirely in C++ and/or QML, you will need the Java Development Kit in order to compile the files that make the integration possible. Note that there is an incompatibility between Android’s SDK Manager tool and the later versions of Oracle’s JDK, making the latest JDK versions unusable together with the Android environment. To work around this, we recommend that you download JDK version 8 for use with Android. You may use the official binaries from Oracle, or an alternative, such as the AdoptOpenJDK project. Download and run the installer and install it in the default location. Step 2: Setting up the Android environment The second step is getting the actual Android development environment. Start by downloading and installing Android Studio. Scroll past the different “beta” and “canary” releases, and you will find the latest stable release. Once the Android Studio has been installed, you can use this to install the “SDK Platform” for you. This is the actual collection of Java classes for a particular Android distribution. When you start Android Studio the first time, it should prompt you to install the SDK Platform. You can safely use the latest version of the SDK, platform 29, which is the suggested default. In addition to the SDK, we also need to install the NDK. This is the development kit used for cross-compiling your C++ code to run on Android. As mentioned above, we will use Android NDK r19c and not the latest release, since there are issues with Android NDK r20 causing compilation errors. The issue will be been addressed in Qt 5.13.1 and Qt 5.12.5, so when you start using those, then upgrading to Android NDK r20 is possible. And as a final step, we need to make sure that we are using version 28.0.3 of the Android build tools rather than the latest version. Note that this is only an issue on Windows hosts. From the starting dialog box of Android Studio, click on Configure and then select SDK Manager. Go to the SDK Tools tab and make sure Show Package Details is checked. Under the Android build tools, make sure you deselect 29.0.0 and select 28.0.3 instead. This will uninstall the non-functioning version of the build tools and install the older one. Click Apply to start the process, and when it is done you will have installed a functioning Android environment. Step 3: Install Qt For this guide, we will be using Qt 5.13.0. If you haven’t already, start by downloading the online installer tool from your Qt Account. When you run the installer, make sure you select the arm64-v8a and armv7a target architectures. These are technical names for, respectively, the 64-bit and 32-bit versions of the ARM family of processors, which is the most commonly used processors on Android devices. Note: For this example in particular, we will also need Qt Purchasing, since it contains the application I am planning to use as demonstration. This can also be selected from the same list. When Qt is finished installing, start Qt Creator and open the Options. Under Devices, select the Android tab and select the directories where you installed the different packages in the previous steps. If everything is set up correctly, Qt Creator will show a green check mark, and you will be ready to do Android development with Qt. Step 4: Setting up project in Qt Creator For this example, I will use the Qt Hangman example. This is a small example we made to show how to implement in-app purchases in a cross-platform way. First we open the example in Qt Creator, which can be done from the Welcome screen. Once it has been opened, Qt Creator will ask us to select which Qt versions we want to use for building it. Select both the 64-bit and 32-bit versions of Qt and click Configure Project. In order to comply with the additional requirements in Google Play, we want to create two APK packages: One for 32-bit devices and one for 64-bit devices. We need to configure each of these separately. This screenshot shows an example setup for the 32-bit build. Important things to notice here: Use a different shadow build directory for each of the builds. Make sure you select the Release configuration. You should also tick the Sign package checkbox to sign your package, otherwise the Google Play store will reject it. With the exception of the build directory, the setup for the 64-bit build should be the same. Select the 64-bit kit on the left-hand side and make the equivalent adjustments there. Step 5: Preparing the manifest In addition, the two packages will need identical AndroidManifest.xml files, except for one detail: The version code of the two has to differ. The version code can be pretty much anything you choose, as long as you keep in mind that when an APK is installed on a device from the store, it will select the APK with the highest version code. As Qt user Fabien Chéreau poined out in a comment to a bug report, you therefore typically want to set the version code of the 64-bit version to be higher than for the 32-bit version, so that a device which supports both will prefer the 64-bit one. As Felix Barz pointed out in the same thread, this can be automated in the .pro file of the project. Here is my slightly modified version of his code: defineReplace(droidVersionCode) { segments = $$split(1, ".") for (segment, segments): vCode = "$$first(vCode)$$format_number($$segment, width=3 zeropad)" contains(ANDROID_TARGET_ARCH, arm64-v8a): \ suffix = 1 else:contains(ANDROID_TARGET_ARCH, armeabi-v7a): \ suffix = 0 # add more cases as needed return($$first(vCode)$$first(suffix)) } VERSION = 1.2.3 ANDROID_VERSION_NAME = $$VERSION ANDROID_VERSION_CODE = $$droidVersionCode($$ANDROID_VERSION_NAME) This neat trick (thanks, Felix!) will convert the application’s VERSION to an integer and append a new digit, on the least significant end, to signify the architecture. So for version 1.2.3 for instance, the version code will be 0010020030 for the 32-bit package and 0010020031 for the 64-bit one. When you generate an AndroidManifest.xml using the button under Build APK in the project settings, this will automatically pick up this version code from the project. Once you have done that and edited the manifest to have your application’s package name and title, the final step is to build the package: First you do a build with one of the two kits, and then you must activate the other kit and do the build again. When you are done, you will have two releasable APK packages, one in each of the build directories you set up earlier. Relative to the build directory, the package will be in android-build\build\outputs\apk\release. Note that for a more efficient setup, you will probably want to automate this process. This is also quite possible, since all the tools used by Qt Creator can be run from the command line. Take a look at the androiddeployqt documentation for more information. Step 6: Publish the application in Google Play The Google Play publishing page is quite self-documenting, and there are many good guides out there on how to do this, so I won’t go through all the steps for filling out the form. In general, just fill out all the information it asks for, provide the images it needs, and make sure all the checkmarks in the left side bar are green. You can add all kinds of content here, so take your time with it. In the end, it will have an impact on how popular your app becomes. Once that has been done, you can create a new release under App Releases and upload your APKs to it. One thing to note is that the first time you do this, you will be asked if you want to allow Google Play to manage your app signing key. For now, you will have to select to Opt Out of this. In order to use this feature, the application has to be in the new “Android App Bundle” format. This is not yet supported by Qt, but we are working to support this as well. In fact, Bogdan Vatra from KDAB (who is also the maintainer of the Android port of Qt) has already posted a patch which addresses the biggest challenge in getting such support in place. When we do get support for it, it will make the release process a little bit more convenient. With the AAB format, Google Play will generate the optimized APKs for different architectures for us, but for now we have to do this manually by setting up multiple kits and building multiple APKs, as I have described in this tutorial. When the two APKs have been uploaded to a release, you should see a listing such as this: Two separate APK packages, each covering a single native platform. By expanding each of the entries, you can see what the “Differentiating APK details” are. These are the criteria used for selecting one over the other when a device is downloading the APK from the Google Play Store. In this case, the differentiating detail should be the native platform. And that is all there is to it: Creating and releasing a Qt application in Google Play with both 32-bit and a 64-bit binaries. When the APKs have been uploaded, you can hit Publish and wait for Google Play to do its automated magic. And if you do have existing 32-bit apps in the store at the moment, make sure you update them with a 64-bit version well before August 2021, as that is when non-compliant apps will no longer be served to 64-bit devices, even if they also support 32-bit binaries. Until then, happy hacking and follow me on Twitter for irregular updates and fun curiosities. The post How to comply with the upcoming requirements in Google Play appeared first on Qt Blog. [Less]

Cutelyst a Qt/C++ Web framework got a new release! This release took a while to be out because I wanted to fix some important stuff, but time is short, I've been working on polishing my UPnpQt library and on a yet to be released FirebaseQt and ... [More] FirebaseQtAdmin (that's been used on a mobile app and REST/WebApp used with Cutelyst), the latter is working quite well although it depends ATM on a Python script to get the Google token, luckly it's a temporary waste of 25MB of RAM each 45 minutes. Back to the release, thanks to Alexander Yudaev it has cpack support now and 顏子鳴 also fixed some bugs and added a deflate feature to RenderView, FastCGI and H2. I'm also very happy we now have more than 500 stars on GitHub :) Have fun https://github.com/cutelyst/cutelyst/releases/tag/v2.8.0 [Less]

We are happy to announce the Qt 3D Studio 2.4 release is now available via the online and offline installers. Here’s a quick summary of the new features and functions in 2.4. For detailed information about the Qt 3D Studio, visit the online ... [More] documentation page or see the older blog posts. Changes in the runtime One of the biggest changes in the 2.4 release is removing the Qt 3D dependency from the Qt 3D Studio runtime and using the architecture we have been using in 1.x releases. The main reason for this change is performance especially in the embedded environments. We have seen quite significant performance differences between the runtimes even after the optimizations done in Qt 3D Studio 2.3 runtime. For details about the performance improvements please see the separate blog post. The change in 3D runtime does not require any other code changes than changing the import statements (e.g. in QML import QtStudio3D.OpenGL 2.4 instead of import QtStudio3D 2.3) and then recompilation with new Qt 3D Studio 2.4 is enough. Please refer to the Getting Started documentation. When opening a presentation created with earlier version of the Qt 3D Studio you might also get a notification that “Some custom materials, effects, and behaviors may not work correctly”. This is related to an updated presentation file format which defines also also alpha for the colors i.e. now colors are type vec4 instead of vec3. Saving the presentation with a newer version removes the notification. Dynamic object creation Qt 3D Studio C++ API supports now also dynamic object creation. This feature is handy in use cases where you have to create new objects to the scene for example based on e.g. sensor values or the scene contains predetermined amount of certain objects. Dynamic Element Example Dynamically created objects can be new instances of objects included in the presentation or object geometry can be also created during runtime. Also object materials can be defined dynamically. Please see dynamicelement example from the runtime installation folder. Vertex Shaders 2.4 version also enables using vertex shaders in the custom materials. Vertex shaders can be used to transform the attributes of vertices i.e. the original objects can be distorted or reshaped in any manner. For more information about creating custom materials please refer to the documentation. Getting started Qt 3D Studio 2.4 is available through Qt online installer under the Developer and Designer Tools section. We also provide standalone offline installers which contain all you need to start designing Qt 3D Studio User Interfaces. Online installer also contains pre-build runtime for Qt 5.12 which is needed for developing Qt applications using Qt 3D Studio UI. Qt online installer and offline installers can be obtained from Qt Download page and commercial license holders can find the packages from Qt Account. Binary packages are available for Windows, Mac and Linux. If you are targeting for embedded systems with running e.g. RTOS you need to build the Qt 3D Studio runtime component for the target operating system. Qt 3D Studio runtime can be found from the git repository. Building instructions can be found from documentation.     The post Qt 3D Studio 2.4 Released appeared first on Qt Blog. [Less]

In the first blog in this series, I showed how we solved the original problem of how to use mmap() to load a large set of data into RAM all at once, in response to a request for help from a bioinformatics group dealing with massive data sets on a ... [More] regular basis. The catch in our solution, however, was that the process still took too long. In this blog, I describe how we solve this, starting with Step 3 of the Process I introduced in Blog 1: 3. Fine-grained Threading The original code we inherited was written on the premise that: Eigen uses OpenMP to utilize multiple cores for vector and matrix operations. Writing out the results of the Monte Carlo simulation is time-consuming and therefore put into its own thread by way of OpenMPI with another OpenMPI critical section doing the actual analysis. Of course, there were some slight flaws in this plan. Eigen’s use of OpenMP is only for some very specific algorithms built into Eigen itself. None of which this analysis code was using, so that was useless. Eigen does make use of vectorization, however, which is good and can in ideal circumstances give a factor of 4 speedup compared to a simplistic implementation. So we wanted to keep that part. The threading for writing results was, shall we say, sub-optimal. Communication between the simulation thread and the writer thread was by way of a lockless list/queue they had found on the interwebs. Sadly, this was implemented with a busy spin loop which just locked the CPU at 100% whilst waiting for data to arrive once every n seconds or minutes. Which means it’s just burning cycles for no good reason. The basic outline algorithm looks something like this: const std::vector colIndices = {0, 1, 2, 3, ... }; const std::vector markerIndices = randomise(colIndices); for (i = 0; i < maxIterations; ++i) { for (j = 0; j < numCols; ++j) { const unsigned int marker = markerIndices[j]; const auto col = data.mappedZ.col(marker); output += doStuff(col); } if (i % numIterations == 0) writeOutput(output); } So, what can we do to make better use of the available cores? For technical reasons related to how Markov Chain Monte Carlo works, we can neither parallelize the outer loop over iterations nor the inner loop over the columns (SNPs). What else can we do? Well, recall that we are dealing with large numbers of individuals – 500,000 of them in fact. So we could split the operations on these 500k elements into smaller chunks and give each chunk to a core to process and then recombine the results at the end. If we use Eigen for each chunk, we still get to keep the SIMD vectorization mentioned earlier. Now, we could do that ourselves but why should we worry about chunking and synchronization when somebody else has already done it and tested it for us? This was an ideal chance for me to try out Intel’s Thread Building Blocks library, TBB for short. As of 2017 this is now available under the Apache 2.0 license and so is suitable for most uses. TBB has just the feature for this kind of quick win in the form of its parallel_for and parallel_reduce template helpers. The former performs the map operation (applies a function to each element in a collection where each is independent). The latter performs the reduce operation, which is essentially a map operation followed by a series of combiner functions, to boil the result down to a single value. These are very easy to use so you can trivially convert a serial piece of code into a threaded piece just by passing in the collection and lambdas representing the map function (and also a combiner function in the case of parallel_reduce). Let’s take the case of a dot (or scalar) product as an example. Given two vectors of equal length, we multiply them together component-wise then sum the results to get the final value. To write a wrapper function that does this in parallel across many cores we can do something like this: const size_t grainSize = 10000; double parallelDotProduct(const VectorXf &Cx, const VectorXd &y_tilde) { const unsigned long startIndex = 0; const unsigned long endIndex = static_cast(y_tilde.size()); auto apply = [&](const blocked_range& r, double initialValue) { const long start = static_cast(r.begin()); const long count = static_cast(r.end() - r.begin()); const auto sum = initialValue + (Cx.segment(start, count).cast() * y_tilde.segment(start, count)).sum(); return sum; }; auto combine = [](double a, double b) { return a + b; }; return parallel_reduce(blocked_range(startIndex, endIndex, grainSize), 0.0, apply, combine); } Here, we pass in the two vectors for which we wish to find the scalar product, and store the start and end indices. We then define two lambda functions. The apply lambda, simply uses the operator * overload on the Eigen VectorXf type and the sum() function to calculate the dot product of the vectors for the subset of contiguous indices passed in via the blockedRange argument. The initialValue argument must be added on. This is just zero in this case, but it allows you to pass in data from other operations if your algorithm needs it. The combine lambda then just adds up the results of each of the outputs of the apply lambda. When we then call parallel_reduce with these two functions, and the range of indices over which they should be called, TBB will split the range behind the scenes into chunks based on a minimum size of the grainSize we pass in. Then it will create a lightweight task object for each chunk and queue these up onto TBB’s work-stealing threadpool. We don’t have to worry about synchronization or locking or threadpools at all. Just call this one helper template and it does what we need! The grain size may need some tuning to get optimal CPU usage based upon how much work the lambdas are performing but as a general rule of thumb, it should be such that there are more chunks (tasks) generated than you have CPU cores. That way the threadpool is less likely to have some cores starved of work. But too many and it will spend too much time in the overhead of scheduling and synchronizing the work and results between threads/cores. I did this for all of the operations in the inner loop’s doStuff() function and for some others in the outer loop which do more work across the large (100,000+ element) vectors and this yielded a very nice improvement in the CPU utilization across cores. So far so good. In the next blog, I’ll show you how we proceed from here, as it turns out this is not the end of the story. The post Little Trouble in Big Data – Part 2 appeared first on KDAB. [Less]

As of the 5.13 release, Qt for WebAssembly now has experimental support for multithreading. This has much of the same benefits as threads on other platforms: the application can offload work to secondary threads to keep the main thread responsive ... [More] , and make use of multiple CPU cores. Secondary threads on the web platform has one additional benefit in that they can block without unwinding the stack. This is not true for main thread, which must return control to the browser after processing an event or risk having the browser show the “page is unresponsive” notification. WebAssembly multithreading has been possible but disabled by default for some time. Browsers have now begun enabling it again, first out is Chrome 67. Old New Thing These days it’s not too often that we get to introduce threads as a feature, so I took the opportunity to touch up Qt’s classic Mandelbrot threading example. Upgrades include now using all CPU cores instead of just one (using QThread::idealThreadCount() to control the amount of worker threads). A quick peek at the task manager confirms this: Rest assured, CPU usage does return to 0% when the application is idle. An example build of the demo and source code are available. The build should run on desktop Chrome, and also on Firefox if you enable javascript.options.shared_memory in about:config page. Practicalities Next, lets look at some of the practicalities of working with multithreaded builds of Qt with Emscripten. If you have experience in this area you’d like to share, please chime in in the comments section below. Emscripten is doing most of the heavy lifting here, and provides support for the pthreads API, implemented using Web Workers and SharedArrayBuffer. Qt then reuses its existing unix QThread implementation. Enabling The default Qt for WebAssembly build disables threads by default. To enable, build from source and configure with the -feature-thread flag. We’ve found that emscripten 1.38.30 works well for threaded builds. Threading-enabled binaries will not run on browsers with SharedArrayBuffer disabled; you may see error messages such as: CompileError: wasm validation error: at offset 5711: shared memory is disabled Error: WebAssembly.Module doesn’t parse at byte 5705: can’t parse resizable limits flags” Main thread deadlocks Calling QThread::wait (or pthread_join) on the main thread may deadlock, since the browser will be blocked from servicing application requests such as starting a new Web Worker for the thread we are waiting on. Possible workarounds include: Pre-allocate Web Workers using QMAKE_WASM_PTHREAD_POOL_SIZE (maps to emscripten PTHREAD_POOL_SIZE) Don’t join worker threads when not needed (e.g. app exit). Earlier versions of the Mandelbrot demo was freezing the tab on exit; solved by disabling application cleanup for the Qt build which powers it. Fixed Memory Size: Emscripten will normally increase the heap size as needed, starting from small initial allocation. This is not yet supported for multithreaded builds and a fixed memory size must be set. We’ve empirically determined that most (desktop) browsers limit this initial allocation to 1GB, and this is the size Qt sets by default. You can change the value by setting QMAKE_WASM_PTHREAD_POOL_SIZE (maps to emscripten PTHREAD_POOL_SIZE) See the documentation or Wiki page for further info on Qt for WebAssembly, or ask below. The post Qt for WebAssembly: Multithreading appeared first on Qt Blog. [Less]

KDAB is sharing the Qt booth at SIGGRAPH in Los Angeles. We’ll be showing some of our profiling and debugging tools as well as our latest QiTissue demo, a desktop Application developed for Quantitative Imaging Systems (Qi) to help cancer researchers ... [More] efficiently handle gigabytes of data (see more about that here), We’ll also give a preview of the related nanoquill coloring App which is due for release in time for the show. Performance enhancing tools you can expect to see in action at SIGGRAPH are: KDAB GammaRay, our Qt introspection tool, and Clazy Static Code Analyzer. Meet us at SIGGRAPH, 2019! The post KDAB at SIGGRAPH – 2019 appeared first on KDAB. [Less]

KDAB is sharing the Qt booth at SIGGRAPH in Los Angeles. We’ll be showing some of our profiling and debugging tools as well as our latest QiTissue demo, a desktop Application developed for Quantitative Imaging Systems (Qi) to help cancer researchers ... [More] efficiently handle gigabytes of data (see more about that here), We’ll also give a preview of the related nanoquill coloring App which is due for release in time for the show. Performance enhancing tools you can expect to see in action at SIGGRAPH are: KDAB GammaRay, our Qt introspection tool, and Clazy Static Code Analyzer. Meet us at SIGGRAPH, 2019! if you can’t make it in person, you might want to try the the Livestream sessions. The post KDAB at SIGGRAPH – 2019 appeared first on KDAB. [Less]

This is the first in a series of blog posts about 3D and the interaction with Qt, KUESA and Qt 3D Studio, and other things that pop up when we’re working on something. I’m a 3D designer, mostly working in blender. Sometimes I come across interesting ... [More] problems and I’ll try to share those here. For example, trying to display things on low-end hardware – where memory is sometimes limited, meaning every polygon and triangle counts;  where the renderer doesn’t do what the designer wants it to, that sort of thing. The problem that I’ll cover today is, how to easily create a reflection in KUESA or Qt 3D Studio. Neither KUESA or Qt 3D Studio will give you free reflections. If you know a little about 3D, you know that requires ray tracing software, not OpenGL. So, I wondered if there would be an easy way to create this effect. I mean, all that a reflection is, is a mirror of an object projected onto a plane, right? So, I wondered, could this be imitated? To recreate this, I’d need to create an exact mirror of the object and duplicate it below the original, and have a floor that is partially transparent. I’ve created a simple scene to show you how this technique works – a scene with two cubes, a ground plane and a point light. Here’s the result of this scene. It’s starting to look like something, but I want to compare it to a ‘real’ reflection. For comparison, the above is a cube on a reflective, rough surface – showing the result using raytracing. You can see here the reflection is different from our example above – the main issue is that the reflection eventually fades out the further away it gets from the contact point.  How to resolve this? This can be mimicked by creating an image texture for the alpha that fades out the model towards the top (or rather the bottom) of the reflection. I can also further enhance the illusion by ensuring that the floor is rough – allowing the texture of the surface to assist the illusion of a reflection. Another difference between the shots is the blurriness on the edge of the mesh – this could be approximated by creating duplicates of the mesh and for each one, increasing the size and reducing the opacity. Depending on the complexity of the model, this may add too many polygons to render, while only adding a subtle effect. So, given that this is a very simple example and not one that would translate well to something that a client might ask for, how can I translate this into a more complex model, such as the car below? I’ll chat about that in the next post. The post 3D – Interactions with Qt, KUESA and Qt Design Studio, Part 1 appeared first on KDAB. [Less]

This is the first in a series of blog posts about 3D and the interaction with Qt, KUESA and Qt 3D Studio, and other things that pop up when we’re working on something. I’m a 3D designer, mostly working in blender. Sometimes I come across interesting ... [More] problems and I’ll try to share those here. For example, trying to display things on low-end hardware – where memory is sometimes limited, meaning every polygon and triangle counts;  where the renderer doesn’t do what the designer wants it to, that sort of thing. The problem that I’ll cover today is, how to easily create a reflection in KUESA or Qt 3D Studio. Neither KUESA or Qt 3D Studio will give you free reflections. If you know a little about 3D, you know that requires ray tracing software, not OpenGL. So, I wondered if there would be an easy way to create this effect. I mean, all that a reflection is, is a mirror of an object projected onto a plane, right? So, I wondered, could this be imitated? To recreate this, I’d need to create an exact mirror of the object and duplicate it below the original, and have a floor that is partially transparent. I’ve created a simple scene to show you how this technique works – a scene with two cubes, a ground plane and a point light.   Here’s the result of this scene. It’s starting to look like something, but I want to compare it to a ‘real’ reflection. For comparison, the above is a cube on a reflective, rough surface – showing the result using raytracing. You can see here the reflection is different from our example above – the main issue is that the reflection eventually fades out the further away it gets from the contact point.  How to resolve this? This can be mimicked by creating an image texture for the alpha that fades out the model towards the top (or rather the bottom) of the reflection. I can also further enhance the illusion by ensuring that the floor is rough – allowing the texture of the surface to assist the illusion of a reflection. Another difference between the shots is the blurriness on the edge of the mesh – this could be approximated by creating duplicates of the mesh and for each one, increasing the size and reducing the opacity. Depending on the complexity of the model, this may add too many polygons to render, while only adding a subtle effect. So, given that this is a very simple example and not one that would translate well to something that a client might ask for, how can I translate this into a more complex model, such as the car below? I’ll chat about that in the next post. The post 3D – Interactions with Qt, KUESA and Qt Design Studio, Part 1 appeared first on KDAB. [Less]