This week brought an exuberant amount of openSUSE Tumbleweed snapshots.

While the rolling release snapped its streak of continuous daily snapshots, Tumbleweed persists releasing numerous snapshots; in total, five have been released so far this week.

The last snapshot, 20211207, updated one package that gamers will appreciate. The computer opponent for the board game Blokus was updated with the release of pentobi 19.1. The bug fixing update provided a work around for a crash that happened during an exit in some situations. The package also avoids a warning with Qt 6 caused by a deprecated signal-handler syntax.

Snapshot 20211206 updated the 3D graphics package Mesa to version 21.3.1. The updated provided mostly AMD, Intel and Zink fixes. The package also added a work around to fix a segfault with the first-person shooter video game Metro Exodus, which announced availability with Linux in April 2021. The highly portable implementation of the Domain Name System protocol bind 9.16.23 fixed CVE-2021-25219 by disabling the lame server cache that would have allowed an attacker to significantly degrade resolver performance. There were some patches removed in the blog 2.26 update. Font rendering package freetype2 2.11.1 improved cmake support and updated the latest experimental COLRv1 Application Programming Interface to OpenType standard 1.9. Another rendering package poppler, which is for pdfs, updated to version 21.12.0 and added a few APIs; one to read/save to file descriptor; one to add images; and one to validate signatures. Many incremental improvements and bug fixes were made in the libvirt 7.10.0 update and a new feature is a binary that helps users figure out the format of Distinguished Name from a certificate file the way it expects in the tls_allowed_dn_list option of the libvirtd.conf configuration file. The userspace components for the Linux Kernel’s drivers and infiniband subsystem package rdma-core 38.0 was the only major version update in the snapshot; it updated kernel headers stddef.h. Other packages to update in the snapshot were gc 8.2.0, kImageAnnotator 0.5.3, strace 5.15 and more.

Snapshot 20211205 was pretty much all about the compiler and kernel with the exception on one other package; parsing library package mxml 3.3 fixed a potential memory leak in mxmlLoad functions and added more error handling to the library. The minor update of gcc11 11.2.1 enabled the cross compilers on the i586 microprocessor and removed the cross compilers for the i386 target. The 20211123 kernel-firmware updated the firmware file for Intel Bluetooth and kernel-source 5.15.6 fixed codecs for ALSA System on Chip errors, some discovery of arm firmware and a couple network facets like fixing the bridge port operation related to Marvell hardware.

Most of the updates in snapshot 20211203 were related to YaST. The update of yast2-storage-ng 4.4.20 fixed regressions for unit tests and replicated the generation of bcache issues to avoid setting the architecture for every test. The yast2-installation 4.4.26 package added a display the of a product’s license when only one product is available, but will not display the product’s selector during an upgrade. Virtualization package xen 4.16.0 made fixes to the Trusted Platform Module in preparation for TPM 2.0 support. A major version update of the text shaping library harfbuzz became available in the snapshot; the 3.1.1 version improved Unicode 14 properties in the shaper and provided COLRv1 tables subsetting support, and various other subsetter fixes. Other packages to update were libstorage-ng 4.4.63, scout 0.2.6, rubygem-cheetah 1.0.0 and rubygem-yast-rake 0.2.43.

Starting the week, KDE users could update to Plasma 5.23.4 in snapshot 20211202. Plasma’s software management GUI Discover made some adjustments on handling Flatpak and other software. Plasma’s power consumption package PowerDevil fixed a bug that had a different notification behavior for a critical battery than that of a low battery. Other packages updated in the snapshot were sssd 2.6.1, which hardened systemd services, and automake 1.16.5, which dropped a couple patches and made the output more reproducible.

After a lot of work, Ritchie-CLI 2.11.3 is officially available in the openSUSE Tumbleweed repository. Ritchie-CLI became official Nov. 11 in Tumbleweed, but was not covered in previous blogs. A big congratulations goes out to the entire ZUP team!

Ritchie is an open-source tool developed by the ZUP Company and allows users to create, store and share automations securely. It optimizes repetitive commands so users have more programming autonomy. Ritchie release notes provides an add Rhythm List Formulas command, a forced update option to run the latest formula version when enabled, lib to support Ritchie-CLI internationalization, repository new version detection using cache, and many other features. Alessandro de Oliveira Faria is working to add a new package to Factory and help from the openSUSE community is welcomed. The package is also in Alessandro’s Open Build Service home project for those interested in testing; there is also cloud testing. More information can be found in the package’s release notes.

Deploying software for HPC clusters is often a complex task, as HCP cluster tend to have a fixed software stack. Also precompiled software is not always fully optimized to hardware of the HPC cluster. With this post I want to describe two tools which try to solve these problems. The first one is spack which is build system for mainly HPC applications and their dependencies. With singularity[1] these HPC applications can be packed into containers which can be executed in user space.

Preparations

As well spack as singularity are available as packages in openSUSE Tumbleweed, Leap and via PackageHub for SLE. The packages can be installed with

# sudo zypper install singularity spack

After the installation you should add all the users which want to use singularity to the singularity group, e.g. with

# sudo usermod -a -G singularity <user_login>

Create singularity definition

Now you have to decide which application to build inside the container. For this example we will gromacs with MPI support. So create the file spack.yaml with following content

spack:
  specs:
  - gromacs+mpi
  - openmpi

  container:
    format: singularity
    images:
      os: "leap:15"
      spack: "latest"

    os_packages:
      final:
      - libgomp1

additionally the multi threading support is enable in the final container with the installation of the OpenMP runtime library libgomp1. The definition file for singualrity can then be created with the command

spack containerize > gromacs-mpi.def

Build the container

With this definition the final application container image can now build with the command

singularity build --fakeroot gromacs-mpi.sif gromacs-mpi.def

where the --fakeroot switch to allow building for non root users. The build will have two phases, where the first one will pull in the container spack/leap15:latest which has spack and necessary build tools installed, build gromacs together with openmpi and move the resulting binaries to the container opensuse/leap:latest. The binairis built with spack are located in the container under /opt/view/bin.

Inspect the container

A single command within the container can be run with

singularity exec gromacs-mpi.sif ls /opt/view/bin

which will list all the binaries installed under /opt/view/bin within the container. You can also open a shell in the container with

singularity shell gromacs-mpi.sif

Please note that the home directory of the user is mounted under $HOME in the container.

Run the container

Now you can run the application, which is gromacs in this case, in parallel with

mpirun singularity exec gromacs-mpi.sif gmx_mpi mdrun\
	-s topol_pme.tpr -maxh 0.50 -resethway\
	-noconfout -nsteps 10000 -g gmx_sing.log

Have a lot of fun

Apptainer

[1] In order to avoid confusion with singularity Community Edition from sylabs, singularity will be renamed to apptainer, but the them of this article ‘apptainer’ is not fully stable, yet. [https://linuxfoundation.org/press-release/new-linux-foundation-project-accelerates-collaboration-on-container-systems-between-enterprise-and-high-performance-computing-environments/]

This video demonstrates an image with a holographic effect generated on the openSUSE platform with the realsense camera using the Realsense Video Creator for Looking Glass Portrait project. Source codes are available at: https://github.com/cabelo/rscreatelg

In this video we will see how to work with immersion using the oculus quest 2 of the metaverse. The calculation of the transformation of the relative coordinates of the keyboard in the virtual reality environment is demonstrated. We can also see the hand traceability with skeleton detection algorithms among other fantastic features.

On the road to AppStream 1.0, a lot of items from the long todo list have been done so far – only one major feature is remaining, external release descriptions, which is a tricky one to implement and specify. For AppStream 1.0 it needs to be present or be rejected though, as it would be a major change in how release data is handled in AppStream.

Besides 1.0 preparation work, the recent 0.15 release and the releases before it come with their very own large set of changes, that are worth a look and may be interesting for your application to support. But first, for a change that affects the implementation and not the XML format:

1. Completely rewritten caching code

Keeping all AppStream data in memory is expensive, especially if the data is huge (as on Debian and Ubuntu with their large repositories generated from desktop-entry files as well) and if processes using AppStream are long-running. The latter is more and more the case, not only does GNOME Software run in the background, KDE uses AppStream in KRunner and Phosh will use it too for reading form factor information. Therefore, AppStream via libappstream provides an on-disk cache that is memory-mapped, so data is only consuming RAM if we are actually doing anything with it.

Previously, AppStream used an LMDB-based cache in the background, with indices for fulltext search and other common search operations. This was a very fast solution, but also came with limitations, LMDB’s maximum key size of 511 bytes became a problem quite often, adjusting the maximum database size (since it has to be set at opening time) was annoyingly tricky, and building dedicated indices for each search operation was very inflexible. In addition to that, the caching code was changed multiple times in the past to allow system-wide metadata to be cached per-user, as some distributions didn’t (want to) build a system-wide cache and therefore ran into performance issues when XML was parsed repeatedly for generation of a temporary cache. In addition to all that, the cache was designed around the concept of “one cache for data from all sources”, which meant that we had to rebuild it entirely if just a small aspect changed, like a MetaInfo file being added to /usr/share/metainfo, which was very inefficient.

To shorten a long story, the old caching code was rewritten with the new concepts of caches not necessarily being system-wide and caches existing for more fine-grained groups of files in mind. The new caching code uses Richard Hughes’ excellent libxmlb internally for memory-mapped data storage. Unlike LMDB, libxmlb knows about the XML document model, so queries can be much more powerful and we do not need to build indices manually. The library is also already used by GNOME Software and fwupd for parsing of (refined) AppStream metadata, so it works quite well for that usecase. As a result, search queries via libappstream are now a bit slower (very much depends on the query, roughly 20% on average), but can be mmuch more powerful. The caching code is a lot more robust, which should speed up startup time of applications. And in addition to all of that, the AsPool class has gained a flag to allow it to monitor AppStream source data for changes and refresh the cache fully automatically and transparently in the background.

All software written against the previous version of the libappstream library should continue to work with the new caching code, but to make use of some of the new features, software using it may need adjustments. A lot of methods have been deprecated too now.

2. Experimental compose support

Compiling MetaInfo and other metadata into AppStream collection metadata, extracting icons, language information, refining data and caching media is an involved process. The appstream-generator tool does this very well for data from Linux distribution sources, but the tool is also pretty “heavyweight” with lots of knobs to adjust, an underlying database and a complex algorithm for icon extraction. Embedding it into other tools via anything else but its command-line API is also not easy (due to D’s GC initialization, and because it was never written with that feature in mind). Sometimes a simpler tool is all you need, so the libappstream-compose library as well as appstreamcli compose are being developed at the moment. The library contains building blocks for developing a tool like appstream-generator while the cli tool allows to simply extract metadata from any directory tree, which can be used by e.g. Flatpak. For this to work well, a lot of appstream-generator‘s D code is translated into plain C, so the implementation stays identical but the language changes.

Ultimately, the generator tool will use libappstream-compose for any general data refinement, and only implement things necessary to extract data from the archive of distributions. New applications (e.g. for new bundling systems and other purposes) can then use the same building blocks to implement new data generators similar to appstream-generator with ease, sharing much of the code that would be identical between implementations anyway.

2. Supporting user input controls

Want to advertise that your application supports touch input? Keyboard input? Has support for graphics tablets? Gamepads? Sure, nothing is easier than that with the new control relation item and supports relation kind (since 0.12.11 / 0.15.0, details):

<supports>
  <control>pointing</control>
  <control>keyboard</control>
  <control>touch</control>
  <control>tablet</control>
</supports>

3. Defining minimum display size requirements

Some applications are unusable below a certain window size, so you do not want to display them in a software center that is running on a device with a small screen, like a phone. In order to encode this information in a flexible way, AppStream now contains a display_length relation item to require or recommend a minimum (or maximum) display size that the described GUI application can work with. For example:

<requires>
  <display_length compare="ge">360</display_length>
</requires>

This will make the application require a display length greater or equal to 300 logical pixels. A logical pixel (also device independent pixel) is the amount of pixels that the application can draw in one direction. Since screens, especially phone screens but also screens on a desktop, can be rotated, the display_length value will be checked against the longest edge of a display by default (by explicitly specifying the shorter edge, this can be changed).

This feature is available since 0.13.0, details. See also Tobias Bernard’s blog entry on this topic.

4. Tags

This is a feature that was originally requested for the LVFS/fwupd, but one of the great things about AppStream is that we can take very project-specific ideas and generalize them so something comes out of them that is useful for many. The new tags tag allows people to tag components with an arbitrary namespaced string. This can be useful for project-internal organization of applications, as well as to convey certain additional properties to a software center, e.g. an application could mark itself as “featured” in a specific software center only. Metadata generators may also add their own tags to components to improve organization. AppStream gives no recommendations as to how these tags are to be interpreted except for them being a strictly optional feature. So any meaning is something clients and metadata authors need to negotiate. It therefore is a more specialized usecase of the already existing custom tag, and I expect it to be primarily useful within larger organizations that produce a lot of software components that need sorting. For example:

<tags>
  <tag namespace="lvfs">vendor-2021q1</tag>
  <tag namespace="plasma">featured</tag>
</tags>

This feature is available since 0.15.0, details.

5. MetaInfo Creator changes

The MetaInfo Creator (source) tool is a very simple web application that provides you with a form to fill out and will then generate MetaInfo XML to add to your project after you have answered all of its questions. It is an easy way for developers to add the required metadata without having to read the specification or any guides at all.

Recently, I added support for the new control and display_length tags, resolved a few minor issues and also added a button to instantly copy the generated output to clipboard so people can paste it into their project. If you want to create a new MetaInfo file, this tool is the best way to do it!

The creator tool will also not transfer any data out of your webbrowser, it is strictly a client-side application.

And that is about it for the most notable changes in AppStream land! Of course there is a lot more, additional tags for the LVFS and content rating have been added, lots of bugs have been squashed, the documentation has been refined a lot and the library has gained a lot of new API to make building software centers easier. Still, there is a lot to do and quite a few open feature requests too. Onwards to 1.0!

Dear Tumbleweed users and hackers,

Tumbleweed has been strong and unstoppable: for the 2nd week in a row we have published daily snapshots, i.e. 7 snapshots since the late review. Snapshots were numbered 1125…1201.

The noteworthy changes in those snapshots were:

• PCRE2 10.39
• Mozilla Firefox 94.0.2
• gpg 2.3.3
• virtualbox 6.1.30
• fprintd 1.94.1
• Ruby CVE fixes: ruby 2.7.5 & ruby 3.0.3
• Linux kernel 5.15.5
• bash 5.1.12: /bin/sh is no longer managed using update-alternatives, but comes as part of bash-sh or alternative *-sh packages (currently only busybox-sh as alternative)
• Tumbleweed moved suse_version from 1550 to 1599 to avoid a possible conflict on internal config files of build with the upcming SLE15SP5/Leap 15.5 plans

Changes that are currently being built/tested in upcoming snapshots and stagings:

• KDE Plasma 5.23.4
• Automake 1.16.5
• Mesa 21.3.1
• Poppler 21.11.0
• Linux kernel 5.15.6
• Harfbuzz 3.1.1: NOTE: this has an ABI break over 2.9.x, but Leap/SLE are moving as well and biting the bullet, and other distros do too
• GCC 11: Enable the full cross compiler, cross-aarch64-gcc11 and cross-riscv64-gcc11 now provide a fully hosted C (and C++) cross compiler, not just a freestanding one
• cmake 3.22.0
• Moving default php version from php7 to php8
• Testing the results when moving system ruby from 2.7 to 3.0
• pipewire 0.3.40, with a move to from pipewire-media-session to wireplumber; currently failing openQA
• openSSL 3.0

November provided a robust month of openSUSE Tumbleweed snapshots, which included 21 releases from Nov. 1 to Nov. 29.

December, which is traditionally a slower month for Tumbleweed releases due to the holiday season, has already produced a snapshot. Snapshot 20211201 gave a major update of the Linux user-space application for modifying Intel’s Extensible Firmware Interface (EFI) Boot Manager. The efibootmgr package updated from version 14 to 17; the changes included fixes for GNU Compiler Collection 7, better parsing and now efibootmgr uses EFIDIR / efibootmgr.efidir like fwupdate had. Scrolling issues when pressing Home and Page Down keys were fixed with the webkit2gtk3 2.34.2 update. Four patches for bash were added in the 5.1.12 version, which fixes a couple trapped signals. The 2.34.1 git version fixed an issue that arose from the 20211125 snapshot; git grep that have a non-UTF8 payload were broke when linked with certain versions of pcre2’s latest release. Other packages to update in the month’s first snapshot were glslang 11.7.1, graphviz 2.49.3, libstorage-ng 4.4.61, mtools 4.0.36 and yast2-update 4.4.5.

Snapshot 20211129 provided an update of the 5.15.5 Linux Kernel, which had some arm fixes for Broadcom’s StrataGX communications processor. Tumbleweed started the month off with the 5.14 kernel. An update of iso-codes 4.8.0 added flag emojis to countries and a new translation for Chinese. LibreOffice also had some translations with the 7.2.3.2 update. Image viewer ristretto 0.12.1 fixed pointer behavior in fullscreen mode as well as a fix for a memory leak when closing the window directly. Other packages to update in the snapshot were Microsoft’s theorem prover z3 4.8.13, libsoup 3.0.3, libsoup2 2.74.2, libwpe 1.12.0 and more.

Updates for both ruby2.7 2.7.5 and ruby3.0 3.0.3 were made in snapshot 20211128. The new Ruby versions fixed the same three Common Vulnerabilities and Exposures; the CVE-2021-41817 security fix could have allowed an attacker to exploit the vulnerability to cause an effective Denial-of-service attack. CVE-2021-41819 and CVE-2021-41816 were the other two vulnerabilities that were fixed in the Ruby updates. The openvpn package updated to version 2.5.4 and had improvements for documentation and man page building. The package also fixed a minor memory leak under certain conditions related to adding routes to the system. A few YaST packages were updated in the snapshot like autoyast2 4.4.23, yast2-installation 4.4.23 and yast2-bootloader 4.4.9, which fixed a crash due to a missing requirement.

Two packages were updated in snapshot 20211127. The fingerprint reader package fprintd 1.94.1 fixed the systemd unit so that udev hotplug events are processed. The fingerprint package also changed PolicyKit strings and had plenty of updated translations thanks to Fedora Weblate contributions. Linux Bluetooth configuration tool blueberry 1.4.5 also had some updated translations and added a turn bluetooth on/off option.

Internet browser Mozilla Firefox 94.0.2 had a Linux specific fix in snapshot 20211126; the fix resolved general instability and crashes caused by a file descriptor leak when moving tabs to the background using WebGL. The update of ImageMagick 7.1.0.16 added support for reading extra channels in a Photoshop Document file. The graphics editing package also fixed an OpenCL build problem. GNU Privacy Guard gpg2 was updated to version 2.3.3 and fixed the Let’s Encrypt certificate chain validation. The update also provided a new configuration file common.conf, which is now used to enable the use of the key database daemon with use-keyboxd; using this option in gpg.conf and gpgsm.conf is supported for a transitional period. An update of virtualbox 6.1.30 fixed a bug on X11 involving drag and drop initiated on a single mouse click in VM storage settings; the package also began to introduce initial support for kernel 5.16. Other packages to update in the snapshot were xapps 2.2.5. newer versions of ncurses and ceph, and Catalan translations were made in the libstorage-ng 4.4.58 update.

Only one package was update in snapshot 20211125. The Perl-Compatible Regular Expressions library is a set of C functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. The pcre2 10.39 version fixed some incorrect detection involving its just-in-time (JIT) compiler and made an update of Unicode 14.0.0 support. Changes in this snapshot recevied fixes in snapshot 20211201.

In this encore to the Run Azure images in Hyper-V or kvm will briefly discuss, if it is possible to create your own Hyper-V Windows Hypervisor as a kvm guest on openSUSE Tumbleweed.