On the road to AppStream 1.0, a lot of items from the long todo list have been done so far – only one major feature is remaining, external release descriptions, which is a tricky one to implement and specify. For AppStream 1.0 it needs to be present or be rejected though, as it would be a major change in how release data is handled in AppStream.

Besides 1.0 preparation work, the recent 0.15 release and the releases before it come with their very own large set of changes, that are worth a look and may be interesting for your application to support. But first, for a change that affects the implementation and not the XML format:

1. Completely rewritten caching code

Keeping all AppStream data in memory is expensive, especially if the data is huge (as on Debian and Ubuntu with their large repositories generated from desktop-entry files as well) and if processes using AppStream are long-running. The latter is more and more the case, not only does GNOME Software run in the background, KDE uses AppStream in KRunner and Phosh will use it too for reading form factor information. Therefore, AppStream via libappstream provides an on-disk cache that is memory-mapped, so data is only consuming RAM if we are actually doing anything with it.

Previously, AppStream used an LMDB-based cache in the background, with indices for fulltext search and other common search operations. This was a very fast solution, but also came with limitations, LMDB’s maximum key size of 511 bytes became a problem quite often, adjusting the maximum database size (since it has to be set at opening time) was annoyingly tricky, and building dedicated indices for each search operation was very inflexible. In addition to that, the caching code was changed multiple times in the past to allow system-wide metadata to be cached per-user, as some distributions didn’t (want to) build a system-wide cache and therefore ran into performance issues when XML was parsed repeatedly for generation of a temporary cache. In addition to all that, the cache was designed around the concept of “one cache for data from all sources”, which meant that we had to rebuild it entirely if just a small aspect changed, like a MetaInfo file being added to /usr/share/metainfo, which was very inefficient.

To shorten a long story, the old caching code was rewritten with the new concepts of caches not necessarily being system-wide and caches existing for more fine-grained groups of files in mind. The new caching code uses Richard Hughes’ excellent libxmlb internally for memory-mapped data storage. Unlike LMDB, libxmlb knows about the XML document model, so queries can be much more powerful and we do not need to build indices manually. The library is also already used by GNOME Software and fwupd for parsing of (refined) AppStream metadata, so it works quite well for that usecase. As a result, search queries via libappstream are now a bit slower (very much depends on the query, roughly 20% on average), but can be mmuch more powerful. The caching code is a lot more robust, which should speed up startup time of applications. And in addition to all of that, the AsPool class has gained a flag to allow it to monitor AppStream source data for changes and refresh the cache fully automatically and transparently in the background.

All software written against the previous version of the libappstream library should continue to work with the new caching code, but to make use of some of the new features, software using it may need adjustments. A lot of methods have been deprecated too now.

2. Experimental compose support

Compiling MetaInfo and other metadata into AppStream collection metadata, extracting icons, language information, refining data and caching media is an involved process. The appstream-generator tool does this very well for data from Linux distribution sources, but the tool is also pretty “heavyweight” with lots of knobs to adjust, an underlying database and a complex algorithm for icon extraction. Embedding it into other tools via anything else but its command-line API is also not easy (due to D’s GC initialization, and because it was never written with that feature in mind). Sometimes a simpler tool is all you need, so the libappstream-compose library as well as appstreamcli compose are being developed at the moment. The library contains building blocks for developing a tool like appstream-generator while the cli tool allows to simply extract metadata from any directory tree, which can be used by e.g. Flatpak. For this to work well, a lot of appstream-generator‘s D code is translated into plain C, so the implementation stays identical but the language changes.

Ultimately, the generator tool will use libappstream-compose for any general data refinement, and only implement things necessary to extract data from the archive of distributions. New applications (e.g. for new bundling systems and other purposes) can then use the same building blocks to implement new data generators similar to appstream-generator with ease, sharing much of the code that would be identical between implementations anyway.

2. Supporting user input controls

Want to advertise that your application supports touch input? Keyboard input? Has support for graphics tablets? Gamepads? Sure, nothing is easier than that with the new control relation item and supports relation kind (since 0.12.11 / 0.15.0, details):

<supports>
  <control>pointing</control>
  <control>keyboard</control>
  <control>touch</control>
  <control>tablet</control>
</supports>

3. Defining minimum display size requirements

Some applications are unusable below a certain window size, so you do not want to display them in a software center that is running on a device with a small screen, like a phone. In order to encode this information in a flexible way, AppStream now contains a display_length relation item to require or recommend a minimum (or maximum) display size that the described GUI application can work with. For example:

<requires>
  <display_length compare="ge">360</display_length>
</requires>

This will make the application require a display length greater or equal to 300 logical pixels. A logical pixel (also device independent pixel) is the amount of pixels that the application can draw in one direction. Since screens, especially phone screens but also screens on a desktop, can be rotated, the display_length value will be checked against the longest edge of a display by default (by explicitly specifying the shorter edge, this can be changed).

This feature is available since 0.13.0, details. See also Tobias Bernard’s blog entry on this topic.

4. Tags

This is a feature that was originally requested for the LVFS/fwupd, but one of the great things about AppStream is that we can take very project-specific ideas and generalize them so something comes out of them that is useful for many. The new tags tag allows people to tag components with an arbitrary namespaced string. This can be useful for project-internal organization of applications, as well as to convey certain additional properties to a software center, e.g. an application could mark itself as “featured” in a specific software center only. Metadata generators may also add their own tags to components to improve organization. AppStream gives no recommendations as to how these tags are to be interpreted except for them being a strictly optional feature. So any meaning is something clients and metadata authors need to negotiate. It therefore is a more specialized usecase of the already existing custom tag, and I expect it to be primarily useful within larger organizations that produce a lot of software components that need sorting. For example:

<tags>
  <tag namespace="lvfs">vendor-2021q1</tag>
  <tag namespace="plasma">featured</tag>
</tags>

This feature is available since 0.15.0, details.

5. MetaInfo Creator changes

The MetaInfo Creator (source) tool is a very simple web application that provides you with a form to fill out and will then generate MetaInfo XML to add to your project after you have answered all of its questions. It is an easy way for developers to add the required metadata without having to read the specification or any guides at all.

Recently, I added support for the new control and display_length tags, resolved a few minor issues and also added a button to instantly copy the generated output to clipboard so people can paste it into their project. If you want to create a new MetaInfo file, this tool is the best way to do it!

The creator tool will also not transfer any data out of your webbrowser, it is strictly a client-side application.

And that is about it for the most notable changes in AppStream land! Of course there is a lot more, additional tags for the LVFS and content rating have been added, lots of bugs have been squashed, the documentation has been refined a lot and the library has gained a lot of new API to make building software centers easier. Still, there is a lot to do and quite a few open feature requests too. Onwards to 1.0!

Dear Tumbleweed users and hackers,

Tumbleweed has been strong and unstoppable: for the 2nd week in a row we have published daily snapshots, i.e. 7 snapshots since the late review. Snapshots were numbered 1125…1201.

The noteworthy changes in those snapshots were:

• PCRE2 10.39
• Mozilla Firefox 94.0.2
• gpg 2.3.3
• virtualbox 6.1.30
• fprintd 1.94.1
• Ruby CVE fixes: ruby 2.7.5 & ruby 3.0.3
• Linux kernel 5.15.5
• bash 5.1.12: /bin/sh is no longer managed using update-alternatives, but comes as part of bash-sh or alternative *-sh packages (currently only busybox-sh as alternative)
• Tumbleweed moved suse_version from 1550 to 1599 to avoid a possible conflict on internal config files of build with the upcming SLE15SP5/Leap 15.5 plans

Changes that are currently being built/tested in upcoming snapshots and stagings:

• KDE Plasma 5.23.4
• Automake 1.16.5
• Mesa 21.3.1
• Poppler 21.11.0
• Linux kernel 5.15.6
• Harfbuzz 3.1.1: NOTE: this has an ABI break over 2.9.x, but Leap/SLE are moving as well and biting the bullet, and other distros do too
• GCC 11: Enable the full cross compiler, cross-aarch64-gcc11 and cross-riscv64-gcc11 now provide a fully hosted C (and C++) cross compiler, not just a freestanding one
• cmake 3.22.0
• Moving default php version from php7 to php8
• Testing the results when moving system ruby from 2.7 to 3.0
• pipewire 0.3.40, with a move to from pipewire-media-session to wireplumber; currently failing openQA
• openSSL 3.0

November provided a robust month of openSUSE Tumbleweed snapshots, which included 21 releases from Nov. 1 to Nov. 29.

December, which is traditionally a slower month for Tumbleweed releases due to the holiday season, has already produced a snapshot. Snapshot 20211201 gave a major update of the Linux user-space application for modifying Intel’s Extensible Firmware Interface (EFI) Boot Manager. The efibootmgr package updated from version 14 to 17; the changes included fixes for GNU Compiler Collection 7, better parsing and now efibootmgr uses EFIDIR / efibootmgr.efidir like fwupdate had. Scrolling issues when pressing Home and Page Down keys were fixed with the webkit2gtk3 2.34.2 update. Four patches for bash were added in the 5.1.12 version, which fixes a couple trapped signals. The 2.34.1 git version fixed an issue that arose from the 20211125 snapshot; git grep that have a non-UTF8 payload were broke when linked with certain versions of pcre2’s latest release. Other packages to update in the month’s first snapshot were glslang 11.7.1, graphviz 2.49.3, libstorage-ng 4.4.61, mtools 4.0.36 and yast2-update 4.4.5.

Snapshot 20211129 provided an update of the 5.15.5 Linux Kernel, which had some arm fixes for Broadcom’s StrataGX communications processor. Tumbleweed started the month off with the 5.14 kernel. An update of iso-codes 4.8.0 added flag emojis to countries and a new translation for Chinese. LibreOffice also had some translations with the 7.2.3.2 update. Image viewer ristretto 0.12.1 fixed pointer behavior in fullscreen mode as well as a fix for a memory leak when closing the window directly. Other packages to update in the snapshot were Microsoft’s theorem prover z3 4.8.13, libsoup 3.0.3, libsoup2 2.74.2, libwpe 1.12.0 and more.

Updates for both ruby2.7 2.7.5 and ruby3.0 3.0.3 were made in snapshot 20211128. The new Ruby versions fixed the same three Common Vulnerabilities and Exposures; the CVE-2021-41817 security fix could have allowed an attacker to exploit the vulnerability to cause an effective Denial-of-service attack. CVE-2021-41819 and CVE-2021-41816 were the other two vulnerabilities that were fixed in the Ruby updates. The openvpn package updated to version 2.5.4 and had improvements for documentation and man page building. The package also fixed a minor memory leak under certain conditions related to adding routes to the system. A few YaST packages were updated in the snapshot like autoyast2 4.4.23, yast2-installation 4.4.23 and yast2-bootloader 4.4.9, which fixed a crash due to a missing requirement.

Two packages were updated in snapshot 20211127. The fingerprint reader package fprintd 1.94.1 fixed the systemd unit so that udev hotplug events are processed. The fingerprint package also changed PolicyKit strings and had plenty of updated translations thanks to Fedora Weblate contributions. Linux Bluetooth configuration tool blueberry 1.4.5 also had some updated translations and added a turn bluetooth on/off option.

Internet browser Mozilla Firefox 94.0.2 had a Linux specific fix in snapshot 20211126; the fix resolved general instability and crashes caused by a file descriptor leak when moving tabs to the background using WebGL. The update of ImageMagick 7.1.0.16 added support for reading extra channels in a Photoshop Document file. The graphics editing package also fixed an OpenCL build problem. GNU Privacy Guard gpg2 was updated to version 2.3.3 and fixed the Let’s Encrypt certificate chain validation. The update also provided a new configuration file common.conf, which is now used to enable the use of the key database daemon with use-keyboxd; using this option in gpg.conf and gpgsm.conf is supported for a transitional period. An update of virtualbox 6.1.30 fixed a bug on X11 involving drag and drop initiated on a single mouse click in VM storage settings; the package also began to introduce initial support for kernel 5.16. Other packages to update in the snapshot were xapps 2.2.5. newer versions of ncurses and ceph, and Catalan translations were made in the libstorage-ng 4.4.58 update.

Only one package was update in snapshot 20211125. The Perl-Compatible Regular Expressions library is a set of C functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. The pcre2 10.39 version fixed some incorrect detection involving its just-in-time (JIT) compiler and made an update of Unicode 14.0.0 support. Changes in this snapshot recevied fixes in snapshot 20211201.

In this encore to the Run Azure images in Hyper-V or kvm will briefly discuss, if it is possible to create your own Hyper-V Windows Hypervisor as a kvm guest on openSUSE Tumbleweed.

When I like a song and learn that it is actually a soundtrack of a movie, I usually look it up on IMDB. Often it belongs to a romantic movie, a super hero movie from Marvel or a TV show. In these cases I do not look any further. But sometimes I get curious while reading the plot or watching the trailer. I’ve found many good movies based on the soundtrack. And of course many others, which I could not watch to the end. Here I list a few examples from both categories.

Beyond the Black Rainbow

The trigger of writing this blog was “Beyond the Black Rainbow”. I first listened to its music on TIDAL in its “My Daily Discovery” playlist. I’ve listened to the whole album a couple of times already when I discovered that it’s actually a soundtrack. It was tagged as horror, but I liked the music so much, that I started watching the film. I never finished it, not even on a second try. It is sluggish and boring, and I fell asleep both times. But I keep listening to the music: synth music with lots of strong basses, nice melodies.

• IMDB: https://www.imdb.com/title/tt1534085/
• TIDAL: https://listen.tidal.com/album/33175732

Morgan

Talking about horror movies. There is also a positive example: Morgan by Max Richter. I fell in love with Max Richter’s music when I listened to his version of The Four Seasons. I checked his other works and found the album Morgan. I liked it and it turned out that it’s a soundtrack. First I was scared that it’s a horror movie, but in the end it was less scary and more thought provoking than an average Harry Potter movie :-)

• TIDAL: https://listen.tidal.com/album/137356356
• IMDB: https://www.imdb.com/title/tt4520364/

Six Days, Six Nights

One of my all-time favorite albums is Six days, Six Nights by Michael Nyman. I do not like listening to songs in loop, but sometimes I listened to A La Folie… five times in a row. I’ve tried to watch the movie, but I did not like it. Often the soundtrack mirrors the story of the movie. Not here. The music is often bright and happy, while the movie is just the opposite. After so many years I am not sure if I ever finished watching it.

• TIDAL: https://listen.tidal.com/album/1475665
• IMDB: https://www.imdb.com/title/tt0111856/

Blade Runner

When I started collecting CDs from Vangelis, I was not yet aware, that many of those were movie soundtracks. I do not like watching sci-fi movies. They limit imagination. As you could guess, I prefer books. Imagine my face, when I learned that Blade Runner is actually a soundtrack of a sci-fi movie :-) However I was curious. I watched it, a couple of different versions. I loved all of them. In the end I even read the original novel and a couple more books from Bradbury.

• IMDB: https://www.imdb.com/title/tt0083658/
• https://listen.tidal.com/album/198197/

These are just the tip of the iceberg (hello Antarctica – another soundtrack by Vangelis). There are a lot more examples, on both sides. Did you ever watch a movie because you already knew the soundtrack? Reach out to me on LinkedIn or Twitter (links in the upper right corner).

Alpha releases of openSUSE Leap 15.4 are now available for download on get.opensuse.org. The fourth minor release of Leap 15 has entered its alpha development stage.

During the Alpha phase, regular Alpha images will be built on a rolling basis until mid-February when the point release is scheduled to transition to a Beta build phase. The beta submission deadline is February 16, according to the roadmap. The Beta phase has a similar model until the General Availability of the release. The rolling builds stop after the Beta phase is complete and Leap transitions into a maintenance and security update phase upon beoming public available.

Distro hoppers, hobbyists, users and tech enthusiasts are encouraged to download the current builds and help test the releases to make sure bugs are taking care of in the alpha and beta phases. People interested in testing it to make sure things are working can click on the “Learn more” button to find the downloads.

The Public Availability of Leap 15.4 is scheduled to be released in June, 2022, according to the release’s roadmap. Users of Leap 15.3 will need to update to the newest version within six months of the Public Availability of Leap 15.4.

Users of openSUSE Leap 15.2 should update this month before it reaches its End of Live; users should update to Leap 15.3 if they have not already done so to make sure they continue to receive maintenance and security updates.

Disclaimer: This blog post is about SUSE Linux Enterprise Server images on Azure and probably only useful to SUSE Employees who work with such images.

Each new MacOS release brings some surprises when it comes to compiling syslog-ng. Just a couple of months ago, I provided you with a couple of pointers on how to compile syslog-ng on MacOS. Since then, MacOS Monterey was released and Homebrew was updated. So, here are some updated instructions for MacOS Monterey.

You can read my blog at https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-monterey

Version 1.9 of sudo was released almost two years ago. One of the major new features was support for Python plugins. Previously, you could only extend sudo by coding in C to better suit your environment, which is not the easiest task to manage. Python makes both coding and distributing the results easier. Starting with Safeguard for Sudo 7.2, Python support is also available in a commercial sudo management solution.

You can read my complete blog at https://blog.sudo.ws/posts/2021/11/python-support-arrives-in-safeguard-for-sudo/