There are times, when keeping your system up-to date does not help you against vulnerabilities. During these times, you want to have your servers and applications hardened as good as possible - including good Apparmor profiles. But even then, something bad can easily happen - and it's very good to see that others take care. Especially if these others are professionals, that take care for you, even if you did not ask them directly.

Tuesday, 2021-08-31, was such a day for our openSUSE infrastructure status page: SonarSource reported to us a pre-auth remote code execution at the https://status.opensuse.org/api/v1/incidents endpoint.

SonarSource, equally driven by studying and understanding real-world vulnerabilities, is trying to help the open-source community to secure their projects. They disclosed vulnerabilities in the open-source status page software Cachet - and informed us directly - that our running version is vulnerable to CVE-2021-39165. Turned out that the Cachet upstream project is meanwhile seen as dead - at least it went out of support by their original maintainers since a while. It went into this unsupported state unnoticed by us - and potentially also unnoticed by many others. A problem, that many other, dead open source projects sadly share.

Thankfully, the openSUSE Security team (well known as first contact for security issues) as well as Christian (as one of our glorious openSUSE heroes) reacted quick and professional:

• SonarSource informed our Security team 2021-08-31, 15:39
• Our Security team opened a ticket for us just two hours later, at 2021-08-31, 17:08
• Already one hour later, at 2021-08-31 18:29, Christian deployed a first hot-fix on our instances (Note: the original admin of the systems was on vacation)
• 2021-08-31 at 23:35, Christian already provided a collection of suspicious requests to the affected URL
• Meanwhile, there was a fix provided in a forked Github repository, which was applied to our installations one day later, 2021-09-01. This made our installations secure again (cross-checked by our Security Team and SonarSource). A response time of one day, even if the original upstream of a project is not available any longer - and the original admin of a system is on vacation! :-)
• ...and we started a long analysis of the "what" and "when"...
• In the end, we identified 6 requests from one suspicious IP, which we couldn't assign to someone we know. So we decided to distrust our installations. There might be a successful attack, even if we could not find any further evidence on the installed system (maybe thanks to the Apparmor profile?) or in the database. BUT: an attacker could have extracted user account data.
• The user accounts of the Cachet application are only used to inform our users about any infrastructure incident. An attacker might be able to log in and report fake incidents - or send out Emails to those, who subscribed to incident reports or updates. Something we don't like to see. Luckily, these accounts are in no way connected to the normal user accounts. They just existed on these systems, for exactly one purpose: informing our users.
• As result, we informed all users of the status.opensuse.org instances that they should change their password on a new system, setup from scratch. This new system is now deployed and in production, while the image of the old system is still available for further investigation.

Big kudos to Thomas Chauchefoin (SonarSource), Gianluca Gabrielli and Marcus Meissner (openSUSE Security Team) and Christian Boltz (openSUSE Heroes) for all their work, their good cooperation and quick reactions!

Our Horde docker images have switched over from Tumbleweed to openSUSE LEAP once again.

Recently our container build CI job in github.com broke down unexpectedly. An investigation showed that Tumbleweed’s core libraries, especially libc, were too new for the CI’s build system, based on Ubuntu LTS.

This is the second time we abandoned the Tumbleweed basis for Horde docker containers. OpenSUSE Leap 15.3 uses a relatively old, but well-maintained, set of base libraries. Both Leap and Tumbleweed deliver PHP 7.4 as a basis for Horde. In both systems, we skip the packaged composer version for a static pick which we will update from time to time. We may switch over to packaged composer if we feel confident.

For users and administrators of the image, both Tumbleweed and Leap 15.3 should feel more or less the same. For end users of the delivered horde setup, there should not be any downsides. We will switch back to the Tumbleweed image in a while when we have picked a more recent version of Ubuntu.

Dear Tumbleweed users and hackers,

After the massive update in the last week due to a full rebuild caused by glibc 2.34, this week seems ‘somewhat’ quieter. Or at least from a Release manager PoV less involvement hungry. Yet, we managed to release 5 snapshots during this week (0923, 0924, 0926, 0927, and 0928).

The most notable changes in those snapshots included:

• Linux kernel 5.14.6
• Mesa 21.2.2
• Mozilla Thunderbird 91.1.1
• Perl 5.34.0: some perl modules could not yet successfully be rebuilt)
• Boost 1.77.0

Matches very much what was announced last week, which is a good sign. Staging projects and builds currently contain:

• GNOME 41.0 (Snapshot 0929+)
• Linux kernel 5.14.8 (Snapshot 0930+)
• Mesa 21.2.3
• Switch to Rust 1.55
• Meson 0.59.2
• KDE Plasma 5.23 (once released)
• RPM 4.17
• gpg 2.3
• openssl 3.0

GNOME 41 has reached openSUSE Factory staging and KDE’s Plasma 5.23 is nearing a release in an openSUSE Tumbleweed snapshot as it progresses through staging.

openSUSE’s rolling release turned out four snapshots this week and updated software packages like Mesa, curl, catfish, PipeWire, Perl and more.

The 20210928 snapshot improved the transferring of data via an update of curl 7.79.1, which made it work with OpenSSH 8.7; the command line tool and library also adjusted a setup to not change connection data upon repeat invokes. An update of inkscape 1.1.1 fixed a crash and improved the startup time of the graphics editor application. Two other packages updated in the snapshot were yast2-network 4.4.26 and yast2-nfs-client 4.4.1; the latter had an update that supports systemd mount options in fstab.

With the 4.16.3 update of the file searching tool catfish, better icon updating and a sidebar background color fix was made in snapshot 20210927. Intel’s dleyna-renderer package updated to 0.7.1; this package, which allows clients to discover and manipulate Digital Media Renderers, provided a build fix and a port to the GUPnP 1.2 object-oriented framework. The desktop environment package menulibre 2.2.3 fixed making diagnostic text selectable on KDE and added support for GNOME-specific categories.

Mozilla Thunderbird 91.1.1 was released in snapshot 20210926. A menu item for disabling subject encryption for a single message was added and the printing of email messages not currently being displayed is no longer supported, which includes printing multiple messages at once. The dnsmasq 2.86 version fixed a bug that caused dnsmasq to lose track of processes forked to handle Transmission Control Protocol Domain Name System connections under heavy loads. Perl 5.34.0 added a patch to fix a build with gdbm 1.20 and fixed a memory leak in regular expression. The rendering of fullscreen zoom was optimized with the gnome-shell 40.5 update, and pipewire 0.3.37 provided some JACK stability improvements with buffer-size and sample rate changes in some apps. Other updates in the snapshot included php7 7.4.24, mousepad 0.5.7, a version bump to ceph, mutter 40.5, python-numpy 1.21.2 and a few updates to YaST packages.

The 20210924 snapshot brought an update of Mesa 21.2.2; the 3D graphics package provides some bug fixes and a ton of work went into getting panfrost closer to being conformant. The creation suite ImageMagick 7.1.0.8 fixed some color conversions and added an option to read a thumbnail of a raw Image and store it as a profile called dng:thumbnail. Chat client pidgin 2.14.7 removed some obsolete translations and fixed a couple leaks. Other packages to update in the snapshot were bubblewrap 0.5.0, webkit2gtk3 2.32.4, gnome-control-center 40.1 and libstorage-ng 4.4.37.

I have just renamed librsvg's master branch to main, as other modules already have.

This is what I did:

• Rename the local branch, and push it:

git branch -m master main
git push origin main

• Change the default branch in gitlab; for librsvg this is in the repository settings / Default branch - change that to main.

• Set the same protection for the main branch as there was for master, if any - repository settings / Protected branches -> create a new protection and copy the settings from master.

• Unprotect the master branch so I can delete it - repository settings / Protected branches -> unprotect the master branch.

• Delete the master branch in the branches list.

• Update the CI and build scripts: for librsvg it was just .gitlab-ci.yml.

• Update your documentation: for librsvg it was just COMPILING.md.

• Notify the release team; I created an issue and one for gnome-build-meta.

Update 2021/Sep/30: Extra things which Philip Withnall suggested:

• Notify gnome-i18n@gnome.org so they can update the damned-lies software. (Librsvg has no translations).

• See if any project have a librsvg.wrap (for Meson?) and notify them. I don't think I'm using search engines correctly...

• Re-protect the master branch to prevent people accidentally pushing to it. Since that branch no longer exists, gitlab lets you create a protection by glob matching on a name, not an actual branch.

Update 2022/Jul/06: Some extra configuration to update:

• If your project has gitlab badges, verify that they point to the main branch in settings/general/badges. You can use %{default_branch} as part of the badge URLs to avoid hardcoding names.

If you have a local checkout, you can do this:

# update from upstream
git fetch origin

# switch to your local master branch
git checkout master

# rename your local branch
git branch -m master main

# Remove the old upstream...
git branch --unset-upstream

# ... and set the new one
git branch -u origin/main

That's all!

When I started my career, I was lucky to work on a legendary Operating System named Novell Netware. I got my first job because of a hacking adventure that me and my roommate Arvind did in our college on top of an upatched Netware installation. The reliability and robustness of Netware may make you believe in magic. But it was just a well-engineered, old-school product. One of the most popular instances of its robustness, was the epic uptime of 16 years as covered in arstechnica.

(Image courtesy: Arstechnica)

This was not an one-off situation either. We had multiple customers with years of uptime. In one of the academic institutes, the uptime was well into decades, that multiple sysadmins changed, but the netware box tirelessly worked on. At some point of time, nobody knew where the server was physically located, as nobody looked at it as everything worked fine.

In almost all the cases, the hardware failed before the software. The software was engineered so well that it would have run forever on superior hardware (albeit not so efficiently capable of using the modern hardware in its true potential). Those days, even the hardware was built to last for decades. It was the good old times before the planned obsolescence. 

Fast forward to today, 2021. I have a Redmi 4 android phone, built by the mass manufacturer Xiaomi. I bought it on May 30th 2017 and still use it everyday. I always purchase things for long-term. I believe in BuyItForLife principles. I maintain my hardware properly (Fully discharge and then recharge, handle with care etc.). Even my prior phone, a Motorola E398 lasted me a about a decade, before the charger gave up.

Today during the lunch break, I went out looking in search of a home. My ever busy teammate Seshachalam sent a message to me on Slack at that time. I got a notification in the Android pull-down notifications. I tried to open Slack to see the message and got this error message:

Whenever I talked to people about POWER, someone asked if I am an IBM Power Champion. My response was that I do not even know what it is, and I am not affiliated with IBM in any way. Recently I came across a blog by Torbjörn Appehl which describes what is an IBM Power Champion and lists the European champions: https://builtonpower.com/2021/09/the-2021-ibm-power-champions-in-europe/.

Finally I know what an IBM Power Champion is, and I feel honored to be mistaken to be one of them :-) Normally I do not care much about titles: I have seen too many empty people with well sounding titles, and fantastic people without any titles. Even with this background I’d be proud to wear the IBM Power Champion badge. However, being a loud POWER advocate does not mean that I feel that I am active enough to warrant this badge.

I must admit, that knowing what an IBM Power Champion is, I am not surprised at all, that I was mistaken for an IBM Power Champion. I am a long time POWER user. Started with RS6000 boxes almost 30 years ago. I helped to install the largest POWER server in Hungary at the turn of the century. I supported Linux on the Genesi Pegasos, a PowerPC workstation, for many years. I was an active contributor and moderator on the power-developer forums and on power.org. And recently I support syslog-ng on POWER. POWER9 provided the best syslog-ng performance for years, and I have a strong suspicion that after a short break the release of POWER10 gives back the performance crown to the POWER architecture. Tead the article I wrote based on my OpenPOWER conference talk last year to see my history in detail and that I am not that active recently: I’m a POWER user

So why do people have the impression that I actively work on POWER technologies? I guess it’s because of my job. If I am enthusiastic about a technology, I talk about it loud and clear. Even if it is not part of my job. And my enthusiasm is contagious. I am a technology evangelist, and by definition it means that I advocate technologies and help them in many possible ways. For my job I work with sudo and syslog-ng, however if I like something, it receives the same treatment – in my free time. You can learn more about being an open source evangelist from my article on opensource.com: What is an open source evangelist?

Of course, I have plans related to POWER, even if I’m not too active. I’d love to test syslog-ng on a POWER10 server. However I’m patient. No matter how much I love syslog-ng, an IBM Power E1080 server is an overkill for syslog-ng both in price and performance. Especially that syslog-ng has an upper limit of 64 threads, and this server has more cores than that :-) But once POWER10 is more widespread and there are smaller boxes available as well, I’d love to verify my assumption, that POWER10 is the currently available best CPU for syslog-ng :-)

And what can I say to POWER and POWER users? Live long and prosper!

This is our third blog post since summer started in Europe and also the third time in a row we write a combined blog post for two development sprints. And it’s also the third consecutive report focused on improvements to the existing codebase rather than on new shiny features. That covers:

• Rethinking some aspects of the management of software, keyboard layouts and NTP configuration
• Heavy work in the internals of yast2-users
• Improvements in the way YaST handles libYUI plugins
• Visual fixes in the packages manager
• More progress in the release tools

Diving into our own history

After 22 years of constant development of the same codebase, it’s not surprising that some parts of YaST2 are a bit intricate. As you know, we are lately investing time in an attempt to shed some light on some of those internals. During these latest sprints it was time for checking the configuration of the NTP client, the behavior of the keyboard layouts, the way we manage software and the administration of users and groups in already installed systems.

We don’t have much finished stuff to present yet, but we expect important changes in those four areas in the future. Regarding the management of users, we hope to report big improvements in the next blog post. Changes in the other three areas are more targeted to the mid-term, but we will keep steadily working on it.

Graceful Handling of Missing libYUI Components

And talking about YaST history and software that is still evolving after more than twenty years, we cannot forget about libYUI - the cornerstone that allows YaST to run in both graphical and text modes. LibYUI is a modular library that offers several backends (like Qt or ncurses) and also a plugins system to implement advanced widgets. So the user can install only one of the given backends or just a subset of all the available widgets. But such flexibility can be a double-edged sword.

During this sprint, we improved how YaST handles the situation in which a missing combination of plugin and backend is needed to perform the action requested by the user. The description of this pull request offers a detailed overview of the situation and the implemented solution, including screenshots.

Where is my header?

Did we mention flexibility comes with a price? :wink: Just multiply the different ways in which YaST can be used by all the architectures and environments supported by the (open)SUSE distributions and you’ll get a glimpse of all the things that can go wrong.

Recently we got a report about YaST not displaying the names of the categories in the list that shows the patches that can be installed in the system. It took us weeks to reproduce the error because it worked just fine in all the systems we tried, including virtual machines and bare metal. But turns out there was indeed in problem… visible only when executing YaST in a KVM virtual machine with a Plasma desktop.

But we finally caught the bug and you can see a screenshot and the corresponding fix in the description of this pull request.

Release Tools: Polishing

As our readers already know, the YaST Team is also trying to help with the maintenance and evolution of the (open)SUSE release tools. And nothing makes the source code more maintainable than getting rid of it. So after some months of discussions, we managed to identify and drop eight obsolete tools.

We also keep improving the documentation, not only for the tools in the repository but also updating and extending the information available in the openSUSE wiki about the openSUSE development process. Last but not least, we improved the automated tests for the tools, hopefully making them more understandable for newcomers in the process.

More news to come

Hopefully the next report will contain more details about finished work and actually released features. Meanwhile, we promise to keep working if you all promise to keep having fun!