Wed, Aug 28th, 2024

Tumbleweed Faces Regression with Wicked as Network Stack

With the switch to dbus-broker as D-Bus implementation in Tumbleweed Snapshot 20240825, a regression was introduced: When using Wicked for network configuration, the system boots up without network connectivity. Systems that use NetworkManager are not affected.

NetworkManager is the default for all new installations since 2022 and desktop installations since 2018, so this mostly affects systems which have been installed before, or have been switched to Wicked manually.

Users are advised to postpone system updates for now if they use Wicked or are unsure. Users who have updated already can use Snapper to rollback to an earlier snapshot.

The root cause appears to be a race condition between Wicked and the D-Bus system, which results in the network stack failing to initialize properly. When Wicked is launched, it struggles to interact with D-Bus, leading to the failure of various dependent network services. This sequence of events will leave the rolling release’s network stack inoperative, often requiring a manual restart to restore network functionality.

To address this problem, initial efforts are focusing on modifying the service dependencies in the systemd service files.

(Image made with DALL-E)

Improving notifications to stay informed about moderation reports

In the latest set of improvements coming to the content moderation feature we focused on improving the experience of moderators reviewing incoming reports. By including more information about the state of reports we make it easier for moderators to comprehend the user reported content in the notifications. This will lower the time the moderators need to respond to reports. Content Moderation is part of the beta program. Our journey into content moderation began back in...

Tue, Aug 27th, 2024

Addressing the Network Stack Issue

In recent testing scenarios involving a build and NetworkManager, a significant issue has surfaced: the network stack becomes non-operational.

Users are advised to postpone system updates for now, but if users have already updated, use Snapper to rollback; it’s important to note that while the issue primarily affects GNOME setups with Wicked, it can also impact servers without these components.

This problem has been consistently reproducible since at least the 20240825 Tumbleweed build. Bind 9.20.1 received an update has changes to DNS query handling and system controls, which may have inadvertently contributed to the network stack issue.

The root cause appears to be a race condition between Wicked and the D-Bus system, which results in the network stack failing to initialize properly. When Wicked is launched, it struggles to interact with D-Bus, leading to the failure of various dependent network services.

System logs show that D-Bus is either not fully active or not recognized by Wicked at the time of initialization, triggering a series of failures across services like DHCP and AutoIPv4.

This sequence of events will leave the rolling release’s network stack inoperative, often requiring a manual restart to restore network functionality.

To address this problem, initial efforts are focusing on modifying the service dependencies in the systemd service files. One proposed solution may be adding After=dbus.service to the Wicked service configuration. However, this adjustment alone may prove insufficient in many cases.

Further investigation is leading to more proposed solutions. The issue also appears to extend beyond Wicked, potentially affecting other services and indicating broader implications for the system’s initialization processes. The transition of NetworkManager and Wicked in some setups has uncovered the critical race condition affecting the network stack’s initialization. While recent adjustments to the systemd service configurations have significantly mitigated the issue, ongoing testing and further refinements are essential to achieve consistent network functionality. Users are advised to use snapper’s rollback to maintain proper network stack initialization.

(Image made with DALL-E)

First steps with Quickwit and syslog-ng

We are always looking for new ways to store log messages. Quickwit is a new contender, designed for log storage, and among others, it also provides an Elasticsearch-compatible API. From this blog, you can learn about Quickwit, and how to forward log messages from syslog-ng to it using the Elasticsearch-compatible API.

Read more at https://www.syslog-ng.com/community/b/blog/posts/first-steps-with-quickwit-and-syslog-ng

syslog-ng logo

Fri, Aug 23rd, 2024

Tumbleweed – Review of the week 2024/34

Dear Tumbleweed users and hackers,

Week 34 seemed to go almost without drama. Most snapshots passed openQA without big incidents. Most! In one snapshot, we tested updating to openSSH 9.8p1—general functionality was fine. Still, the SELinux policies have not yet been adjusted, which resulted in OpenSSH servers not starting up on MicroOS-based systems. This is nothing we want to give out to our users so we held back snapshot 0821. This will be worked out and openSSH 9.8p1 will be delivered as soon as possible. With this taken into account, 5 snapshots passed QA and could be published (0816, 0817, 0818, 0819, and 0820)

The five snapshots brought you the following changes:

  • Linux kernel 6.10.5: this helped unblock the s390 port
  • PCRE2 10.44
  • PHP 8.3.10
  • Bash 5.2.32
  • systemd 256.5
  • osc 1.9.0, fixing CVE-2024-22034. The file storage on disk has been updated, which causes issues with obs-service-source_validator not being able to handle the new layout. A fix is being worked on (https://github.com/openSUSE/obs-service-source_validator/pull/141) and we will deliver this as part of the Update channel and in future snapshots as soon as possible.

Looking at the staging areas, it seems like the vacation period is ending – and more things are getting ready soon. Currently, the teams are working on those changes:

  • LibreOffice 24.8.0
  • KDE Gear 24.08.0
  • Mozilla Firefox 129.0.1
  • perl-Bootloader will be renamed to update-bootloader: it’s been a while since there was no perl code in there anymore
  • dbus-broker: All staging tests have passed. We plan on integrating this into full snapshots early next week
  • GCC 14: phase 2: use gcc14 as the default compiler – All relevant build failures in Ring0 and Ring1 have been resolved. This has moved ‘up’ (to Staging:O) to get Staging QA runs. In rare cases, this might find some runtime issues stemming from the new compiler, but we do not think this would happen. Taking current progress into account, we should be able to switch by the end of August (dates are predictions, no commitment)

Wed, Aug 21st, 2024

We are switching syslog-ng containers from Debian Testing to Stable

For many years, the official syslog-ng container and development containers were based on Debian Testing. We are switching to Debian Stable now. Learn about the history and the reasons for the change now.

Read more at https://www.syslog-ng.com/community/b/blog/posts/we-are-switching-syslog-ng-containers-from-debian-testing-to-stable

syslog-ng logo

Mon, Aug 19th, 2024

openQA-Bites posted at 05:55

openqa-mq shows you openQA's RabbitMQ events

openqa-mq is a small CLI tool that receives openQA related events from RabbitMQ. It is part of the openqa-mon packages and will work for OSD and for OOO.

Fri, Aug 16th, 2024

Tumbleweed – Review of the week 2024/33

Dear Tumbleweed users and hackers,

Week 33 was busy, but busy in a good way. We managed to clear almost all stagings out, except the ‘long lasting’ topics like GCC, and dbus-broker, which we carried for a few weeks already. Other than that, the queue has been emptied (At the time of writing, there are now 54 pending requests to Factory). Summer vacation helped us achieve this result. And the fact, that we produced 7 snapshots (one discarded) during the last week.

The six published snapshots (0809, 0810, 0811, 0812, 0813, and 0815) brought you those changes:

  • GCC 13.3.1
  • glibc 2.40
  • KDE Frameworks 6.5.0
  • Mozilla Firefox 129.0
  • NetworkManager 1.48.8
  • binutils 2.43
  • cURL 8.9.1
  • Linux kernel 6.10.4
  • GO 1.22 has become the new default Go compiler version
  • FFMPEG default has switched from version 6 to version 7

As mentioned, stagings are almost empty – the few things currently left are:

  • Linux kernel 6.10.5
  • dbus-broker: some progress was made last week; most QA tests are fine, there is just a race condition on shutdown (likely not new, but dbus-daemon might have waited longer to report it, by when the system had completely shut down and the error has been ‘swallowed’)
  • GCC 14: phase 2: use gcc14 as the default compiler – great progress has been made and we believe we will be able to switch during Week 34

Thu, Aug 15th, 2024

he syslog-ng Insider 2024-08: 4.8.0 release; Prometheus; Amazon Linux

The August syslog-ng newsletter is now on-line:

  • Version 4.8.0 of syslog-ng improves FreeBSD and MacOS support
  • syslog-ng Prometheus exporter
  • Experimental syslog-ng packages for Amazon Linux 2023

It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-08-4-8-0-release-prometheus-amazon-linux

syslog-ng logo

Wed, Aug 14th, 2024

Introducing Labels and Bug Report Links

We’ve introduced several new features in OBS designed to foster collaboration among OBS users. Customized labels for better organization, setting custom links for your bug tracker and markdown formatting for project/package descriptions. Those features are intended to give you more insight into your work, helping you stay focused on what matters most. These updates are part of the Foster Collaboration and Labels beta programs. You can find more information about the beta program here. Introduction...