I love high performance computers, and some of my best friends work in high performance computing (HPC). Obviously, sometimes we also talk about logging. Recently we not just talked, but I also helped Gábor in his first steps with syslog-ng. He summarized his experiences in a blog:

Logs are one of those indispensable things in IT when things go wrong. Having worked in technical support for software products in a past life, I’ve likely looked at hundreds (or more) logs over the years, helping to identify issues. So, I really appreciate the importance of logs, but I can honestly say that I never really thought about a logging strategy for the systems on my home network - primarily those running Linux.

One of my longtime friends, Peter Czanik, who also works in IT, happens to be a logging guru as well as an IBM Champion for Power Systems (yeah!). So it’s only natural that we get to talking about logging. He is often complaining that even at IT security conferences people are unaware of the importance of central logging. So, why is it so important? For security it’s obvious: logs are stored independently from the compromised system, so they cannot be modified or deleted by the attacker. But central logging is beneficial for the HPC operator as well. First of all, it’s availability. You can read the logs even if one of your nodes becomes unreachable. Instead of trying to breath life into the failed node, you can just take a look at the logs and see a broken hard drive, or a similar deadly problem. And it is also convenience, as all logs are available at a single location. Logging into each node on the 3 node cluster to check locally saved logs is inconvenient but doable. On a 10 node cluster it takes a long time. On a 100 node cluster a couple of working days. While, if your logs are collected to a central location, maybe a single grep command, or search in a Kibana or similar web interface.

Those who follow my blog will know that I’ve been tinkering with a Turing Pi V1 system lately. You can read my latest post here. For me, the Turing Pi has always been a cluster in a box. My Turing Pi is fully populated with 7 compute modules. I’ve designed Node 1 to be the NFS server and LSF manager for the cluster. LSF is a workload scheduler for high-performance computing (HPC) from IBM. Naturally I turned to Peter for his guidance on this, and the result is this blog. Peter recommended that I use syslog-ng for log aggregation and also helped me through some of my first steps with syslog-ng. And the goal was to aggregate both the system (syslog) as well as LSF logs on Node 1. TL;DR it was easy to get it all working. But I encourage you to read on to better understand the nuances and necessary configuration both syslog-ng and LSF that was needed.

Read the rest at: https://www.gaborsamu.com/blog/turingpi_syslog-ng_lsf/

The openSUSE continues its Contribution Workshop series this week and has new episodes covering topics essential for newcomers and seasoned contributors. Upcoming Episodes

Episode 5: Contributing to openSUSE Leap - Project Structure, Feature Tracking, Package Updates for SLES Packages

• Date: April 4
• Time: 19:15 UTC
• Where: Live on both YouTube and Twitter

Episode 5 will dive deep into the world of openSUSE Leap by providing insights into the project, its structure, tools, how the team tracks feature and what the process is for building and updating SLES (SUSE Linux Enterprise Server) packages. This episode is a must-watch and will be live for anyone interested in the development processes behind openSUSE Leap so they understand how to contribute to the ecosystem.

Episode 6: Host Your Own openSUSE Mirror

• Date: Postponed
• Time: ASAP
• Where: openSUSE official channels

Episode 6 is expected to guide viewers through the process of hosting their own openSUSE mirror. This session is particularly geared toward those interested in infrastructure support and ensuring the accessibility and distribution of openSUSE’s resources worldwide.

These workshops not only offer a platform for learning but also for contributors to ask questions and engage directly with developers, maintainers and experienced members of the openSUSE community.

Whether you’re looking to dive into package maintenance, infrastructure, or understanding the overall project landscape, these episodes are tailored to provide a comprehensive overview and practical advice.

The following episodes were already released:

• Episode 1: openSUSE Contribution Workshop: Basic use of OBS osc using a version bump as an example
• Episode 2: openSUSE Contribution Workshop: From 0 to an rpm package packaging GNU Hello
• Episode 3: openSUSE Contribution Workshop: openSUSE Leap 15.6 Beta Bug Day
• Episode 4: openSUSE Contribution Workshop: Packaging Rust in Open Build Service

Table of Contents

• 1) Introduction
• 2) Unsafe Configuration Item “reposdir” in Whitelist
• 3) Bugfix and CVE Assignment
• 4) Discovery Process
• 5) Timeline
• 6) References

1) Introduction

CVE-2024-1929 that we previously reported for the dnf5 D-Bus component has not been completely fixed. This post deals with the remaining issue we discovered.

2) Unsafe Configuration Item “reposdir” in Whitelist

The problem with CVE-2024-1929 was that the dnf5 D-Bus daemon accepted arbitrary configuration parameters from unprivileged users, which allowed a local root exploit by tricking the daemon into loading a user controlled “plugin”. All of this happened before Polkit authentication was even started.

The original bugfix consists of a whitelist of configuration items, that unprivileged users are allowed to override, when using the dnf5 D-Bus interface. While checking each of the whitelisted items, we found that the setting “reposdir” allows to specify the path to an arbitrary directory, in which repository configuration files (*.repo) will be processed by the privileged dnf5 daemon.

The dnf5 library code does not check whether non-root users control the directory in question. The code does check for file type and filename extension of contained files; it follows symlinks and is subject to a race condition, however:

    std::filesystem::directory_iterator di(dir_path, ec);
    std::vector<std::filesystem::path> paths;

    for (auto & dentry : di) {
        auto & path = dentry.path();
        if (dentry.is_regular_file() && path.extension() == ".repo") {
            paths.push_back(path);
        }
    }

    std::sort(paths.begin(), paths.end());

    for (auto & path : paths) {
        create_repos_from_file(path);
    }

By the time the (checked) path is passed to create_repos_from_file(), the user controlling the directory can replace it with an arbitrary other file or symlink, thereby tricking the library to operate on arbitrary file types and file paths.

On one hand, this poses a Denial-of-Service attack vector by making the daemon operate on a blocking file (e.g. named FIFO special file) or a very large file that causes an out-of-memory situation (e.g. /dev/zero). On the other hand, this can be used to let the daemon process privileged files like /etc/shadow. The file in question is parsed as an INI file. Error diagnostics resulting from parsing privileged files could cause information leaks, if these diagnostics are accessible to unprivileged users. In the case of libdnf5, no such user accessible diagnostics should exist, though.

Even more interestingly, a local attacker can place a valid repository configuration file in this directory. This configuration file allows to specify a plethora of additional configuration options. This makes various additional code paths in libdnf5 accessible to the attacker. This, and the possibility to configure arbitrary repositories, could very well allow further privilege escalation, although we did not investigate more deeply if and how this would be possible.

This follow-up issue confirms the sentiment expressed in our original report, that one has to be extremely careful about feeding untrusted input into the libdnf5 library, which is not designed to run in mixed security scope setups.

3) Bugfix and CVE Assignment

The bugfix simply consists of the removal of the “reposdir” entry from the whitelist of configuration items. Upstream release 5.1.17 contains the bugfix. The Red Hat security team assigned CVE-2024-2746 to track this incomplete fix of CVE-2024-1929.

4) Discovery Process

We noticed the incomplete fix only at a late time, when our openSUSE dnf5 package maintainer asked for the inclusion of the fixed package into openSUSE Tumbleweed. It is unfortunate that this happened too late to prevent an incomplete fix for CVE-2024-1929, and thus made a follow-up CVE assignment and coordinated release process necessary.

Since the original issues had been handled as part of a coordinated disclosure process, there should have been a review of the proposed patches before publication. Due to the circumstances of an early publication of the fixes, outside of the coordinated release process, there never was a defined point in time for us to actually review them. We aim to avoid such situations in the future by being more careful about reviewing patches, especially when no straightforward coordinated release process can be established with upstream.

5) Timeline

6) References

• Previous report of dnf5 D-Bus daemon security issues
• Bugfix commit for the follow-up security issue
• dnf5 Bugfix release 5.1.17

Version 4.2 of syslog-ng introduced a healthcheck option to syslog-ng-ctl. It prints three syslog-ng-related metrics on screen – if it can reach syslog-ng, that is. You can use it from scripts to monitor the health of syslog-ng.

https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-health-check