Dear Tumbleweed users and hackers,

Last week’s report was written on Friday, but it was only published on Monday by accident. As it nonetheless only covered changes to Friday, I will include changes to Tumbleweed since I last WROTE a review – not since I last published one. This means this weekly review covers the six snapshots 0202, 0204…0208. This week, glibc was updated to version 2.39, and Python modules are newly also built for Python 3.12. For this kind of change, we had to give the control to rebuild the dependency chains to OBS, which in turn resulted in larger snapshots. As those updates were done, the rebuild strategy was reset to ‘rebuild packages with local source changes plus things identified by our bot needing a rebuild’

The most interesting changes of this week were:

• glibc 2.39
• Python 3.12 (python modules built for it, but /usr/bin/python3 will still point to Python 3.11 for now)
• Gstreamer 1.22.9
• QEmu 8.2.0
• timezone 2024a
• Mesa 23.3.5
• AppArmor 3.1.7
• Linux kernel 6.7.4

With glibc and python off the queue, some of the larger changes are gone from our todo. But we’re far from done with the list – and new things keep on appearing. Just the way we like it. We are currently testing the integration of these changes:

• Python 3.12.2
• A bunch of cleanup work to eliminate more of python2 (boo#1219306)
• dbus-broker: a big step forward; upgrades seem to be an issue that needs to be addressed
• libxml 2.12.x: slow progress
• c-ares 1.26.0: The build cycle could be addressed (splitting tests out into a 2nd run). We expect to ship this soon.
• GCC 14: our usual 2-phase approach to introduce it. Currently working on phase 1, meaning GCC14 will be providing the base libraries (libgcc_s1, libstdc++…). The compiler itself will stay at version 13 for now. Only one issue left: qemu fails to build; phase 2 testing has started, but will take several weeks

This whole blog post was born from the idea to clarify how to nicely force a newer compiler on older distros. But of course we have to start with the basics.

RPM conditionals basics

RPM supports two main forms of conditionals %if and %ifarch. The last one is easy … it lets us do architecture specific things:

%ifarch x86_64 aarch64
[snip]
%endif

The %if is a bit more flexible. We can use it to compare strings.

At the end of 2023 we announced Agama 7 stating that version is the first prototype we could consider to be (quoting ourselves) "functional enough", covering areas such as localization, network configuration, storage setup, authentication basis and some software selection. Now it's time to go deeper into every area... and we have a plan for that.

The Agama roadmap for 2024​

They say "plans are useless, but planning is indispensable". So we decided to do the indispensable exercise of planning the first months of 2024 and we came with this useless plan.

Although we will keep delivering new versions of Agama at a relatively constant pace, we established two milestones as check points. The first milestone is expected around mid April, the second one is scheduled to mid July. Then we used those two milestones to group the next tasks we want to tackle.

April's milestone should result in a revamped architecture for Agama which should not longer depend on Cockpit and in a more comprehensive user interface for configuring the storage setup. Both aspects are explained in depth in this blog post.

July's milestone should bring mechanisms to make Agama more adaptable and many improvements to unattended installation, turning Agama into a worthy contender to AutoYaST.

Let's dive into the improvements expected for the first milestone.

Architectural changes​

So far, we have built Agama on top of the infrastructure provided by the Cockpit project. That allowed us to bootstrap the project quickly without having to invest too much on aspects like authentication or serving files to the web interface. But after more than a year of Agama development we now have a clear view on how we want to do certain things, and Cockpit is starting to feel like a limiting factor.

See more details at this Github discussion, but in short we concluded the small of amount of functionality we are getting out of Cockpit does not justify the strong dependency, especially now that Cockpit is adopting Python as an integral part of its runtime.

So we will invest the following months in changing a bit the approach as described in the mentioned discussion at Github. That should unblock many paths to improve Agama in the near future.

A more powerful user interface for the storage proposal​

The mentioned architectural changes are important for remote or unattended installation and also regarding integration of Agama into bigger solutions, but they may not be that visible for the casual user. That doesn't imply next months will be boring in the area of interactive installation. Quite the opposite, we plan many improvements in the Agama's proposal page that allows to tweak the storage configuration.

The new interface aims to be easy enough for newcomers, as you can see in the following mock-up. But we know (open)SUSE users have big expectations in terms of customizing their setups. Thus, we updated the document that describes how the new interface will work and all the possibilities it will offer for those who decide to go beyond the initial proposal. If you want to check whether your basic needs would be covered, don't hesitate to take a look to the document and the extended mock-ups included on it.

If interface descriptions and mock-ups are not your cup of tea, worry not. We already started implementing some parts of the new interface, so you will be able to try the changes for real as they land incrementally at upcoming Agama prototypes.

openSUSE conference in the horizon​

If you look closely at the dates of both milestones described above, you will notice there is something happening almost in the middle - openSUSE Conference 2024!

We hope at that point in time Agama will be able to replace YaST for some scenarios and distributions. And, as such, we would like to use the conference to chat with the community about the possible future of Agama at openSUSE.

But, as we have mentioned in several previous occasions, the installation experience goes beyond the installer itself. The environment in which the installer is executed is also a crucial aspect. So apart from replacing YaST with Agama we would also need to replace the current so-called "installation image" with some modern alternative. So far, the testing Agama Live ISO has served us for demo purposes, but we would appreciate any help at building a system suitable for more real installation scenarios.

If you have good ideas about reducing the size of the live image, properly integrating the distribution repositories into it, streamlining the boot process, or any other topic... you know where to find us.

Stay in touch​

As said, your contributions and opinions are a key element to make sure Agama reach its goals, so never hesitate to contact the YaST team at the YaST Development mailing list, our #yast channel at Libera.chat or the Agama project at GitHub.

Help us to make 2024 the year of the new lizard!

Dear Tumbleweed users and hackers,

Once again I dared not to be at my desk last Friday, which resulted in me having to cover two weeks of Tumbleweed updates again. Quite a few larger things are happening, and you certainly want to know what has been coming – and will be coming – your way. The review covers the 12 snapshots released since my last ‘weekly review’ (0119, 0121…0126, 0128…0201)

The most relevant changes to your Tumbleweed system have been:

• rpm-config-SUSE: enable full ksym() dependencies in Tumbleweed
• libvirt 10.0.0
• PHP 8.2.15
• NetworkManager-applet 1.36.0
• Linux kernel 6.7.1 & 6.7.2. During the 6.7.1 lifetime, there were some package layout changes, which unfortunately resulted in build counters being reused; this had shown as ‘file conflicts’ in some cases. The later update to 6.7.2 solved this again for good.
• PAM 1.6.0
• Mesa 23.3.3 & 23.3.4
• Ruby 3.3 (rebuild, to get all ruby3.3-rubygem packages built). Ruby 3.3 is the new system default. Ruby 3.2 is still in the repos, which in turn means zypper dup will not clean it up from your system. If you have Ruby mostly for yast, you can likely uninstall ruby3.2 without loss of functionality (make sure to carefully check the packages to be removed – if in doubt, week it installed)
• Mozilla Firefox 122.0
• Postfix 3.8.5
• Ghostscript 10.02.1
• cURL 8.6.0
• RPM 4.19.1: some stricter spec file parsing. As a packager, make sure to read https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/HG2JKUIKDTWQQIQSA43A4VWHX7YKJQT3

These things are currently being staged and being prepared for future inclusion into Tumbleweed:

• glibc 2.39
• Python 3.12 (python modules built for it, but /usr/bin/python3 will still point to Python 3.11 for now)
• GStreamer 1.22.9
• QEmu 8.2.0: causes build failures of ovmf
• dbus-broker: a big step forward; upgrades seem to be an issue that needs to be addressed
• libxml 2.12.x: slow progress
• openSSL 3.2.0
• c-ares 1.21.0: nA new cycle has formed: appstream-glib, c-ares, curl, googletest, nghttp2, python311. This should be eliminated, as cycles cause massive trouble when branching new code streams
• GCC 14: our usual 2-phase approach to introduce it. Currently working on phase 1, meaning GCC14 will be providing the base libraries (libgcc_s1, libstdc++…). The compiler itself will stay at version 13 for now.

Once we integrate glibc 2.39 plus the python 3.12 changes, we will let OBS sort the dep chain for the new Python 3.12 modules, as this task is not handled by our external bot that usually takes care of the rebuild strategy. This will result in a huge snapshot, likely to be published early next week.

Python is a interpreted language, so the python code are just text files with the .py extension. For simple scripts it's really easy to have your files located, but when you starts to use dependencies and different projects with different requirements the thing starts to get more complex.

PYTHONPATH

The Python interpreter uses a list of paths to try to locate python modules, for example this is what you can get in a modern GNU/Linux distribution by default:

Python 3.11.7 (main, Dec 15 2023, 10:49:17) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
 '/usr/lib64/python311.zip',
 '/usr/lib64/python3.11',
 '/usr/lib64/python3.11/lib-dynload',
 '/usr/lib64/python3.11/site-packages',
 '/usr/lib64/python3.11/_import_failed',
 '/usr/lib/python3.11/site-packages']

These are the default paths where the python modules are installed. If you install any python module using your linux packaging tool, the python code will be placed inside the site-packages folder.

So system installed python modules can be located in:

• /usr/lib/python3.11/site-packages for modules that are architecture independent (pure python, all .py files)
• /usr/lib64/python3.11/site-packages for modules that depends on the arquitecture, that's something that uses low level libraries and needs to build so there are some .so files.

pip

When you need a new python dependency you can try to install from your GNU/Linux distribution using the default package manager like zypper, dnf or apt, and those python files will be placed in the system paths that you can see above.

But distributions doesn't pack all the python modules and even if they do, you can require an specific version that's different from the one packaged in your favourite distribution, so in python it's common to install dependencies from the Python Package Index (PyPI).

Python has a tool to install and manage Python packages that looks for desired python modules in PyPI.

You can install new dependencies with pip just like:

$ pip install django

And that command looks for the django python module in the PyPI, downloads and install it, in your user $HOME/.local/lib/python3.11/site-packages folder if you use --user, or in a global system path like /usr/local/lib or /usr/lib if you run pip as root.

But the usage of pip directly in the system is something not recommended today, and even it's disabled in some distributions, like openSUSE Tumbleweed.

[danigm@localhost ~] $ pip install django
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try
    zypper install python311-xyz, where xyz is the package
    you are trying to install.

    If you wish to install a non-rpm packaged Python package,
    create a virtual environment using python3.11 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-rpm packaged Python application,
    it may be easiest to use `pipx install xyz`, which will manage a
    virtual environment for you. Install pipx via `zypper install python311-pipx` .

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

virtualenvs

Following the current recommendation, the correct way of installing third party python modules is to use virtualenvs.

The virtualenvs are just specific folders where you install your python modules and some scripts that make's easy to use it in combination with your system libraries so you don't need to modify the PYTHONPATH manually.

So if you've a custom project and want to install python modules you can create your own virtualenv and use pip to install dependencies there:

[danigm@localhost tmp] $ python3 -m venv myenv
[danigm@localhost tmp] $ . ./myenv/bin/activate
(myenv) [danigm@localhost tmp] $ pip install django
Collecting django
...
Successfully installed asgiref-3.7.2 django-5.0.1 sqlparse-0.4.4

So all dependencies are installed in my new virtualenv folder and if I use the python from the virtualenv it's using those paths, so all the modules installed there are usable inside that virtualenv:

(myenv) [danigm@localhost tmp] $ ls myenv/lib/python3.11/site-packages/django/
apps  contrib  db        forms  __init__.py  middleware   shortcuts.py  templatetags  urls   views
conf  core     dispatch  http   __main__.py  __pycache__  template      test          utils
(myenv) [danigm@localhost tmp] $ python3 -c "import django; print(django.__version__)"
5.0.1
(myenv) [danigm@localhost tmp] $ deactivate

With virtualenvs you can have multiple python projects, with different dependencies, isolated, so you use different dependencies when you activate your desired virtualenv:

• activate $ . ./myenv/bin/activate
• deactivate $ deactivate

High level tools to handle virtualenvs

The venv module is a default Python module and as you can see above, it's really simple to use, but there are some tools that provides some tooling around it, to make it easy for you, so usually you don't need to use venv directly.

pipx

For final python tools, that you are not going to use as dependencies in your python code, the recommended tool to use is pipx.

The tool creates virtualenv automatically and links the binaries so you don't need to worry about anything, just use as a way to install third party python applications and update/uninstall using it. The pipx won't mess your system libraries and each installation will use a different virtualenv, so even tools with incompatible dependencies will work nicely together in the same system.

Libraries, for Python developers

In the case of Python developers, when you need to manage dependencies for your project, there are a lot of nice high level tools for managing dependencies.

• PDM
• hatch
• micropipenv
• pip-tools
• pipenv
• poetry

These tools provides different ways of managing dependencies, but all of them relies in the use of venv, creating the virtualenv in different locations and providing tools to enable/disable and manage dependencies inside those virtualenvs.

For example, poetry creates virtualenvs by default inside the .cache folder, in my case I can find all poetry created virtualenvs in:

/home/danigm/.cache/pypoetry/virtualenvs/

Most of these tools add other utilities on top of the dependency management. Just for installing python modules easily you can always use default venv and pip modules, but for more complex projects it's worth to investigate high level tools, because it'll make easy to manage your project dependencies and virtualenvs.

Conclusion

There are a lot of python code inside any modern Linux distribution and if you're a python developer it's possible to have a lot of python code. Make sure to know the source of your modules and do not mix different environments to avoid future headaches.

As a final trick, if you don't know where's the actual code of some python module in your running python script, you can always ask:

>>> import django
>>> django.__file__
'/tmp/myenv/lib64/python3.11/site-packages/django/__init__.py'

This could be even more complicated if you start to use containers and different python versions, so keep you dependencies clean and up to date and make sue that you know where is your Python code.

Do you have to forward large amounts of logs between two syslog-ng instances? OTLP (OpenTelemetry protocol) support in syslog-ng was contributed by Axoflow, and it can solve this problem. Just like the ewmm() destination, syslog-ng-otlp() forwards most name-value pairs, however, unlike a tcp() connection, it scales well with multiple CPU cores.

Support for OpenTelemetry was added to syslog-ng a couple of releases ago. OpenTelemetry is an observability framework, mainly used in Linux / Cloud / Kubernetes environments. However, I already had users asking to make this feature available on FreeBSD. (It already worked once, but now it fails to compile again.)

Version 4.6.0 added many new OTLP-related enhancements. Batching and multiple workers make OTLP connections significantly faster, while compression can save you bandwidth at the expense of some more CPU usage. This changes the syslog-ng-otlp() destination from an interesting experiment into something really useful. It enables you to send a lot more log messages between two syslog-ng instances than with a tcp() connection, while using less bandwidth.

Read more at https://www.syslog-ng.com/community/b/blog/posts/using-opentelemetry-between-syslog-ng-instances

The openSUSE Conference in Nuremberg, Germany, at the end of June may feel like a long time away, but if you are planning to oSC24, there are topics that need action now before timelines become too short.

Getting a visa may take some time. There are certain requirements necessary to receive a visa for those who are not citizens of a Schengen country. You may need a formal invitation letter that fully explains the nature of your visit.

An overview of visa requirements/exemptions for entry into the Federal Republic of Germany can be found at the Federal Foreign Office website. You must apply for a Schengen visa through the German embassy or consulate in your country.

If you plan to attend the conference and are coming from a country where you will need a formal invitation letter, email ddemaio(at)opensuse.org with the email subject “oSC24 Visa”.

Also important to consider is the Travel Support Program. People who want to use the TSP for this year’s openSUSE Conference must read the How to Apply section and follow the appropriate steps. The TSP is now being done by the Geeko Foundation, so be aware that some new procedures and certain limitations apply. Read the info on the wiki and use tsp.opensuse.org to apply. Requests will be accepted until May 10. The TSP is a helpful way to get people in the community to attend the conference. Funding is limited, but help to support the community’s attendance.

Here are a few reminders regarding the TSP:

• Please read the TSP policy page carefully before you apply.
• The TSP can reimburse up to 80% of travel and/or lodging costs. That includes hotel, hostel, plane, train, bus
• Important: Food and all local expenses are on you!
• We want to sponsor as many people as possible so please check the best deal.
• No receipts = no money. It is the rule! (Original receipts are required from German residences.)

The call for papers is open until April 15. Submit a talk today.

Presentations can be submitted for the following length of time:

• Lightning Talk (15 mins)
• Virtual Lightning Talk (15 mins)
• Short Talk (30 mins)
• Virtual Talk (30 mins)
• Long Talk (45 mins)
• Workshop (1 hour)
• Open 4 Business (15 mins)

The following tracks are listed for the conference:

• Cloud and Containers
• Community
• Embedded Systems and Edge Computing
• New Technologies
• Open Source
• openSUSE
• Open 4 Business

Welcome to the monthly update for openSUSE Tumbleweed for January 2024. This will be the new format going forward as recommended by those contributing to the marketing efforts of openSUSE. These updates will highlight major changes, improvements and key issues addressed in openSUSE Tumbleweed snapshots throughout the month. The aim is to keep the community and users informed about the developments and updates of the distribution. Should readers desire a more frequent amount of information about openSUSE Tumbleweed snapshots, readers are advised to subscribe to the openSUSE Factory mailing list.

New Features and Enhancements

• Linux Kernel: Updates to versions 6.6.7, 6.6.9, 6.6.10, 6.6.11 and 6.7.1.
  • Fixes have been applied for memory management and security vulnerabilities, enhancing overall system safety.
  • Support for new hardware models
    • PCI: Adds ACS quirks for more Zhaoxin Root Ports, enhancing compatibility and performance for Zhaoxin’s CPUs and motherboards.
    • ALSA (Advanced Linux Sound Architecture): Added driver properties for cs35l41 for Lenovo Legion Slim 7 Gen 8 series, and introduced support for additional Dell models without _DSD, along with fixes for HP Envy X360 13-ay0xxx’s mute and mic-mute LEDs, indicating broader compatibility for sound hardware in laptops.
    • LEDs: The ledtrig-tty module receives updates to the free allocated ttyname buffer on deactivation, impacting how LED triggers are handled for terminal activities.
• Mozilla Firefox: Updates to version 121.0 and 121.0.1
  • The update resolves a bug that caused Firefox to hang when loading sites with column-based layouts, enhancing stability and performance.
  • Fixes applied to ensure rounded corners for videos and proper closure of Firefox that prevents USB security key conflicts with other applications.
• KDE Frameworks: Update for version 5.114.0.
  • Significant updates include fixes in Extra CMake Modules, introduction of holidays in Kenya observed by KHolidays, and quality settings adjustments for AVIF in KImageFormats.
  • Key improvements in KIO for handling malformed Exec entries, accessibility enhancements in Kirigami, and stability fixes in KJobWidgets to prevent potential use-after-free errors.
• Mesa: Updates to 23.3.3
  • Focus on Python 3.6 build fixes and enhancements in driver support.
  • The release introduces NVK, a new Vulkan driver for NVIDIA hardware, which marks a step forward in support for NVIDIA GPUs, yet it remains in the experimental phase.
  • Improved graphics performance and compatibility Asahi and RADV and enhancements of OpenGL ES and Vulkan capabilities
  • Introduces critical updates like the requirement of libvulkan1 for Mesa-dri to support zink/swrast driver fallbacks, which further improves the overall user experience with graphics applications and games.
• systemd: Updates to version 254.8
  • Reverts patches related to udev device node updates and workarounds for issues. Took a cautious approach to fixing reported bugs and ensuring stability in device management systems.
  • Adjustments to udev ensure the proper existence and ownership of %_modulesloaddir, facilitating smoother module installation by other packages, thereby improving system configuration and module management.
• PHP: Updated from version 8.2.14 to 8.2.15,
  • Fix for a false positive SSA integrity verification failure and a resolution for Autoconf warnings during cross-compilation.
  • The CLI built-in web server now correctly handles timeouts when using router scripts in conjunction with max_input_time.
  • Fixes a crash when using stream_wrapper_register with FFI\CData and interaction issues between FFI::new and observers.
  • IntlDateFormatter now correctly accepts ‘C’ as a valid locale.
  • A hanging issue in the Hash extension for large strings with sha512 is resolved.
• GStreamer: Updates to version 1.22.8
  • Addressing vulnerabilities within the AV1 video codec parser.
  • Fixes include resolving a potential deadlock in the avdec video decoder with FFmpeg 6.1
  • Improvements in reverse playback and seeking in qtdemux for files with raw audio streams
  • Enhancements to the GstPlay and GstPlayer libraries
  • Updates to the Cerbero build tool to address python 3.12 string escape warnings
• Samba : Updates to version 4.19.4
  • Addresses issues like the inability of net changesecretpw to set the machine account password with an empty secrets.tdb,
  • Improves documentation generation with respect to XML_CATALOG_FILES environment variable.
  • Resolved issues where smbd did not detect ctdb public IPv6 addresses for multichannel exclusion, and the force user = localunixuser setting was ineffective when allow trusted domains = no.
  • Addressed critical vulnerabilities and bugs, such as visible Deleted Object tombstones in AD LDAP to normal users CVE-2018-14628, and various smbget authentication and functionality fixes, enhancing security and user experience.

Security Updates

This month’s updates include critical security patches across various packages. Notable security improvements were integrated into the Firefox, systemd, Samba and PHP updates and more.

Bug Fixes

• xorg-x11-server 21.1.11 and xwayland 23.2.4: These updates addressed multiple CVEs, improving security and stability in the display server protocols. A list of this CVEs can be found in the security advisory.
• gnutls 3.8.3: CVE-2024-0553 was a vulnerability that allows timing attacks in RSA-PSK, risking data leak and a fix for CVE-2024-0567 was made, which is a flaw in cockpit’s certificate validation that enables remote denial of service attacks.
• java-11-openjdk 11.0.22.0: Multiple CVEs. CVE-2024-20919, CVE-2024-20926 , CVE-2024-20921, CVE-2024-20918, CVE-2024-20945, CVE-2024-20952
• samba 4.19.4: CVE-2018-14628 an authenticated but unprivileged attacker could have discovered the names and preserved attributes of deleted objects in the LDAP store.
• python-Jinja2 3.1.3: CVE-2024-22195 was a flaw where the xmlattr filter improperly allows space-containing keys, enabling attackers to inject harmful attributes through user inputs.
• rdma-core 49.1: Although specific CVEs addressed in the update were not mentioned, the update is part of regular maintenance to ensure stability and security​.

Contributing to openSUSE Tumbleweed

Your contributions and feedback make openSUSE Tumbleweed better with every update. Whether reporting bugs, suggesting features, or participating in community discussions, your involvement is highly valued.

Conclusion

We will continue to refine and enhance this format. We look forward to another exciting year of development and community engagement with openSUSE Tumbleweed. See you all at FOSDEM next week. Happy computing!