A recent post on the YaST blog about Agama’s roadmap looks at the new installer as functional enough to embark on tasks ranging from localization and network configuration to storage setup and initial software selection.

For those who don’t follow the YaST blog, here is what lies ahead for Agama in 2024.

The team has outlined a strategy for this year and, despite the fluidity of its development, the team is committed to a steady release schedule for Agama with two significant milestones. The first is set for mid-April and the other toward mid-July.

The milestone in April is set to revolutionize Agama’s architecture. It will be moving away from its reliance on Cockpit toward a more autonomous framework that is coupled with a refined user interface that aims to streamline storage configurations.

The aim of the second milestone is to improve Agama’s flexibility and capabilities for unattended installations, which seeks to position Agama as a formidable alternative to AutoYaST.

The scaffolding provided by the Cockpit Project makes the vision for Agama’s future clear in evolving a direction of a new path. The coming months will be dedicated to redefining this approach to ensure Agama’s growth is unhindered by external dependencies.

While architectural modifications lay the groundwork for future advancements, an equal focus must be made to enhance the user experience. The revamped storage configuration interface will be both user-friendly for newcomers and more versatile for the experience. This aims to provide a balance of simplicity and customization.

The openSUSE Conference 2024 is nestled between the milestones and the team will make use of this event to serve as a platform for discussing Agama’s potential to redefine the installation experience within the openSUSE ecosystem. insights and contributions are vital to Agama’s success so stakeholders are encouraged to engage with the team, share ideas and participate in the ongoing development of Agama.

Read more information about Agama on the YaST blog.

The February syslog-ng newsletter is now on-line:

• Version 4.5.0 of syslog-ng is now available with OpenObserve JSON API support
• Syslog-ng PE can now send logs to Google BigQuery
• Syslog-ng can now do a full configuration check
• How build services make life easier for upstream developers

It is available at https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-02-openobserve-configuration-check-build-services

The openSUSE community is pleased to announce that it will have short sessions aimed at encouraging people on how to contribute to the project.

A group of volunteers will present short 15-minute sessions that are streamed and/or recorded on openSUSE’s YouTube channel that are aimed at teaching people about packaging, using the Open Build Service, creating tests for openQA and other development areas.

The first session about “Basic use of OBS/osc using a version bump as an example” is set to begin tomorrow, on Feb. 15 at 21:00 UTC.

Update: The “Packaging Guidelines (Patch Policies) and Submission of New Packages” session scheduled for Feb. 27 at 16:00 UTC has been postponed.

More sessions are expected to be scheduled for future dates.

The sessions are listed on the openSUSE Calendar; look for the Contribution Workshop sessions marked in orange.

Those who are interested in presenting should fill in the blank area for future sessions listed in the email about the events.

Giving a session is a great way to give back to the community and provides opportunities to teach others skills and knowledge about open-source development.

Dear Tumbleweed users and hackers,

Last week’s report was written on Friday, but it was only published on Monday by accident. As it nonetheless only covered changes to Friday, I will include changes to Tumbleweed since I last WROTE a review – not since I last published one. This means this weekly review covers the six snapshots 0202, 0204…0208. This week, glibc was updated to version 2.39, and Python modules are newly also built for Python 3.12. For this kind of change, we had to give the control to rebuild the dependency chains to OBS, which in turn resulted in larger snapshots. As those updates were done, the rebuild strategy was reset to ‘rebuild packages with local source changes plus things identified by our bot needing a rebuild’

The most interesting changes of this week were:

• glibc 2.39
• Python 3.12 (python modules built for it, but /usr/bin/python3 will still point to Python 3.11 for now)
• Gstreamer 1.22.9
• QEmu 8.2.0
• timezone 2024a
• Mesa 23.3.5
• AppArmor 3.1.7
• Linux kernel 6.7.4

With glibc and python off the queue, some of the larger changes are gone from our todo. But we’re far from done with the list – and new things keep on appearing. Just the way we like it. We are currently testing the integration of these changes:

• Python 3.12.2
• A bunch of cleanup work to eliminate more of python2 (boo#1219306)
• dbus-broker: a big step forward; upgrades seem to be an issue that needs to be addressed
• libxml 2.12.x: slow progress
• c-ares 1.26.0: The build cycle could be addressed (splitting tests out into a 2nd run). We expect to ship this soon.
• GCC 14: our usual 2-phase approach to introduce it. Currently working on phase 1, meaning GCC14 will be providing the base libraries (libgcc_s1, libstdc++…). The compiler itself will stay at version 13 for now. Only one issue left: qemu fails to build; phase 2 testing has started, but will take several weeks

At the end of 2023 we announced Agama 7 stating that version is the first prototype we could consider to be (quoting ourselves) “functional enough”, covering areas such as localization, network configuration, storage setup, authentication basis and some software selection. Now it’s time to go deeper into every area… and we have a plan for that.

The Agama Roadmap for 2024

They say “plans are useless, but planning is indispensable”. So we decided to do the indispensable exercise of planning the first months of 2024 and we came with this useless plan.

Although we will keep delivering new versions of Agama at a relatively constant pace, we established two milestones as check points. The first milestone is expected around mid April, the second one is scheduled to mid July. Then we used those two milestones to group the next tasks we want to tackle.

April’s milestone should result in a revamped architecture for Agama which should not longer depend on Cockpit and in a more comprehensive user interface for configuring the storage setup. Both aspects are explained in depth in this blog post.

July’s milestone should bring mechanisms to make Agama more adaptable and many improvements to unattended installation, turning Agama into a worthy contender to AutoYaST.

Let’s dive into the improvements expected for the first milestone.

Architectural Changes

So far, we have built Agama on top of the infrastructure provided by the Cockpit project. That allowed us to bootstrap the project quickly without having to invest too much on aspects like authentication or serving files to the web interface. But after more than a year of Agama development we now have a clear view on how we want to do certain things, and Cockpit is starting to feel like a limiting factor.

See more details at this Github discussion, but in short we concluded the small of amount of functionality we are getting out of Cockpit does not justify the strong dependency, especially now that Cockpit is adopting Python as an integral part of its runtime.

So we will invest the following months in changing a bit the approach as described in the mentioned discussion at Github. That should unblock many paths to improve Agama in the near future.

A More Powerful User Interface for the Storage Proposal

The mentioned architectural changes are important for remote or unattended installation and also regarding integration of Agama into bigger solutions, but they may not be that visible for the casual user. That doesn’t imply next months will be boring in the area of interactive installation. Quite the opposite, we plan many improvements in the Agama’s proposal page that allows to tweak the storage configuration.

The new interface aims to be easy enough for newcomers, as you can see in the following mock-up. But we know (open)SUSE users have big expectations in terms of customizing their setups. Thus, we updated the document that describes how the new interface will work and all the possibilities it will offer for those who decide to go beyond the initial proposal. If you want to check whether your basic needs would be covered, don’t hesitate to take a look to the document and the extended mock-ups included on it.

If interface descriptions and mock-ups are not your cup of tea, worry not. We already started implementing some parts of the new interface, so you will be able to try the changes for real as they land incrementally at upcoming Agama prototypes.

openSUSE Conference in the horizon

If you look closely at the dates of both milestones described above, you will notice there is something happening almost in the middle - openSUSE Conference 2024!

We hope at that point in time Agama will be able to replace YaST for some scenarios and distributions. And, as such, we would like to use the conference to chat with the community about the possible future of Agama at openSUSE.

But, as we have mentioned in several previous occasions, the installation experience goes beyond the installer itself. The environment in which the installer is executed is also a crucial aspect. So apart from replacing YaST with Agama we would also need to replace the current so-called “installation image” with some modern alternative. So far, the testing Agama Live ISO has served us for demo purposes, but we would appreciate any help at building a system suitable for more real installation scenarios.

If you have good ideas about reducing the size of the live image, properly integrating the distribution repositories into it, streamlining the boot process, or any other topic… you know where to find us.

Stay in touch

As said, your contributions and opinions are a key element to make sure Agama reach its goals, so never hesitate to contact the YaST team at the YaST Development mailing list, our #yast channel at Libera.chat or the Agama project at GitHub.

Help us to make 2024 the year of the new lizard!

Dear Tumbleweed users and hackers,

Once again I dared not to be at my desk last Friday, which resulted in me having to cover two weeks of Tumbleweed updates again. Quite a few larger things are happening, and you certainly want to know what has been coming – and will be coming – your way. The review covers the 12 snapshots released since my last ‘weekly review’ (0119, 0121…0126, 0128…0201)

The most relevant changes to your Tumbleweed system have been:

• rpm-config-SUSE: enable full ksym() dependencies in Tumbleweed
• libvirt 10.0.0
• PHP 8.2.15
• NetworkManager-applet 1.36.0
• Linux kernel 6.7.1 & 6.7.2. During the 6.7.1 lifetime, there were some package layout changes, which unfortunately resulted in build counters being reused; this had shown as ‘file conflicts’ in some cases. The later update to 6.7.2 solved this again for good.
• PAM 1.6.0
• Mesa 23.3.3 & 23.3.4
• Ruby 3.3 (rebuild, to get all ruby3.3-rubygem packages built). Ruby 3.3 is the new system default. Ruby 3.2 is still in the repos, which in turn means zypper dup will not clean it up from your system. If you have Ruby mostly for yast, you can likely uninstall ruby3.2 without loss of functionality (make sure to carefully check the packages to be removed – if in doubt, week it installed)
• Mozilla Firefox 122.0
• Postfix 3.8.5
• Ghostscript 10.02.1
• cURL 8.6.0
• RPM 4.19.1: some stricter spec file parsing. As a packager, make sure to read https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/HG2JKUIKDTWQQIQSA43A4VWHX7YKJQT3

These things are currently being staged and being prepared for future inclusion into Tumbleweed:

• glibc 2.39
• Python 3.12 (python modules built for it, but /usr/bin/python3 will still point to Python 3.11 for now)
• GStreamer 1.22.9
• QEmu 8.2.0: causes build failures of ovmf
• dbus-broker: a big step forward; upgrades seem to be an issue that needs to be addressed
• libxml 2.12.x: slow progress
• openSSL 3.2.0
• c-ares 1.21.0: nA new cycle has formed: appstream-glib, c-ares, curl, googletest, nghttp2, python311. This should be eliminated, as cycles cause massive trouble when branching new code streams
• GCC 14: our usual 2-phase approach to introduce it. Currently working on phase 1, meaning GCC14 will be providing the base libraries (libgcc_s1, libstdc++…). The compiler itself will stay at version 13 for now.

Once we integrate glibc 2.39 plus the python 3.12 changes, we will let OBS sort the dep chain for the new Python 3.12 modules, as this task is not handled by our external bot that usually takes care of the rebuild strategy. This will result in a huge snapshot, likely to be published early next week.

Python is a interpreted language, so the python code are just text files with the .py extension. For simple scripts it's really easy to have your files located, but when you starts to use dependencies and different projects with different requirements the thing starts to get more complex.

PYTHONPATH

The Python interpreter uses a list of paths to try to locate python modules, for example this is what you can get in a modern GNU/Linux distribution by default:

Python 3.11.7 (main, Dec 15 2023, 10:49:17) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
 '/usr/lib64/python311.zip',
 '/usr/lib64/python3.11',
 '/usr/lib64/python3.11/lib-dynload',
 '/usr/lib64/python3.11/site-packages',
 '/usr/lib64/python3.11/_import_failed',
 '/usr/lib/python3.11/site-packages']

These are the default paths where the python modules are installed. If you install any python module using your linux packaging tool, the python code will be placed inside the site-packages folder.

So system installed python modules can be located in:

• /usr/lib/python3.11/site-packages for modules that are architecture independent (pure python, all .py files)
• /usr/lib64/python3.11/site-packages for modules that depends on the arquitecture, that's something that uses low level libraries and needs to build so there are some .so files.

pip

When you need a new python dependency you can try to install from your GNU/Linux distribution using the default package manager like zypper, dnf or apt, and those python files will be placed in the system paths that you can see above.

But distributions doesn't pack all the python modules and even if they do, you can require an specific version that's different from the one packaged in your favourite distribution, so in python it's common to install dependencies from the Python Package Index (PyPI).

Python has a tool to install and manage Python packages that looks for desired python modules in PyPI.

You can install new dependencies with pip just like:

$ pip install django

And that command looks for the django python module in the PyPI, downloads and install it, in your user $HOME/.local/lib/python3.11/site-packages folder if you use --user, or in a global system path like /usr/local/lib or /usr/lib if you run pip as root.

But the usage of pip directly in the system is something not recommended today, and even it's disabled in some distributions, like openSUSE Tumbleweed.

[danigm@localhost ~] $ pip install django
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try
    zypper install python311-xyz, where xyz is the package
    you are trying to install.

    If you wish to install a non-rpm packaged Python package,
    create a virtual environment using python3.11 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-rpm packaged Python application,
    it may be easiest to use `pipx install xyz`, which will manage a
    virtual environment for you. Install pipx via `zypper install python311-pipx` .

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

virtualenvs

Following the current recommendation, the correct way of installing third party python modules is to use virtualenvs.

The virtualenvs are just specific folders where you install your python modules and some scripts that make's easy to use it in combination with your system libraries so you don't need to modify the PYTHONPATH manually.

So if you've a custom project and want to install python modules you can create your own virtualenv and use pip to install dependencies there:

[danigm@localhost tmp] $ python3 -m venv myenv
[danigm@localhost tmp] $ . ./myenv/bin/activate
(myenv) [danigm@localhost tmp] $ pip install django
Collecting django
...
Successfully installed asgiref-3.7.2 django-5.0.1 sqlparse-0.4.4

So all dependencies are installed in my new virtualenv folder and if I use the python from the virtualenv it's using those paths, so all the modules installed there are usable inside that virtualenv:

(myenv) [danigm@localhost tmp] $ ls myenv/lib/python3.11/site-packages/django/
apps  contrib  db        forms  __init__.py  middleware   shortcuts.py  templatetags  urls   views
conf  core     dispatch  http   __main__.py  __pycache__  template      test          utils
(myenv) [danigm@localhost tmp] $ python3 -c "import django; print(django.__version__)"
5.0.1
(myenv) [danigm@localhost tmp] $ deactivate

With virtualenvs you can have multiple python projects, with different dependencies, isolated, so you use different dependencies when you activate your desired virtualenv:

• activate $ . ./myenv/bin/activate
• deactivate $ deactivate

High level tools to handle virtualenvs

The venv module is a default Python module and as you can see above, it's really simple to use, but there are some tools that provides some tooling around it, to make it easy for you, so usually you don't need to use venv directly.

pipx

For final python tools, that you are not going to use as dependencies in your python code, the recommended tool to use is pipx.

The tool creates virtualenv automatically and links the binaries so you don't need to worry about anything, just use as a way to install third party python applications and update/uninstall using it. The pipx won't mess your system libraries and each installation will use a different virtualenv, so even tools with incompatible dependencies will work nicely together in the same system.

Libraries, for Python developers

In the case of Python developers, when you need to manage dependencies for your project, there are a lot of nice high level tools for managing dependencies.

• PDM
• hatch
• micropipenv
• pip-tools
• pipenv
• poetry

These tools provides different ways of managing dependencies, but all of them relies in the use of venv, creating the virtualenv in different locations and providing tools to enable/disable and manage dependencies inside those virtualenvs.

For example, poetry creates virtualenvs by default inside the .cache folder, in my case I can find all poetry created virtualenvs in:

/home/danigm/.cache/pypoetry/virtualenvs/

Most of these tools add other utilities on top of the dependency management. Just for installing python modules easily you can always use default venv and pip modules, but for more complex projects it's worth to investigate high level tools, because it'll make easy to manage your project dependencies and virtualenvs.

Conclusion

There are a lot of python code inside any modern Linux distribution and if you're a python developer it's possible to have a lot of python code. Make sure to know the source of your modules and do not mix different environments to avoid future headaches.

As a final trick, if you don't know where's the actual code of some python module in your running python script, you can always ask:

>>> import django
>>> django.__file__
'/tmp/myenv/lib64/python3.11/site-packages/django/__init__.py'

This could be even more complicated if you start to use containers and different python versions, so keep you dependencies clean and up to date and make sue that you know where is your Python code.