Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS

openSUSE Tumbleweed and MicroOS are now delivering an image that is using systemd-boot as boot loader and full disk encryption based also on systemd. The unlock of the encrypted device can be done via the traditional password, a TPM2 (a crypto-device that is already present in your system) that will attach the device if the system is in good health, or a FIDO2 key that will validate the ownership of a token.

There is a lot to explain here, but basically those changes are in the direction of moving the distribution into a more safe place. For one side is making the design of the distribution much more simple, and for another it is following the current trends about security that other distributions are also aligning with.

So, lets start with the beginning …

systemd-boot

We all know and love GRUB2. It is a good boot loader. It is also big, complex, rich, massive and tends to move slow on the development side.

The openSUSE package for this boot loader contains more than 200 patches. Some of those patches are there for the last 5, 6 … 10 years. That is both an indication of the talent of the maintainers, but also can signal an issue in how slow the upstream contribution process can be.

GRUB2 supports all the relevant systems, including mainframes, arm or powerpc. Multiple types of file systems, including btrfs or NTFS. It contains a full network stack, an USB stack, a terminal, can be scripted … In some sense, it is almost a mini OS by itself.

But then UEFI happened 18 years ago, making almost all the features provided by GRUB2 somehow redundant. The system firmware was already providing most of these functionalities as services that can be consumed by the operating system, the boot loader or any other user provided application. And of course GRUB2 supported UEFI too.

Soon the Linux kernel gained the option of being compiled as an EFI binary, via a stub that can be attached to the kernel code. This implies that the kernel itself could be launched by the firmware directly, making the boot loader something optional in most of the cases.

Over time new and more straightforward boot loaders focused on UEFI appeared, like gummiboot. Later this code was integrated into systemd and renamed as systemd-boot.

The code is very simple. Many orders of magnitude simpler than GRUB2. It is basically a very small EFI binary that presents a menu with the different boot loader entries (text files described in the Boot Loader Specification or BLS for short), and a call to the UEFI LoadImage function to delegate the execution to the selected kernel.

This boot loader can also work with the new unified kernel images (UKI), that are files that aggregate in a single unit the kernel, the command line, and the initrd. Those UKIs can be very handy for image based distributions, and openSUSE plans to support them as well.

Providing systemd-boot as an alternative for GRUB2 is something that openSUSE wanted to do for a long time. In August 2023 there was an announcement on the Factory mailing list about Tumbleweed supporting systemd-boot.

The announcement references a wiki entry that explains how to migrate an installation using GRUB2 to systemd-boot manually. Soon after the announcement, yast-bootloader gained support for it for new installations.

Supporting another boot loader comes with a cost. As argued, the code base is smaller, with less bugs and more easy to reason about. But the UEFI dependency decreases the amount of supported architectures (x86-64 and aarch64). That problem can be very much alleviated by providing another patch for GRUB2 to support the BLS entries, so the architecture of the distribution after the boot loader can be independent of the boot loader itself. The good news is that the patch already exists, and could potentially be added into the package.

Another problem is that systemd-boot does not speak btrfs. As an EFI binary, it can read files only from a FAT32 file system. This limitation can be resolved by moving the kernel and the initrd into the EFI system partition (ESP).

Finally, there is also the consideration of supporting snapshots in Tumbleweed and transactions in MicroOS. From the boot loader the user should be able to select what snapshot to boot from, like it is actually possible to do when using GRUB2. Both concepts are implemented using btrfs subvolumes, and there is only a subset of kernel, command line, initrd combinations that are valid for each of those subvolumes.

For example, let’s say we have two snapshots in our system, and each of these represents a system that has two kernels installed. It is possible that those two kernels are not the same across all the snapshots. Maybe one of the upgrades replaced one kernel with a newer version. We need some tool that can do the bookkeeping required to associate the correct combination that will produce a successful boot into any of those snapshots, creating the boot entries under those restrictions.

This tool is sdbootutil. Every time snapper creates or destroys a snapshot (for example, when the system gets updated), it will call this tool that will analyze the content of the snapshots, making sure that the corresponding kernel is installed in the ESP, a valid initrd for this kernel is present (if not it will be created calling mkinitrd) and a boot entry is created that connects the kernel, the initrd and the snapshot via the command line. It also takes care of other details, like checking the free space on the partition.

Usually his process works transparently, but is good to remember that we can force a clean state with:

sdbootutil add-all-kernels
sdbootutil remove-all-kernels

Just in case, you know …

Full disk encryption

The other aspect that we want to announce is the support of full disk encryption (FDE) based on systemd.

FDE is not the new kid on the block. GRUB2 could unlock LUKS volumes since long ago using the cryptomount command. Traditionally this will request the password from the user two times: once when the boot loader does the unlock and again when the initrd does the same later. There are ways to avoid the second request injecting the password into the initrd or, if you are using the openSUSE package, it will inject the password transparently into the initrd.

Recently GRUB2 gained two new features: partial support of LUKS2 encrypted devices (using PBKDF2 as key derivation function instead of the more secure and recommended Argon2id) and a key protection mechanism that can store secrets in devices like the TPM2.

TPM2

Explaining how TPM2 works in detail is a topic for another post, but for now we can think of it as a crypto device that be used to unlock secrets only when certain conditions related to the state of the system are met. The TPM2 will unlock the secret if the system is in a healthy state.

This term is a technical one, and is related to assert that the system is in a known good state. In other words, we know for sure that the firmware has not been tampered with, the boot loader is the one that we installed and has not been replaced, that the kernel is exactly the one that comes from the distribution, that the kernel command line is the one that we expect, and that the initrd that we used does not contain any extra binary that we do not control.

Internally the TPM2 has some registers, known as platform configuration register (PCR). In the TPM2 specification there are 24 of them and the size of one is enough to store the value of a hash function, like SHA1 or SHA256. They are separated by banks: one per supported hash function, but this is too much detail for now.

Those registers are kind of special. We can reset them, usually setting the value to 0. We can read the value, or we can “extend” them. The write operation is designed in a way that we cannot set any random value in the register, except the result of the associated hash function concatenating the current PCR value and a new value provided by the user.

The current value of the PCR can only be produced by extending this register using exactly the same sequence of values. If we change even one bit of one of the values, we will produce a wildly different final result for the same PCR.

This feature is used in a process known as “measured boot”, where each stage in the boot chain is measured before it is executed. This means that before the initial stages of the firmware are running, there is a process that will calculate the hash of the code in memory, and extend one of the PCRs using this value. This is repeated until the very end of the boot sequence: the kernel and the initrd.

When measured boot is in place, the final values of the first 10 PCRs will contain values than can only be predicted if the machine is using a well known version of firmware, boot loader and kernel, together with the associated data like certificates, configuration files, or kernel parameters. If one of those elements change (for example, by using a different secure boot certificate), it will generate PCR values different from the ones that we expect.

TPM2 chips are very interesting devices, and the set of features go far beyond measured boot. If you want to learn more I recommend resources like this or this.

TPM2 for FDE

Anyway, the gist here is that we can create a “policy” that can instruct the TPM2 to decrypt a secret only if certain PCRs contains the expected values. The details are a bit different, but for now lets use this model as a good first approximation.

The idea is that we can encrypt a password with the values of certain PCR registers, so GRUB2 can later attach the LUKS2 device if the TPM2 can recover the password, validating the health of the system until this point. If the TPM2 fails to decrypt it, that would mean that some PCR has not the expected value and some stage in the boot process changed. In this situation GRUB2 will ask the password from the user to continue loading the kernel and the rest of the system. It delegates the trust about the new state to the user.

GRUB2 also provides a tool to seal secrets under the current values of a subset of PCRs. This is nice but also presents several problems. One is that maybe we are setting the system up in a way that we know the PCRs values will change during the next boot (for example, during the first installation, a boot loader upgrade or a firmware update). In this case sealing the password using the current register values is useless: we need to be able to predict the new ones and use those hypothetical values to do the sealing.

The other problem is more insidious and will become critical later. The expected values can change frequently and can not be unique. Maybe there is a set of valid ones. We can choose to boot from a different kernel or from a different snapshot. The TPM2 provides a solution for this using something known as authorized policies. They are a way of creating policies that can change, but they are validated by a signature. In essence, we create a public and a private key, and we create multiple PCR policies that are signed using the private key. Now the TPM2 can validate the signature using the public part, and unseal the secret using the PCRs values stored in the new policy.

Since early 2023 openSUSE provides the pcr-oracle tool to help with the prediction of the PCR registers, and encrypt a key under those values using both PCR policies or authorized policies. Using this tool we can now seal a secret under a set of PCRs values that can change!

In the openSUSE wiki we can find more documentation about those topics, including instructions about how to use it in our installation.

Using systemd for disk encryption

With GRUB2 the FDE is working properly, so why look for something else? One reason is very evident: this architecture can only work … well … only if our openSUSE GRUB2 version is used. It will not work for other boot loaders like systemd-boot. In fact it will not work with the the upstream version of GRUB2 itself.

But there is a second reason: we can argue that there is not a full measured boot in place with GRUB2. If the boot loader needs to unlock the device before it can load the kernel, is natural that the PCR policies that will evaluate the health of the system cannot make asserts on the kernel, command line or initrd that will be used. Those will be loaded after the LUKS2 device has been opened.

The use of systemd-boot gives us an alternative architecture for FDE that can work properly with any boot loader that follows the BLS (remember, there is a patch for GRUB2 to support it somewhere, so it is not excluded a priori), and provides the chance to do a full measured boot attestation before unlocking the device.

One difference is that the kernel and the initrd will be placed in the unencrypted ESP, and the unlock of the sysroot will be done from inside the initrd using the different options that systemd-cryptsetup offers. Currently it can unlock the device using a normal password, a TPM2 with authorized policies (with optionally a PIN that must be entered by the user) or a FIDO2 key device. In the /etc/crypttab file we need to describe the unlocking mechanism.

pcr-oracle has been extended to support the creation of authorized policies that systemd can understand. They are stored in a JSON file that contains multiple predictions, each one of them indicating the PCRs involved, the TPM2 policy hash, the fingerprint of the public key and the signature of the policy. This, together with the public key PEM file, composes all the data required for systemd-cryptsetup to use the TPM2 for the unseal of the LUKS2 key.

The RSA 2048 key used to sign the policy can be created with openssl or with pcr-oracle itself. A note of caution: if the private key gets leaked, this is a game over for the expected security that the TPM2 could provide. Luckily the solution is cheap in this case: generate a new key, re-register the key in the LUKS2 key slot with systemd-cryptenroll and use sdbootutil to regenerate the predictions for each boot entry. Yeah … we will document all the process in the “systemd-fde” wiki page and provide better tools, but trust me, it is indeed a cheap operation.

openSUSE is providing a MicroOS image named kvm-and-xen-sdboot that shows how all of this is working. This image contains some of the already mentioned tools integrated and some other new ones:

• systemd-boot: Boot loader used instead of the default GRUB2
• sdbootutil: Helper scripts to synchronize the boot entries of the system
• pcr-oracle: Predict the PCRs values for the next boot, and creates the authorized policies for systemd
• disk-encryption-tool: Encrypt the device where sysroot is located on the first boot
• dracut-pcr-signature: dracut module that will load the predictions into the initrd from the ESP

Those tools are designed to work together for this new FDE architecture. What follows is a brief description on how all is connected.

Once we get the new MicroOS qcow2 image and we setup the VM, we can proceed with the boot process. If the VM has a virtual TPM2 device it will start measuring the executed code and data, extending the corresponding PCRs. Once systemd-boot has been reached, it will find the correct boot entry for this session and will read the corresponding kernel and initrd from it.

At this moment the image is not encrypted. Inside the initrd that is used during this first boot, the disk-encryption-tool script will be called. Using some heuristics it will find the partition that belongs to sysroot (where the system is located), and will resize it to reserve 32MB for the LUKS2 header. After that it will use all the magic that cryptsetup provides to re-encrypt the device using a locally generated password. This password, as of today, corresponds to the recovery key that will be presented to the user at the end and the user should take note and keep it safe.

After the re-encryption, the system /etc/crypttab will be updated to communicate that this device is now encrypted and should be managed with different tools later.

At the end of the initrd we switch to the new sysroot, now finally located in an encrypted device. The disk-encryption-tool script already did its main job, but it installed two modules for jeos-firstboot, that will be executed on the first boot of the system, which is currently happening!

The first module, enroll, will detect if there is a FIDO2 key inserted and a TPM2 available. If so it will present a dialog asking what do you want to use to unlock the system. The second module will ask the user if the root password will also be enrolled in the LUKS2 header as a new key, and will show the recovery key generated earlier.

As of today it is not advisable to register both. As we described earlier the FIDO2 key will make more sense if we are using a laptop or a desktop machine and we want unlock the encrypted device with proof of a token that we own. This is an interactive process. The TPM2 makes more sense on situations where we do not want to interact with the system, and we want to automatically unlock the device only if we can assert the health of the system (no tamper occured in the boot chain).

If we register the FIDO2 key, systemd-cryptenroll will be called and we will be asked to press the button two times and the installation process will be over. At the next boot we will be required to present the key, and if the key is missing, the recovery password will be asked.

If we register the TPM2 device, a new RSA 2048 key gets generated and stored (the public and private parts) in /etc/systemd and systemd-cryptenroll will be used to enroll the public key and to annotate the PCRs that are used in the sealing of the LUKS2 key. By default we will be using 0, 2, 4, 7, and 9. You can check the meaning in this reference. PCRs 0 and 2 will measure all the UEFI firmware code. PCR 4 will measure the boot loader (systemd-boot) and the kernel (also UEFI binaries). PCR 7 will register all the secure boot certificates, and PCR 9 will be used by the kernel to measure the command line and the initrd.

This covers pretty much all that can make sense, but it is the user who has the final word on what to measure. The reason is that the predictions are done inside sdbootutil that, remember, will be automatically executed after each change in the system (updates, package removal, snapshots management, etc), and this tool will produce predictions only for the PCRs registered in the LUKS2 header.

Regardless of the selected unlocking mechanism, the /etc/crypttab file will be updated with this selection and a new initrd will be generated to contain this information for the next boot.

Finally, the last component, dracut-pcr-signature will be responsible that during the subsequent boots all the information that systemd-cryptsetup requires for the unlock will be present “on-the-fly” inside the initrd. It should be noted that the initrd will require the JSON file with the policies and the key, but those cannot be included in the initrd! The moment that we make a prediction of a PCR that is extended with the hash of the initrd, that is all, and we cannot touch the initrd anymore as this would produce a new hash and automatically will invalidate the prediction.

This dracut module will be executed before the systemd-cryptsetup generator for any encrypted devices has started, and will search in the ESP partition for a tpm2-pcr-signature.json file, that contains all the valid prediction for the current boot. Once this file is in place, the systemd-crypsetup will be able to assert the device in the current state is the expected one and the boot process can continue until the end.

Future

The image is here, and is a sound PoC. It provides a much more simple architecture and will place some components in the correct place. This will help a lot in the next stages, as there are some other things that we want to do with the distribution in relation to FDE.

One pretty clear disk-encryption-tool has limited use outside image based installation. Part of this code should be living in YaST and in Agama. The installer is already creating LUKS2 devices, so it should be “easy” to extend it in a way that works for us.

Ideally, the jeos-firstboot modules should also live in the installer, but somehow they make sense here too. In any case the functionality should not be separated, and both should be merged.

The encryption tool is doing something right from the very start: the master key, together with all the user keys are generated during installation time, but one possible improvement is generating the recovery key a bit later using the systemd tools. It is a small detail, but separating system keys from users keys can simplify the architecture.

Another aspect to improve is that the user may want to use the TPM2 and the FIDO2 key at the same time. For example, by default the TPM2 is used, and if the stage changed in a way that fails the prediction (or there is a security breach that has been detected), the user can delegate the unlock to the FIDO2 key, instead of using a password.

The sdbootutil script contains a bunch of features that should be also living in systemd. Working with upstream will make this tool obsolete with time, which would be more good news.

Another improvement that we can help with in systemd is to improve the diagnosis about the reasons making the TPM2 reject the unseal of the LUKS2 key. Today we have a general fail message without reporting what PCR or what measured component inside the PCR is reporting a different hash than the one predicted. This will help a lot understating what did go wrong. Was the boot loader changed? Or something in the firmware?

pcr-oracle is a very good tool for predicting the next PCR values. It was very easy to extend to parse the new events in the log related with the full measured boot process, including the kernel, systemd-boot extensions on PCR 12, or generating the JSON document required by systemd. The new systemd 255 (released a week ago from the time of writing this) includes a similar tool named systemd-pcrlock that can help us in providing the improved diagnosis that we are looking for. Evaluating this tool to do the predictions will be done soon too.

As today Type#1 and Type#2 entries from the BLS are not isomorphic. There are sections in the EFI file of the UKI format that do not exist in the text representation. Maybe we will decide to use UKIs in the future, or maybe not. So a good improvement is working on helping with this unification, that will (among other things) provide a standard way of splitting the JSON file and associating the predictions to each boot loader entry.

Generating and registering a new key, or selecting a different set of PCRs is today a manual process. The current tools can be extended to help in those processes, or better documentation could be provided.

The new approach for FDE is not about excluding GRUB2 from the equation. It is about providing a chance of using different boot loaders that follows the BLS. Validating that a proper patched (duh!) GRUB2 can work with all this is still something to be done.

Also, another thing that needs to be validated and improved are installations with multiple encrypted disks. In principle the design and the code is supporting it (even when the PCR registers per volume are different). openQA will do wonders here.

And finally, we should rethink if the UKIs do make sense for openSUSE or not. If we go in that direction, the private key used for signing the policies will be kept in OBS and those policies will also be generated in the build service, using a different set of PCR values.

In any case, there is a bunch of work ahead of us.

If it’s Christmas, then for many people it means watching Home alone. Related memes have appeared in social media already in the summer: “100 more days before you watch Home Alone again” ;-) Well, I do not recall when I watched it for the last time, so probably not in the past decade…

However, even I know that the most famous song from the movie is Carol of the Bells. I was surprised to hear how many musicians make a cover of a song from a film. Well, if you check out the Wikipedia article above, you will see that this is not a song by John Williams. Long story short: the song originates from Ukraine, and the English version is less than a century old. Let’s start listening to the most famous version:

TIDAL: https://listen.tidal.com/album/52597873/track/52597885

There are many cover versions of this song. Even the Wikipedia article has a long list, but if you search for “Carol of the Bells” on YouTube or TIDAL, you will find many more. Probably the most famous, apart from “Home Alone”, is made by The Piano Guys:

TIDAL: https://listen.tidal.com/album/23004975/track/23004979

Of course, there are also countless metal versions. I listened to a few, and I liked the version by Charlie Parra del Riego the most:

TIDAL: https://listen.tidal.com/album/221145965/track/221145967

You know a song is an integral part of popular culture when it appears in a game soundtrack:

TIDAL: https://listen.tidal.com/album/333849250/track/333849251

And for now, something completely different. One of the most famous Hungarian folk music groups is Kaláka. Every year they hold a Christmas concert, where they sing many Christmas songs, Hungarian folk traditions and others, as well. A couple of times I was lucky enough to participate in these concerts. The following video shows how they walk in to the stage singing:

TIDAL: https://listen.tidal.com/album/162358955

My Mini Delta Also Doesn't work with Cura :(

I need to why it works with OctoPi but why OctoPrint and Cura can't detect my printer on plain old OpenSuse. Some missing plugin or something?

A New Print

This morning I wanted to see if I could print something new. This entails getting Cura set up on my OpenSuse machine. I was worried it would be difficult, but it was anything but. I was easily able to find the download Cura page, and to download the AppImage version.

I wasn't quite sure how to actually use it, but it turned out I could just chmod and run it.

Cura started up quickly, and walked me through creating an account (not sure if that is needed) and configuring my printing.

After I followed the wizard, I could see that my printer was nicely configured.

Then I did a little searching to find something that I wanted, but was also quick and easy to print. I found this.

I downloaded the STL file and loaded into Cura, and then clicked slice. I forgot to change some defaults which I only slightly regret, you'll see why in a minute.

After saving the generated gcode, I opened an old gcode file that I had and replaced the first part of the new file with these old commands. I learned from trial and error the best set of start up commands years ago, so I always reuse them. I stuck the commands here in case you have the same printing and are interested.

Then I just needed to load the new gcode file into Octoprint. I forgot to configure Cura to tell it not to create that base, as I think it isn't necessary for this kind of print, but, no harm done, really.

And now it's printing. The print is going fine. Would probably be cleaner if I didn't keep shaking the work bench as a type this :) Just as the print was finishing up, I realized that the filament I was using is simply transparent, not glow in the dark. It made the key ring sort of weird looking. I swear I had a spool of glow in the dark filament, but, if so, I have no idea what I did with it.

Resurrecting my 3D Printer

A few years back I got myself a Monoprice Mini Delta 3d printer. I chose it because it was inexpensive, and self-leveling, so seemed like a great way to get started with the hobby, and it really delivered.

My first print a few years ago:

I started out printing parts to organize my electronics tool bench, and you can see (a lot of the prints I made on thingiverse)[https://www.thingiverse.com/rickspencer3/makes].

I haven't printing in a while though, and I have some free time, and there are a few simple things I wanted to print, so I pulled the printer out of the closet, all covered in dust and everything. First problem, the spool of black PLA that I had left on it had turned rather brittle. So I pulled out the old PLA, and put on a spool of glow in the dark PLA that I happened to have kept in a plastic back with a silicon moisture absorbing pack, so it still seemed good.

In installed Octoprint on my OpenSuse Leap machine and attached it, but I couldn't make it connect. This turned out to be a common problem with my printer and Linux in general (not a Suse-specific issue). On a side note, installing Octoprint with pip3 on Suse was dead easy, a real please.

In any case, I had Octopi running on rpi that is still attached to my workbench, so I scrounged up a power supply for it, attached a tiny monitor, booted it up and ... it still works!

You can see that I was printing out ear savers for face masks, so that gives you a sense of the last time I printed. Since the ear savers are very fast prints, I decided it was a good test print to do, so I decided to start with that, and it worked!

Here you can see the printed ear saver with some of the black PLA still sort of mixed in:

Here's the whole set up printing away:

Overall, I am really happy with the printer and set up. Considering this has all been collecting dust for years, and that I was easily able to get it up and running again with such little drama made me really happy. Now I can move on to getting Cura working and printing some of the new things I wanted to print.

Barcelona, Naples, Rome

It's 6:30am and I am sitting with my morning cup of coffee rocking the sweet sweet jetlag after 10 full days in Europe, mostly Italy.

What was I doing there? The motivation for the trip is that I got a new job. Even though I don't start until January 2nd, they invited me out to join their annual Sales Kick Off event in Barcelona. So, I had a ticket in hand to Europe anyway, and I had some free time.

Barcelona?

I arrived in at the airport in Barcelona after an easy flight, thanks to having a "lie flat" seat on United. I was, of course, totally jet-lagged, so I took only 2 photos over the 3.5 days that I was in Spain. The first day, before the event started, I took a metro ride into the city and visited a Christmas market, but didn't take any pictures. The real purpose of the trip was to keep from falling asleep.

As you can see from the above photo, it was a largish event. While I hadn't officially started yet, the event was real work for me.

I was absorbing as much information about the company, customers, and products that I could. Second, I got to meet a few of the people on my new team.

I left the event feeling very confident in my choice to join. I especially appreciated meeting the folks from my new team, as they were all top notch in terms of capabilities, but also I found them very easy to get along with.

Also, the event wasn't really in Barcelona, it was off season in this crazy family theme park.

Naples

Herculaneum

My primary goal of visiting Italy was to visit Herculaneum. I have been wishing to visit for years. I prepared for this part of the trip by reading as much as I could about the site, and watched some college level lectures and tours. So when I went, I was already well acquainted with the major parts of it.

I was so engrossed with the site, that I didn't end up taking any pictures. I figure that this is one of the most well documented sites in the world anyway, and I wanted to stay in the moment.

Here is one of the videos that I watched in preparation for the tour that I found very helpful.

I visited the site twice, once alone, and then again with my wife. Let me say that the site did not disappoint. You can really get the feel of the city and what life was like when you are there. The experience is quite different than Pompeii, which we also visited.

Naples

I chose a hotel for it's location, being quite close to Herculaneum. We did, however, take the quick train ride into the city to visit the Christmas market, and to eat at a restaurant that my brother in law, Craig, recommended.

Naples was insanely crowded. The city was crushed with people in the region in town for shopping and visiting the Christmas market. People in Naples seem to really really love Christmas.

The Christmas markets (Christmas Ally was the one we visited) were loaded with elaborate models for making different kinds of nativity scenes. It was way too crowded and hectic to get pictures, but I did get this video (for you Craig):

We had a super nice dinner at Mimi alla Ferrovia.

We got there a little early, and there was one other couple waiting. After we ate, we noticed that there was quite a line of folks waiting to get in.

I got the Pasta alla Genovese and a glass of red wine. Both delicious. I even made a big batch of Pasta alla Genovese when I got home, but it didn't compare of course. But it was fun to make, and I have the time to wait hours for things to cook.

Overall, the crowds in Naples were way too much for me. The only bigger crowds I have ever seen were at Queen's Day in Amsterdam, but that was a different vibe.

Pompeii

If the crowds in Naples were overwhelming, our day in Pompeii was an amazing antidote. It was a very easy train ride from our hotel, and it was a nice sunny dat.

Unlike Herculaneum, I did no research about Pompeii specifically, though I had watched various documentaries about it and such in the past. The city, for various reasons, was much more "ruined" than Herculaneum, so it was harder to get a feel for what it was like when it was a functioning city.

Due to being less engrossed from my previous research, I was more tuned into taking pictures, though. Pompeii is probably one of the most photographed places in the world, but here are a couple of pictures to evoke what the day was like.

Rome

I loved our time in Naples and Herculaneum, so we decided to extend our trip with a few days in Rome. My wife had been many times, but I had never been there. I was interested in seeing the ruins, especially, the Forum.

We stayed at a hostel called The Beehive, though, in reality, we had a hotel room sort of embedded in the hostel. The location near the train station was great. The room was very comfortable, and the people were very nice. Having access to the shared kitchen and living room was a huge benefit as well.

Getting to Rome was quite easy with the fast train from Naples. About 1h15 trip, and very smooth. On our first day there I did some laundry, but we also visited the city.

Here we are the Trevi Fountain.

There was an amazing Gelato place not too far from the Beehive, we ate there a few times.

That first night we also ate another restaurant Craig recommended, Urbana47. We had a great night there. We drank delicious wine. My wife had the rabbit, but I had the simple pasta with butter and anchovies. Positively delicious.

We stumbled out of there after great food, wine, and whiskey.

Colosseum and Forum Tour

The next morning we did some hard core site seeing. We hired a tour guide to take us through the Colosseum and to take us through a quick tour of the Forum. They recently unlocked the highest level of the Colosseum, but you had to buy special tickets for it, which our tour guide showed us how to buy. It was amazing to see the place from that vantage point.

We also did a quick run through of the Forum and especially Palatine Hill, but this was largely in preparation for a return visit in a couple of days.

Heart of Rome Tour

• map of tour
• a pic or two
• more food: Pinsere

Back to the Forum

• map of tour
• more pics

Dear Tumbleweed users and hackers,

Full steam ahead to the end of the year. It seems nothing makes Tumbleweed slow down. The number of requests coming in keeps being high and you had the chance to upgrade your machine 7 times this week.

The seven snapshots (1207…1213) of this week brought you these changes:

• KDE Plasma 5.27.10
• KDE Gear 23.08.4
• Late-comers of GNOME 45.2 (i.e mutter)
• Tar 1.35
• util-linux 2.39.3
• Mozilla Firefox 120.0.1
• Linux kernel 6.6.6

The next snapshot is already on the way to the mirrors, Stagings are ready to move on and we can confirm the following changes being worked on:

• KDE Frameworks 5.113.0
• Boost 1.84.0
• cURL 8.5.0
• RPM 4.19.x
• cmake 3.28.1 (ceph still failing to build)
• Ruby 3.3 tests have started (yast failures identified so far)
• libxml 2.12.x: slow progress
• openSSL 3.2.0
• c-ares 1.21.0: breaks nodejs
• wxWidgets 3.2.3: breaks wxPython bindings
• dbus-broker: no progress: openQA fails to launch the network stack in the installer
• Addition of Kalpa to the regular Tumbleweed deliverables

The past few weeks have been an exciting time for the openSUSE Project as discussions about the visual identity of the project offers a glimpse into people’s various views about the project and its brand identity.

The recent conclusion of the openSUSE logos contest has sparked extensive discussions among both members actively engaged in the openSUSE Project and those participating in it.

Our logo contest has provided us with a wealth of creative input and diverse perspectives that lay a strong foundation for deliberations on the future direction for the project.

The contest provided a voice for the many who aren’t as vocal as some about selecting a new logo. While there were some who were vocal on various platforms, the contest gave openSUSE Project members an opportunity to gauge how the broader community perceives the project.

The submissions and voting outcomes offered a glimpse into the collective vision of open-source enthusiasts who may not be directly involved in the project but are crucial stakeholders nonetheless.

As efforts move forward with the outcome of this, inclusivity and community involvement remain at the core of the decision-making process.

During the community meeting this week where the results were discussed, participants expressed the view that members of the openSUSE Project have an opportunity to participate in the selection of our new logo, and that SUSE, which holds the trademark to the openSUSE logo, be involved with the process for selecting a branding decision with regard to the results. After all, this decision impacts the collective identity.

To facilitate this, there is a plan to organize a vote between the current logo and the proposed new design, allowing our community to have a say in this important decision. Furthermore, members of the project are collaborating with SUSE on the implications of the branding initiatives and some have expressed the desire for SUSE’s input to ensure there is an aligned vision for the future of openSUSE.

A two-step approach to spearhead the discussions and decision-making processes with key stakeholders is crucial in driving a plan of action forward and implementing any changes to our branding strategy.

Outlined below are the proposed steps that will guide the project through this journey:

Step 1:

• Evaluation of Contest Results (completed)
• Assessing the Path Forward (completed)
• Engaging with SUSE for Brand Consideration & Assessing Contributions
• Presentation of Branding Strategy
• Community Voting Engagement
• Organizing a voting process for openSUSE Project members to choose between the Old and New Logo

Step 2:

• Collaboration with Stakeholders
• Aligning Logo Ownership with SUSE
• Trademarking and Implementation

The aim is to ensure transparency, inclusivity, and alignment with the collective goals of the openSUSE Project throughout this process.

For a detailed review of the survey results from the logo contest, visit our Logo Contest page.

People interested in becoming a member of openSUSE should visit our wiki on How to become a member.

Thank you for your continued support and enthusiasm. A big thank you to all the people who submitted a design and those who voted. Winners of the contests have been contacted about winning and will be shipped a Geeko Mystery Box.

The winners of the openSUSE logo contest across various categories are as follows:

• openSUSE: AO01

• Tumbleweed: Three-way tie AO31, AO05, and AO11

• Leap: AO01

• Slowroll: AO03

• Kalpa: AO03

This report is about a range of predictable /tmp path issues in various applications in the budgie-extras repository. This repository contains a range of helper applications for the Budgie desktop environment.

During a routine review of applications that are autostarted in X11 environments I found the issues 1) to 4) outlined below. Upstream found two additional cases of predictable /tmp path uses that they addressed, as outlined in items 5) and 6). Upstream released version 1.7.1 today which fixes all the issues.

Introduction

The affected programs are mostly written in the Vala programming language, some are also scripted in Python. In all cases predictable paths in /tmp containing only the username or no variable components at all are used. In these paths regular files or directories are created. The paths are often used as a kind of inter-process-communication between two or more budgie-extras components.

The impact of the issues differs a lot depending on the actual affected program and ranges from denial-of-service to information leaks to integrity issues through manipulation of the data which is used e.g. for displaying images on the desktop. All the issues are restricted to local attackers, naturally.

Without the Linux kernel’s protect symlink sysctl setting the severity of the issues will in some cases be worse. Even with this protection enabled it is often possible to pre-create the files or directories as another local user, granting world read and write access, which will cause the budgie-extras applications to use them even though they are attacker controlled.

Without the Linux kernel’s symlink protection many of these findings where files are created look like they might allow symlink attacks to have files created in arbitrary locations. The Vala file creation calls I looked into are mostly translated into the following system call, though:

openat(AT_FDCWD, <path>, O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666)

Even tough this is missing the O_NOFOLLOW flag, symlinks would not be followed, due to the combination of O_CREAT and O_EXCL. I will point out cases where symlink attacks might still be possible in spite of this.

As a quick fix for all of these issues I suggested to use $XDG_RUNTIME_DIR instead of /tmp. This directory is private to the logged in user and cannot be manipulated by other users in the system. In the instances where these files are used as a simple communication mechanism (“trigger” logic) it could be considered using sockets, the D-Bus session bus or named FIFOs instead - also placed in safe locations, of course.

Following is a detailed listing of all individual issues based on the budgie-extras Git repository tag for version 1.7.0.

1) budgie-window-shuffler (CVE-2023-49344)

1.1) Path /tmp/shufflerapplettrigger_<user>

In src/shuffler_control.vala line 1740 first an attempt is made to delete this path. Then it starts monitoring the path, reacting to its creation, by automatically selecting (popping up) the “Applet” listbox GUI entry.

The counterpart to this is found in applet/src/ShufflerApplet.vala line 91, where this file is created to let the settings dialog open.

The worst that can happen here is likely confusing the victims GUI so it is low severity.

1.2) Path /tmp/<user>_shufflertriggers/layoutspopup

In src/toggle_layouts_popup.vala line 62 first an attempt is made to create the directory, ignoring any potential errors - considering it to already exist. Then the “layoutspopup” file is created within the directory. Depending on program evaluation logic the string “fromcontrol” is written to the file, otherwise the file remains empty.

In src/layouts_popup.vala line 1384 monitoring for this file is setup, its content is read (it is checked whether it contains “fromcontrol”) and then a popup window is either created or destroyed, depending on the current program state.

Another user in the system can pre-create this directory and then control the creation and destruction of the popup dialog, thereby confusing the victim’s GUI. By placing a FIFO instead of a regular file at “layoutspopup”, the layout popup will be subject to denial-of-service (either by blocking it indefinitely or by feeding it large amounts of data, leading to an out-of-memory situation).

Without the Linux kernel’s symlink protection the issue can be used to make the layouts_popup program read from arbitrary files, or to operate in arbitrary directories.

1.3) Path /tmp/<user>_running_layout

In src/run_layout.vala line 203 this file is created to “temporarily disable possibly set windowrules”. In line 379 this path is (needlessly) constructed again and passed to function create_busyfile(), although this parameter remains unused by the function. In line 478 stat() and unlink() are attempted on the file.

In src/windowshufflerdaemon.vala line 831 an existence check for this file is made. If it exists then the run_rule program will not be executed for any windows.

This path allows a local attacker to prevent the victim’s run_rule ever to be executed.

1.4) Path /tmp/<user>_gridtrigger

In src/togglegui.vala line 33 an existence check is made for this path and depending on the outcome it is either created as an empty file, or deleted.

In src/windowshufflerdaemon.vala line 992 in function actonfile() there is a reaction to the creation and deletion of this path. Depending on this the gridguiruns boolean is set to true or false respectively. If it is set to false then a window will be destroyed in line 1148.

In src/gridwindow.vala line 637 a monitor is setup for this file and depending on it being created or being deleted the gridwindow is being displayed or destroyed.

This path basically allows a local attacker to make the “grid window” managed by the gridwindow program appear, thereby confusing the victim’s GUI. The other way around windowshufflerdaemon can be caused to destroy its “preview window” if this state file is under a local attacker’s control.

1.5) Path /tmp/shuffler-warning.png

In src/windowshufflerdaemon.vala line 1017 in function create_warningbg() this path is used to write a programmatically created PNG image into. In function show_awarning() in line 338 the program sizeexceeds_warning is executed which in turn in src/sizeexceeds_warning.vala line 68 displays the generated PNG image on the desktop.

A local attacker can attempt to place arbitrary PNG data in this path and have it displayed on the victim’s desktop. Placing crafted PNG data could allow to exploit further security issues in image processing libraries.

1.6) Path /tmp/<user>_istestingtask

This path is potentially created in src/layouts_popup.vala line 492. The file receives data from the GUI interface. In src/run_layout.vala line 407 this path is picked up again and its content is interpreted in extractlayout_fromfile().

Since this file’s content is evaluated and used for further program logic there is a chance for a local attacker to massively break the run_layout program’s logic or maybe even achieve code execution. The Linux kernel’s protected_regular sysctl setting comes to the rescue here, though. The open() with O_CREAT will fail. It can then still present a denial-of-service vector, though.

Upstream Fix

This is fixed in upstream commit 11b0201. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

2) budgie-wpreviews (CVE-2023-49347)

2.1) Path /tmp/<user>_window-previews

This path is used for a directory. In src/separate_shot.vala line 43 it is created, errors are ignored. In line 105 screenshots of certain X11 windows are placed in the directory following the name scheme <window-id>.<workspace-name>.png.

In src/previews_creator.vala line 74 an attempt to create the directory the same way is found. In line 241 the directory is iterated over and each file found there, independently of its name, will be assembled in a file list. This file list is luckily only used for removing files of non-existent windows in this program.

In src/previews_daemon.vala line 719 there is another attempt to create the directory the same way as in the other two locations. In line 523 the directory is again iterated over and a list of the contained filenames is assembled, independently of their names. In line 404 the filenames are interpreted and split into X11 window IDs and workspace names again. It seems the code expects all filenames to match the pattern, if this is not the case then the program will likely crash. The resulting file list is (luckily) matched against the existing X11 window IDs in line 421.

Even without exploiting the fixed temporary directory path this directory has security issues, since it is created world-readable. Any other users in the system can access the window screenshots that are created there and thus this is an information leak.

Since all errors trying to create the directory are ignored, another local user can pre-create this directory world-writable, and the wpreviews applications will still use the directory which is now under attacker control. The attacker can place additional PNG image files there, trying to confuse the victim’s GUI experience. A local DoS against the previews_daemon seems also possible by placing non-conforming files into the directory. Since the previews_daemon only uses files from the directory for which an existing X11 window is found, the complexity for a local attacker to inject arbitrary PNG files into the preview logic is raised. It can still be possible by observing the PNG files created by e.g. the separate_shot program and replacing them with crafted data.

Without the Linux kernel’s symlink protection a local attacker can place a symlink there instead of a directory, causing the programs to operate in arbitrary other directory locations.

2.2) Paths /tmp/<user>_prvtrigger_*, /tmp/<user>_previoustrigger, /tmp/<user>_nexttrigger

This long list of trigger files:

/tmp/<user>_prvtrigger_all
/tmp/<user>_prvtrigger_current
/tmp/<user>_prvtrigger_all_hotcorner
/tmp/<user>_prvtrigger_curr_hotcorner
/tmp/<user>_previoustrigger
/tmp/<user>_prvtrigger_all
/tmp/<user>_nexttrigger

is used both in src/previews_triggers.vala line 43 and src/previews_daemon.vala line 664.

The previews_triggers program selects one of these trigger paths depending on command line arguments, various logical evaluations and depending on whether some of the paths already exist. The selected path is then simply created with empty content.

In previews_daemon these paths are monitored and their existence is evaluated in a complex fashion to display previews of existing windows.

In conjunction with the issues in 2.1) this can be used to display attacker controlled images on the victim’s screen at arbitrary times, provided that the victim user is running the previews_daemon.

Apart from the security related problems this group of files for controlling a daemons behaviour seems ill devised. Instead proper IPC mechanisms should be used.

Upstream Fix

This is fixed in upstream commit 588cbe6. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

3) budgie-takeabreak (CVE-2023-49345)

3.1) Path /tmp/nextbreak_<user>

This file is read in budgie_takeabreak.py line 245 and the resulting string is split on “.”, the first element resulting from this is used as the new “time” displayed in the GUI.

In takeabreak_run line 80 this path is created and the next “break time” is written to it.

A local attacker can pre-create this file and have arbitrary string content displayed instead of the actual “next time”. A denial-of-service will also be possible e.g. by placing a FIFO there.

Upstream Fix

This is fixed in upstream commit 588cbe6. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

4) budgie-weathershow (CVE-2023-49346)

4.1) Path /tmp/<username>_weatherdata

In src/weathershow/WeatherShow.vala line 354 the current “weather data” is written to this location. Before this an attempt is made to delete an already existing file. Errors for both, deletion and creation of the file, are ignored unconditionally.

In line 236 the content from this file is read and interpreted for updating GUI window data.

A local attacker can pre-create this file and thus manipulate the data displayed by the weather applet. Also a denial-of-service will be possible e.g. by placing a FIFO there.

Upstream Fix

This is fixed in upstream commit 0092025. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

5) budgie-clockworks (CVE-2023-49342)

This issue was not discovered by me but by upstream while working on the fixes for the other issues I reported. For completeness I mention it in this report as well.

5.1) Path /tmp/<user>_clockworks

This path is used as a directory in the Python script cwtools.py. It is reused if it already exists. The scripts generates SVG vector graphics in there, converts them to the PNG image format and saving them in the users home directory in ~/.config/budgie-extras/clockworks.

Here, again, the image data can be manipulated by a local attacker by pre-creating this directory. In this case the data will even be persisted in the user’s home directory. Crafted SVG of PNG data could be placed in the directory to try attacking the image processing libraries used.

Upstream Fix

This is fixed in upstream commit d030837. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

6) budgie-dropby (CVE-2023-49343)

Like issue 5), this issue was not discovered by me but by upstream while working on the other issues I reported. For completeness I mention it in this report as well.

6.1) Path /tmp/<user>_keepdropbywin

This path is used as a “timer” file in the checkonwin and dropover Python scripts. The file’s content is not evaluated, but the checkonwin script runs wmctrl -c dropby_popup if the file doesn’t exist for more than six seconds.

The Python openat() call uses O_CREAT | O_EXCL flags so symlink attacks are not a problem even without kernel symlink protection. Other users in the system can shorten the “timer” logic of the checkonwin script, though, by creating the file path at an arbitrary time.

6.2) Path /tmp/<user>_call_dropby

The script budgie_dropby.py creates this file as a trigger for the dropover script which reacts to the creation of this path by scanning the current list of USB block devices and their mount points in the system. A GUI dialog is displayed or updated as a reaction to this.

A local attacker can cause this dialog to be displayed by creating this file. It can also be used as a kind of local DoS vector to keep the dropover script busy all the time, iterating over block devices.

6.3) Path /tmp/<user>_dropby_icon_copy

This is used as a trigger file in budgie_dropby.py. If the file is created then a GUI dialog is changed and updated. In the copy_flash Python script this trigger is created to signal that some files have been copied.

A local attacker can cause this dialog to be displayed by creating this file at arbitrary times.

Upstream Fix

This is fixed in upstream commit e75c94a. The public /tmp directory has been replaced by the user’s private $XDG_RUNTIME_DIR, with a fallback to the user’s home directory.

7) Timeline

8) References

• budgie-extras GitHub repository
• budgie-extras v1.7.1 bugfix release
• budgie-extras v1.7.0 tag: this was the code based used in this review
• bugfix commit for window-shuffler application
• bugfix commit for the wpreviews application
• bugfix commit for the takeabreak application
• bugfix commit for the weathershow application
• bugfix commit for the clockworks application
• bugfix commit for the dropby application