setting up my system for my new job

equipment

I am starting my new job at Suse on Tuesday. As if to build the excitement over the holidays, they send me a boxes with equipment to get started. This gave me a good opportunity to clear off my desk and set up for a clean start.

They sent me a Thinkpad, a docking station, keyboard and mouse, and headphones. So, I set up all the equipment and made sure everything was comfortable to work with.

software set up

installing opensuse

It was recommended to me to install Leap, with I was happy with because I have been using it over my break on some other computers that I have. So I have already installed and set it up twice, and I had a USB key for it laying around.

I turned off Secure Boot on my laptop. I forgot all about the Secure Boot mess, and my BeeLink computers didn’t require my to make any configuration changes to install OpenSuse. Anyway, it was pretty easy to get the installer kicked off.

While the installer was running, I went about my day, and came back later delighted to see it on the Leap login page.

I logged in and was up and running.

I opened settings to see what wasn’t working, but everything so far was configured perfectly. Even the 4k display was set up properly, and I could tweak the settings if I wanted.

opensuse choice of install options

Three applications that I use everyday are Chrome, Visual Studio Code, and Slack. I also wanted to install Zoom. I don’t think that Suse the company uses Zoom, but other folks do, so I wanted to have it ready.

This is an area where I think Suse really shines. Suse is not trying to force me down a path of installing and using software specifically from their repositories in their specific packaging format. My order of preference for running apps is to:

1. Set up a repository and install from there.
2. Download an archive and install it locally.
3. Download a built binary and run it locally.

I am not interested in FlatPak or Snappy packages. For people who do like those formats, they seem to work fine on OpenSuse, but it turns out that I could follow my personal preferences when I was getting set up.

installing chrome

For better or worse, I am pretty embedded into the Google ecosystem. As such, the first thing I do with any computer is install Chrome and log into my Google account. Of course, I have to use Firefox once to find and install Chrome.

The way I did it was to download the rpm, and click on it.

This opened YaST2 for me, and it was a simple matter of clicking buttons to get Chrome installed.

Then I logged in and made sure I could get to my email and omg.lol account, and all was good.

installing vscode

Code is nice because they let me follow my #1 preference. They had clear instructions for Suse on their downloads page. Just add the repo, and use zypper to install the package.

Worked like a dream:

install slack

For Slack, I downloaded the rpm from the slack download page, double clicked on it, and YaST2 installed it with no issues:

Also “just worked”:

install and run zoom

For some reason, I assumed that installing Zoom was going to be a mess, but … on the contrary, I visited their (linux download page)[https://zoom.us/download?os=linux] and could easily select an archive for my system:

I double clicked on the archive, went through YaST2 again, and it worked as smooth as silk:

webcam and headset

Suse included Plantronics headset with my equipment. It took me a lot of fiddling to get it working on Beelink, but it worked out of the box on Thinkpad. My Webcam also “just worked.”

I watched some videos to make sure all was well, and I had a great 4k experience with good sound.

The ownCloud product Infinite Scale is going to be released in version five soon. The latest stable version is 4.0.5 and I am sure everybody checked it out already and is blown away by it’s performance, elegance and ease of use.

No, not yet?

Ok, well, in that case, here comes rescue: With the little script described here, it becomes really easy to start Infinite Scale to check it out on your computer. It makes it really easy and quick for you, without Linux super admin powers whatsoever.

To use it, you just need to open a terminal on your machine and cd into a directory somewhere in your home where you can afford to host some bytes.

Without further preparation, you type the following command line (NOT as user root please):

curl -L https://owncloud.com/runocis.sh | /bin/bash

What it does is that it automatically pulls the latest stable version of Infinite Scale from the official download server of ownCloud onto your computer. For that, it creates a configuration and a start script, and starts the server. The script detects the platform on which your’re running to download the right binary version. It also looks up the hostname and configures the installation for that name.

Once the server was started Infinite Scale’s web client can be accessed by pointing a browser to the URL https://your-hostname:9200/. Since this is an installation for testing purposes, it does not have a proper certificate configured. That is why your browser is complaining about the cert, and you have to calm it. And indeed, that is one of the reasons why you’re not supposed to use this sneak peak in production or even exposed to the internet.

For the nerds, the script does not really do magic, but just curls the golang single binary of Infinite Scale down to the machine into a sandbox directory, chmod it to be executable and create a working config and a data dir. All happens with the priviledges of the logged in user, no sudo or root involved. You’re encouraged to double check the install script using for example the command curl -L https://owncloud.com/runocis.sh | less - of course you never should trust anybody running scripts from the internet on your machine.

If the server is stopped by pressing Ctrl-C, it later can be started again by the script runocis.sh that was kindly left behind in the sandbox as well.

The installer was tested on these three platforms: 64 bit AMD/Intel CPU based Linux machines, 64 bit Raspberry Pi with Raspbian OS and MacOSX. The flavour of Linux should not make a difference.

If you encounter a problem with the script or if you have suggestions to improve, please find it in my this’n that section on Github. I am happy to receive issue reports or pull requests.

For further information and setups suitable for production please refer to the Infinite Scale documentation.

More Shoots Development

I have not just been 3D printing and wrapping Christmas gifts this week, I've been writing software as well!

A couple of weeks ago I put up my first bit of code for shoots.

I think the best way that I can describe it is that I am developing a data warehouse for pandas developers.

When I first posted the code, you could:

1. Put a dataframe on the server.
2. Get a dataframe from the server.
3. Query a dataframe on the server using SQL.
4. List dataframes.

Since then I've added:

1. Organize dataframes in buckets.
2. Tell the server where to save the dataframes.
3. More and better tests.
4. Documentation.
5. Ability to resample data on the server in 2 ways.

I think that the project is starting to get useful. There are a few key things that I think are needed it so that I can really recommend using it:

1. Security - support TLS on the server, require TLS on the client.
2. Security - generate an admin token required to do anything on the server.
3. Concat dataframes on the server.
4. Clean dataframes on the server (sort, remove dupes, etc...).

After getting those core things done, I am thinking:

1. Ability to store the data in object storage instead of on disk.
2. Ability to run Python to run on the server to avoid round trips to the client for things like alerting and such.

Shoots Dataframe Storage

I just posted some code for a project that I have been working on very part time in the last few weeks.

Shoots is dataframe storage server. Currently is supports pandas, but most likely I will add support for polars in the future.

Shoots is entirely written in Python and is designed for Python users.

Shoots comes in 2 parts, a server and a client library.

Shoots is very early software, but is in a usable state. It is stored on github. Issues and contributions welcome.

shoots_server

The server tries to be a fairly faithful Apache Flight Server, meaning that you should be able to use the Apache Arrow Flight client libraries directly. It is entirely built upon the upstream Apache Arrow project.

Under the hood, the server receives and serves pandas dataframes, storing thenm on disk in Apache Parquet format. However, shoots is designed so that, as a user, you don't need to know about the underlying storage formats and libraries.

shoots_client

The client pieces wrap the Apache FlightClient to offer an interface for pandas developers, abstracting away the Apache Arrow and Flight concepts.

usage

run the server

There are currently no run time options, so running the server is a simple matter of running the python module, depending on your system:

python shoots_server.py

or

python3 shoots_server.py

storing a dataframe

Use the client library to create an instance of the client, and "put" a dataframe. Assuming you are running locally:

from shoots_client import ShootsClient
from shoots_client import PutMode
import pandas as pd

shoots = ShootsClient("localhost", 8081)
df = pd.read_csv('sensor_data.csv')

shoots.put("sensor_data", dataframe=df, mode=PutMode.REPLACE)

retrieving data

You can simply get a dataframe back by using its name:

df0 = shoots.get("sensor_data")
print(df0)

You can also submit a sql query to bring back a subset of the data:

sql = 'select "Sensor_1" from sensor_data where "Sensor_2" < .2'
df1 = shoots.get("sensor_data", sql=sql)
print(df1)

Shoots use Apache DataFusion for executing SQL. The DataFusion dialect is well document.

listing dataframes

You can retrieve a list of dataframes and their schemas, using the list() method.

results = shoots.list()
print("dataframes stored:")
for r in results:
    print(r["name"])

dataframes stored:
sensor_data

deleting dataframes

You can delete a dataframe using the delete() method:

shoots.delete("sensor_data")

buckets

You can organize your dataframes in buckets. This is essentially a directory where your dataframes are stored.

creating buckets

Buckets are implicitly created as needed if you use the "bucket" parameter in put():

shoots.put("sensor_data", dataframe=df, mode=PutMode.REPLACE, bucket="my-bucket")
df1 = shoots.get("sensor_data", bucket="my-bucket")
print(df1)

listing buckets

You can use the buckets() method to list available buckets:

print("buckets:")
print(shoots.buckets())

buckets:
['my-bucket', 'foo']

deleting buckets

You can delete buckets with the delete_bucket() method. You can force a deletion of all contents using BucketDeleteMode.DELETE_CONTENTS:

print("buckets before deletion:")
print(shoots.buckets())
shoots.delete_bucket("my-bucket", mode=BucketDeleteMode.DELETE_CONTENTS)
print("buckets after deletion:")
print(shoots.buckets())

buckets before deletion:
['my-bucket', 'foo']
buckets after deletion:
['foo']

Roadmap

I intend to work on the following in the coming weeks, in no particular order:

• [ ] add a runtime option for the root bucket directory, use it for testing
• [ ] document code and generate docs
• [ ] pip packaging
• [ ] pattern matching for list()
• [ ] downsampling via sql on the server
• [ ] combining dataframes on the server
• [ ] compressing and cleaning dataframes on teh server
• [ ] authentication

Starting with version 2.58.0, librsvg will no longer use gdk-pixbuf to decode raster images that are referenced from SVG documents. For example, an <image> element like this:

<image href="foo.jpg" width="100" height="100"/>

I have just pushed a merge request to make librsvg use the image-rs crate to decode raster images. This is part of two related changes:

• Don't load SVG sub-documents with gdk-pixbuf. For historical reasons, librsvg's original C code did not even bother to detect if <image href="foo.svg"/> referenced another SVG document; it would just ask gdk-pixbuf to render it as for any other image. This works more or less fine in SVG1.1, but for SVG2 we actually have to pay attention to the attributes in the <image> element and the child SVG document, and it's just easier to recurse into librsvg directly.

• Don't load raster images with gdk-pixbuf. Now that Loupe is using the same Rust crates to decode raster images, I think we can begin to move the rest of the platform to not using memory-unsafe codecs.

Librsvg still compiles and installs the gdk-pixbuf loader that lets other applications render SVG documents as if they were raster images; nothing is changed there. The C APIs that create GdkPixbuf objects also remain unchanged.

Testers wanted

As you can imagine, this is the sort of change that gives me a bit of anxiety. Say whatever you want about memory-unsafe code in libpng and libjpeg-turbo, but they are tested and fuzzed all the way to hell, all the time. The Rust crates for decoding images have not been as heavily developed, and there is still plenty of interesting work to do there in terms of performance and support for the more exotic variants of those file formats. I think this is a good opportunity to find exactly what they might be lacking.

Please test the main branch of librsvg, especially if you render documents which have <image> elements! The changes above are easy to roll back or to make optional if too much trouble appears.

Dear Tumbleweed users and hackers,

This is going to be the last weekly review – for this year. Yes, you’re right: there is one more week left for the year, but I’ll try to stay away from Computers during this time. Ana will make sure to keep Tumbleweed rolling though. She’s been the main driver for the last few months, meaning there is no change.

During the last week, a total of 6 snapshots (1214, 1215, 1217, 1218, 1219, and 1221) have been published (one was held back by openQA and discarded).and brought you these changes:

• kudmp 2.0.0
• cURL 8.5.0
• SQLite 3.44.2
• Boost 1.84.0
• KDE Frameworks 5.113.0
• Node.JS 21.4.0
• OpenSSH: CVE-2023-48795 fixed
• cmake 3.28.1
• Sudo 1.9.15p4
• ppp 2.5.0
• Kalpa patterns and configuration included (images pending)
• More work for systemd-boot / full-disk-encryption, see https://news.opensuse.org/2023/12/20/systemd-fde/

Progress in staging is unsurprising and a bit slow, as many people are stepping away from the computers and dedicating their time differently. The following things are still being worked on:

• Mozilla Firefox 121.0
• Appstream 1.0.0
• RPM 4.19.x: llvm17 fails to build (?)
• Ruby 3.3 tests have started (yast failures identified so far)
• libxml 2.12.x: slow progress
• openSSL 3.2.0
• c-ares 1.21.0: breaks nodejs
• wxWidgets 3.2.3: breaks wxPython bindings
• dbus-broker: no progress: openQA fails to launch the network stack in the installer

More Low Drama Printing

I am just continually amazed by how forgiving my Mini Delta is. I ordered some bright green filament so I could play with some Suse logos, and I created a model with 4 of them.

The print bed was a bit dirty, with some pieces of brim from old prints that I couldn't easily get off. I just went with it. Everything printed well, and I love how they turned out.

Just need to remove the supports for the hook part. I'll wait until the new Suse logo situation if figured out before I experiment more.

It's vacation season in Europe and most of the YaST development team will be busy for a couple of weeks singing Chistmas songs, celebrating the arrival of a new year and opening presents from the Three Wise Men. But we didn't want to leave the openSUSE fans idle for so long. So we decided to release another prototype of Agama, so you can have fun testing it and giving us feedback.

Meet Agama 7 and its corresponding Agama Live ISO images for testing.

Localization settings for the target system​

One of the most significant changes compared to previous versions is the possibility of fully configuring all the localization aspects of system to be installed. That includes the language, the keyboard layout and the time zone.

Note these aspects don't affect the Agama interface. The language used by Agama during the installation process is determined by the browser. However, it can be changed through the Agama sidebar as already explained in our previous blog post.

Keyboard selector for local installation​

But what happens when the browser you use to connect to Agama is not in a remote machine properly configured but is the full-screen browser offered by the Agama Live testing image? Don't worry; your localization needs are also covered.

As you can see in the description of this pull request, you can use the Agama sidebar to change the keyboard layout if the browser is running in the same system executing Agama, as is the case of Agama Live.

We plan to create a proper welcome screen for users who don't know enough English to reach that sidebar. Meanwhile we hope the current options will be enough to use Agama in your preferred language and get a correctly localized system as result of the installation.

Reduce RAM usage in the testing image​

Talking about the Agama Live testing image, we recently reduced the amount of RAM it requires to install openSUSE. In fact, Agama's memory usage has mostly stayed the same, but we enabled zram by default in the live image and now everything should work better in systems with a limited amount of RAM. If you want to disable zram, just open a terminal and execute the following command.

systemctl stop zramswap

To be honest, we cannot foresee any good reason to disable zram, so let us know if you think we are mistaken.

Installation of openSUSE MicroOS​

We also reorganized the Agama configuration included in the openSUSE flavor of Agama Live. Now you can use the very same testing image to install openSUSE Tumbleweed and openSUSE MicroOS.

First step to simplify selection of software​

In Agama 5 we introduced a software patterns selector to allow the user to customize, to some extent, which software to install. It was just a prototype but it unveiled some troubles:

• There are way too many patterns and it is easy to get lost.
• As some products share the same repositories (i.e., Tumbleweed and MicroOS), you might mix patterns from both products.

To mitigate those problems, Agama displays now a curated list of patterns for each product. However, the current implementation is still a prototype, as we plan to refine the approach and add more features, like proper conflict handling.

Registration support​

As you may know, some SUSE products are expected to be registered in the SUSE Customer Center using a product key. Agama now supports this use case, although with some limitations. The most relevant one is that it does not support any proxy of the SUSE Customer Center (e.g., Repository Mirroring Tool).

Another constraint of the current implementation is that it does not support products with optional registration.

However, there are plans to remove those limitations sooner than later.

More customization options​

The storage area got its share of regular enhancements too. The first noticeable change is that you can tell Agama how to make space to install the system. Do you want to remove the current content and use the whole disk? Or do you prefer to shrink existing partitions to make space? Or would you like to use only the available space? We expect these options to cover most of the use cases, although we plan to add support for a more fine-grained approach.

Another significant change in the storage area is allowing users to select the file system for each mount point. It includes enabling or disabling snapshots if you want to use Btrfs.

Network bonding​

The network configuration is one of the areas we want to focus on the short term. Until now, Agama only supports simple use cases over wired and wireless devices. However, it is limited if you use a network-based storage device (e.g., iSCSI) to install the system.

For that reason, in close collaboration with our network experts at SUSE, we are adding support for more scenarios. In this release, you can set up a bonding connection, although this feature is limited by now to the unattended installation case. The example below shows how to define the connection.

{
  "id": "Bonding Connection 1",
  "interface": "bond",
  "bond": {
    "ports": ["eth0", "eth1"],
    "mode": "active-backup",
    "options": "primary=eth1"
  }
}

Other improvements​

• Preserve the installation logs for easier error reporting and debugging.
• Make it possible to translate the messages in the installation progress screen. Our superb team of translators is already working on them.
• Several accessibility improvements in the web user interface by adding aria-label, aria-live-region and aria-busy attributes where needed.

The year of Agama​

Agama 7 is the first prototype we could consider to be "functional enough". It finally covers all the areas we consider as essencial during the installation of the system: localization, network configuration, storage setup, authentication basis (including the optional creation of a first user) and an intentionally simplistic selection of the software to install.

Of course, all those areas need to evolve much further and that's exactly the goal for this new year - take Agama to the next level in which it could even replace YaST for some scenarios and distributions.

Your contributions and opinions are an important element for that. So don't hesitate to contact the YaST team at the YaST Development mailing list, our #yast channel at Libera.chat or the Agama project at GitHub.

See you again in 2024!

It’s vacation season in Europe and most of the YaST development team will be busy for a couple of weeks singing Chistmas songs, celebrating the arrival of a new year and opening presents from the Three Wise Men. But we didn’t want to leave the openSUSE fans idle for so long. So we decided to release another prototype of Agama, so you can have fun testing it and giving us feedback.

Meet Agama 7 and its corresponding Agama Live ISO images for testing.

Localization Settings for the Target System

One of the most significant changes compared to previous versions is the possibility of fully configuring all the localization aspects of system to be installed. That includes the language, the keyboard layout and the time zone.

Note these aspects don’t affect the Agama interface. The language used by Agama during the installation process is determined by the browser. However, it can be changed through the Agama sidebar as already explained in our previous blog post.

Keyboard Selector for Local Installation

But what happens when the browser you use to connect to Agama is not in a remote machine properly configured but is the full-screen browser offered by the Agama Live testing image? Don’t worry; your localization needs are also covered.

As you can see in the description of this pull request, you can use the Agama sidebar to change the keyboard layout if the browser is running in the same system executing Agama, as is the case of Agama Live.

We plan to create a proper welcome screen for users who don’t know enough English to reach that sidebar. Meanwhile we hope the current options will be enough to use Agama in your preferred language and get a correctly localized system as result of the installation.

Reduce RAM Usage in the Testing Image

Talking about the Agama Live testing image, we recently reduced the amount of RAM it requires to install openSUSE. In fact, Agama’s memory usage has mostly stayed the same, but we enabled zram by default in the live image and now everything should work better in systems with a limited amount of RAM. If you want to disable zram, just open a terminal and execute the following command.

systemctl stop zramswap

To be honest, we cannot foresee any good reason to disable zram, so let us know if you think we are mistaken.

Installation of openSUSE MicroOS

We also reorganized the Agama configuration included in the openSUSE flavor of Agama Live. Now you can use the very same testing image to install openSUSE Tumbleweed and openSUSE MicroOS.

First Step to Simplify Selection of Software

In Agama 5 we introduced a software patterns selector to allow the user to customize, to some extent, which software to install. It was just a prototype but it unveiled some troubles:

• There are way too many patterns and it is easy to get lost.
• As some products share the same repositories (i.e., Tumbleweed and MicroOS), you might mix patterns from both products.

To mitigate those problems, Agama displays now a curated list of patterns for each product. However, the current implementation is still a prototype, as we plan to refine the approach and add more features, like proper conflict handling.

Registration Support

As you may know, some SUSE products are expected to be registered in the SUSE Customer Center using a product key. Agama now supports this use case, although with some limitations. The most relevant one is that it does not support any proxy of the SUSE Customer Center (e.g., Repository Mirroring Tool).

Another constraint of the current implementation is that it does not support products with optional registration.

However, there are plans to remove those limitations sooner than later.

More Customization Options

The storage area got its share of regular enhancements too. The first noticeable change is that you can tell Agama how to make space to install the system. Do you want to remove the current content and use the whole disk? Or do you prefer to shrink existing partitions to make space? Or would you like to use only the available space? We expect these options to cover most of the use cases, although we plan to add support for a more fine-grained approach.

Another significant change in the storage area is allowing users to select the file system for each mount point. It includes enabling or disabling snapshots if you want to use Btrfs.

Network Bonding

The network configuration is one of the areas we want to focus on the short term. Until now, Agama only supports simple use cases over wired and wireless devices. However, it is limited if you use a network-based storage device (e.g., iSCSI) to install the system.

For that reason, in close collaboration with our network experts at SUSE, we are adding support for more scenarios. In this release, you can set up a bonding connection, although this feature is limited by now to the unattended installation case. The example below shows how to define the connection.

{
  "id": "Bonding Connection 1",
  "interface": "bond",
  "bond": {
      "ports": ["eth0", "eth1"],
      "mode": "active-backup",
      "options": "primary=eth1"
  }
}

Other Improvements

• Preserve the installation logs for easier error reporting and debugging.
• Make it possible to translate the messages in the installation progress screen. Our superb team of translators is already working on them.
• Several accessibility improvements in the web user interface by adding aria-label, aria-live-region and aria-busy attributes where needed.

The Year of Agama

Agama 7 is the first prototype we could consider to be “functional enough”. It finally covers all the areas we consider as essencial during the installation of the system: localization, network configuration, storage setup, authentication basis (including the optional creation of a first user) and an intentionally simplistic selection of the software to install.

Of course, all those areas need to evolve much further and that’s exactly the goal for this new year - take Agama to the next level in which it could even replace YaST for some scenarios and distributions.

Your contributions and opinions are an important element for that. So don’t hesitate to contact the YaST team at the YaST Development mailing list, our #yast channel at Libera.chat or the Agama project at GitHub.

See you again in 2024!