Some important changes are afoot in librsvg.

Changes to continuous integration

Some days ago, Dunja Lalic rewrote the continuous integration scripts to be much faster. A complete pipeline used to take about 90 minutes to run, now it takes about 15 minutes on average.

The description of changes is interesting. The idea is to make tests fail as fast as possible, close to the beginning of the pipeline. To speed up the whole pipeline, Dunja did the following:

• Move the cargo check stage to the beginning. This test means, "does this even have a chance of compiling?".

• Have the code style and formatting tests, cargo clippy and cargo fmt, run in parallel with the unit tests. These lints can fail, but they are easy to fix after one is finished modifying the main code.

• Run the unit tests and the smoke tests in debug mode, so they compile quickly.

• Run the complete integration test suite in release mode. This takes longer to compile, but there are some slow tests that benefit a lot from faster execution.

• Move the release tests until the end, and only run them once a week — or whenever, by hand. These take a good amount of time to run, because they do a full make distcheck and autotools is slow. Even then, now these tests take 30-40 minutes, instead of the 90 from before.

• Between each stage of the pipeline, don't cache what doesn't help reduce compilation time. It seems that keeping around a big cache, with the whole build target, between each pipeline stage can be worse than not having one at all.

Test suite in Rust

Beteen Sven Neumann, Dunja Lalic, and myself we have finally ported the test suite to Rust: all of librsvg's tests are now in Rust, except for the C API tests. We had to do a few things:

• Review the old tests and remove some obsolete ones.

• Port each of the test modules to Rust. They are small, but each one has special little things — test for crashes in the XML loading code, test for crashes during rendering, test the library's security limits.

• Fix the small tests that come as part of the documentation.

• Untangle the reference tests and port them to Rust.

• Move little chunks of code around so the unit tests and integration tests can share utilities to compare images, compute file paths for test fixtures, etc.

The most complicated thing to port was the reference tests. These are the most important ones; each test loads an SVG document, renders it, and compares the result to a reference PNG image. There are some complications in the tests; they have to create a special configuration for Fontconfig and Pango, so as to have reproducible font rendering. The pango-rs bindings do not cover this part of Pango, so we had to do some things by hand.

Anyway, the tests are now in Rust. One nice thing is that now the tests run automatically in parallel, across all CPU cores, so we save on total testing time.

What's next: cargo-c and publish to crates.io

We want to be able to publish librsvg in crates.io as a normal crate; this implies being able to compile, test, and publish entirely from Cargo. The compilation and testing part is done.

Now, we have to reorganize the code so it can be published to crates.io. Librsvg comes in three parts, rsvg_internals with the implementation of the library, librsvg with the traditional C API, and librsvg_crate with the Rust API. However, to publish the Rust API to crates.io, it would be more convenient to have a single crate instead of one with the internals and one with the API.

The next step is thus to reorganize the code:

• Make it possible to implement the C API as a compile-time option on top of the normal Rust code. We want to use cargo-c to compile the traditional shared library librsvg.so, instead of depending on C tools for compiling and linking.

• Combine rsvg_internals and librsvg_crate in a single crate, to publish them together. Crates.io has a 10 MB limit per crate; now that the test suite lives in a separate tests crate, this shouldn't be a problem.

• I would like to polish the public error types before publishing the Rust API; right now they expose some implementation details that are of no interest to callers of the library.

What remains to be ported to Rust?

Only two things, which amount to less than 900 lines of C code:

• rsvg-convert - the command-line program that everyone uses to convert SVG to PNG and other formats. Fortunately, Sven Neumann wrote some fantastic tests for rsvg-convert, as it is like an API that we need to keep stable: if we change the command-line options or the program's behavior, we would break everyone's scripts.

• The gdk-pixbuf module for loading SVG. Alberto Ruiz has started porting it to Rust. The generic part of this code could later serve to wrap other Rust image codecs and plug them to gdk-pixbuf.

I did a presentation during the virtual Hacktoberfest Mauritius on Friday night, 23 October 2020. I had the freedom to choose a topic. So, I decided to talk about setting up Flatpak on openSUSE.

What is Flatpak?

Flatpak is a system for building, distributing, and running sandboxed desktop applications. It was developed in 2015 by Alex Larsson and is currently maintained by an independent community of developers. It is written in C and its source code is available on GitHub.

Installation process is straightforward using zypper.

zypper in flatpak

The flatpak utility can then be used at the command line to add a repository. In our case, we are going to add the flathub repository.

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo

We specify the option remote-add to indicate that we wish to add a repository, --if-not-exists to execute the command only if the repo does not exist on the system already, then followed by a name for the repo. We named the repo flathub but we could have called it something else as well, e.g myfavoriterepo. Lastly, we specify the url of the repository.

Next, we check whether the repo has been added.

ish@coffee-bar:~> flatpak remotes
Name    Options
flathub system

We can now start searching and installing apps from the flathub repo.

ish@coffee-bar:~> flatpak search postman
Name             Description                                                 Application ID                  Version          Branch         Remotes
Postman          Postman is a complete API development environment.          com.getpostman.Postman          7.31.0           stable         flathub

We install the application by specifying the Application ID.

flatpak install com.getpostman.Postman

The necessary icons and application shorcuts will be created as the installation completes. Therefore, launching the application from the desktop will follow the usual steps.

However, if you wish to run the application from the command line, e.g Postman, you cannot do so by simply typing postman at the terminal prompt. It should be run as follows:

flatpak run com.getpostman.Postman

Sandbox

With Flatpak, each application is built and run in an isolated environment. By default, the application can only 'see' itself and its runtime. Access to user files, network, graphics sockets, subsystems on the bus and devices have to be explicitly granted.

Under the hood

Flatpak uses a number of pre-existing technologies. It generally isn’t necessary to be familiar with these in order to use Flatpak, although in some cases it might be useful. They include:

• The bubblewrap utility from Project Atomic, which lets unprivileged users set up and run containers, using kernel features such as:
  - Cgroups
  - Namespaces
  - Bind mounts
  - Seccomp rules
• systemd to set up cgroups for sandboxes
• The OSTree system for versioning and distributing filesystem trees

Saamiyah pinged me a few days ago about the Hacktoberfest event that she was organising and asked whether I would be free to present a topic. Sure, why not?

As many tech meetups at the moment, the Hacktoberfest event also was virtual. It was hosted on the Jitsi instance of the Mauritius Software Craftsmanship Community. The event was scheduled to start at 19h30 on Friday, i.e last evening. I was late to join but "luckily" so was everybody.

Local hacktoberfest is live 😊 #mauritius pic.twitter.com/qFoNmFDn78

— Saamiyah 🇲🇺 (@smearthelove) October 23, 2020

As the event started, we had some small-talk before the presentations really kick-off. We talked about flexi-time, work-from-(home|anywhere), and how they compare to being full-time in office.

Saamiyah did the first presentation in which she talked about the concept of Hacktoberfest, a month where open source software is celebrated. Sandeep spoke about local open source projects on GitHub.

I spoke about Flatpak and how to set it up on openSUSE. I concluded my presentation with an observation on the rise of Flatpak fuelled by the idea of having immutable & maintenance-free systems such as Fedora Silverblue and openSUSE MicroOS Desktop.

The presentation slides are available at speakerdeck.com.

The next speaker, Alan, spoke about Docker Swarm, followed by Pritvi who talked about software licenses. It was 22h00 by that time and I could not stay longer. I bid goodnight on the chat room and wished everyone to continue having fun!

21st Noodling of jam packed excitement… not really. This is the 21st hot-pocket-sized podcast that won’t scorch roof of your mouth. I have a small collection of vintage or near vintage gaming consoles. I lean mostly in the Nintendo party as I think they have a great grasp on what is fun. I don’t always … Continue reading Noodlings 21 | Inspiration Is Around You →

Dear Tumbleweed users and hackers,

During this week, we have only released 3 snapshots (1019, 1021, and 1022). a bunch of snapshots has been tested and discarded due to some bugs we, and surely either you, did not want to see on your machines. But as usual; lesser snapshots do not mean less change, as things just cumulate until we feel confident to send a snapshot out again.

The three snapshot contained, amidst others, these changes:

• Mesa 20.2.1
• openSSL 1.1.1h
• Mozilla Thunderbird 78.3.3
• Linux kernel 5.8.15, incl. backported fix for bleeding tooth
• Ruby 2.7.2
• KDE Plasma 5.20.1

This leaves the staging projects left with these planned changes:

• jsoncpp 1.9.4: libyui fixes are ready
• nasm 2.15.5: Addresses the breakage seen last week with nasm 2.15.4
• Mozilla Firefox 82.0
• Cups 2.3.3
• GStreamer 1.18.0
• Ruby 2.6 will be removed from Tumbleweed (ruby2.6-rubygem-* is already gone, now the interpreter will disappear too)
• PostgreSQL 13
• Linux kernel 5.9.1
• GNOME 3.38.1: mozjs78 is now available in Tumbleweed
• openssl 3.0 (long-term; no progress in the last few weeks)
• RPM 4.16: steady progress made with package fixes.

The past week has delivered two openSUSE Tumbleweed snapshot.

Some of the package updates in the snapshots include newer versions of Node.js, OpenSSL, Mesa, Apparmor, ImageMagick and AutoYaST.

The latest snapshot, 20201021, is trending stable at a 98 rating on the Tumbleweed snapshot reviewer. This snapshot updated Mozilla Thunderbird to version 78.3.3 and improved support for encrypting with subkeys with OpenPGP. The new email client version also added support for wayland mode/autodetection in a startup wrapper. The security kernel module Apparmor added missing permissions to several profiles and abstractions. The 5.9 version of ethtool arrived in the snapshot and improved compatibility between system call ioctl and netlink output. The Linux Kernel updated to 5.8.15 and fixed a close proximity Common Vulnerabilities and Exposure, CVE-2020-12352, that could allow a remote attacker in adjacent range to use the flaw to leak small portions of stack memory by sending specially crafted Bluetooth AMP Packets. Node.js 14.14.0 had some bug fixes and a few changes like the behaviour of a new fs.rm method that follows the UNIX rm command. The update of the ruby2.7 package to 2.7.2 turned off deprecation warnings by default.

The 20201019 updated several RubyGems; also known as Rails version 6.0.3.4, the gems’ update addressed CVE-2020-8264, which was a XSS vulnerability while the application server was in development mode. ImageMagick had a small update in the 7.0.10.34 version to check for linux-compatible sendfiles. Mesa and Mesa-drivers were updated to version 20.2.1; the graphics library includes Intel Rocket Lake Platform Support. NetworkManager 1.26.4 added support for the DHCPv4 vendor class identifier options and fixed peer group tracking of Wi-Fi P2P connections. GNU’s bison parser updated to version 3.7.3; the bison executable is no longer linked uselessly against libreadline. AutoYaST has a few changes from its previous version in the rolling release and the minor release fixed the progress bar length during autoinstallation initialization. OpenSSL’s 1.1.1h version enabled ‘MinProtocol’ and ‘MaxProtocol’ to configure both TLS and DTLS contexts. YaST had several package updates including an update to yast2-firewall 4.3.6, which warns users when the SSH port is closed or when the service is disabled and the configured authentication is only based on the SSH key. Other notable packages to update in the snapshot were libstorage-ng translations, pipewire 0.3.13, qrencode 4.1.1 and vim 8.2.1840, which included several fixes and a few that addressed crashes of the text editor. The snapshot is trending stable at a 97 rating.

During the openSUSE + LibreOffice Virtual Conference, there were 2 presentations on what’s next for openSUSE Leap. These presentations also touched on Closing the Leap Gap. This is a project which tries to resolve / minimize the differences in packages between openSUSE Leap and SUSE Linux Enterprise (SLE), by unifying the code base and the development process. More details on this project can be found here.

On the 20th October, there was a Go-No Go decision to be made. This decision is documented on the wiki, the engineering release notes and on the openSUSE Factory mailing list. There was a Conditional No Go given on the proposal to create an in-between release called openSUSE Leap 15.2.1. That means that the Jump and Leap unification will most likely happen in Leap 15.3. I think that this is a reasonable decision, which provides a better timeline for the openSUSE and SUSE teams to work out all of the outstanding issues.

As an openSUSE Leap user you will likely not notice any change. Things in Leap will continue to work just fine. But from Leap 15.3, the migration from Leap to SLE will be much easier to make. This means that openSUSE Leap will become a direct competitor to CentOS in terms of being an Enterprise Grade Linux Distribution. And for SLE customers, it will be much safer to use (Leap) community packages on top of SLE.

Published on: 22 October 2020

Source for the header image can be found here and is licensed under the Creative Commons Attribution 2.0 Generic license. The image is edited for use on this website.

The Open Build Service Connector extension brings the Open Build Service into Visual Studio Code!

Everything Starts With a Bookmark

That’s right. The Open Build Service Connector is built around bookmarks of packages and projects. Bookmarks can be used to browse a project, its packages and its files. Additionally, you can view the configured repositories and adjust project paths and architectures.

Individual packages or whole projects can be checked out directly from within Visual Studio Code to the file system similarly as one would do via osc. OBS’ version control is seamlessly integrated into Visual Studio Code’s Source Control module and can be used in a comparable fashion to the git extension.

Branch, Build and Collaborate

One of the cornerstones of the openSUSE project has always been collaboration, which is also fundamentally built into the Open Build Service. The Open Build Service Connector follows this philosophy and allows you to branch any bookmarked packages, check them out locally, modify them, build them via osc and submit your changes back as a submitrequest.

Contribute

We need your help! Give the extension a try, check out if it fits your use case and give us feedback over on GitHub.

In case you want to access OBS API via Node.js or a even a browser, then you can use the new wrapper library open-build-service-api.

My inbox tells me I started using GMail around 2004. The oldest mail I can find in my archive is from 16 years ago. After Gmail, Google Photos, Keep, Docs, Drive and Fit followed.

I have reasons to stop. Whether your reasons are privacy, the U.S. as a data harbor, GMail becoming sluggish, karma for killing Inbox, fear about getting your account locked, or you found a better email provider, the objective of this post is not to convince you about my reasons but to help you with a migration plan and showing you alternatives.

Breaking the dependency on Google services is really hard. This dependency was a showstopper and motivator at the same time. If you are locked-in at this level, something is wrong.

After 16 years, I was not planning to stop in a single day, but step by step. This post highlights the first steps into that direction.

My new requirements are:

• My data stored in a privacy compatible jurisdiction: European Union, Nordic countries, Switzerland.
• Managed/hosted services are OK, as long they are in a privacy respecting jurisdiction and I pay for the product (I am not the product).
• Services should use open-source software where possible.

Migrating away from GMail

After evaluating several mail providers including Tutanota, ProtonMail, Mailfence, Soverin, Runbox, I decided for https://mailbox.org:

• Provides IMAP and can be used with generic existing open-source clients
• Company has a focus on privacy
• Attractive price
• Located in Germany
• Based on Open-Xchange, which is Open-Source
• Provides a Calendar feature based on open standards (CalDAV)
• Provides encryption in several forms
• Company values: privacy, eco-friendly, and work-life balance for their employees align with my own

Mailbox.org is not perfect. 2FA is bolted-on like many other parts of the application. I don’t think you can beat Google when it comes to security, but the risk of getting your account locked at Google and no escalation path or human to talk to makes all the technicallities irrelevant.

To migrate, I planned to use mbsync, which I already use to download my work email in my mu4e/Emacs setup. The idea is to create two channels, one for GMail, one for the new provider, download the whole GMail archive (forcing pull in a sync), and then force a push on the new provider.

Downloading all my mail with mbsync did not work. GMail has download limits for IMAP. The next thing to try was Google Takeout, a service that allows you to dowload your Google data. This gave me an mbox with all my GMail messages. mbsync only works with Maildir, so I tried to upload the mbox messages with Thunderbird, but did not get far. At the end, I used mb2md to convert the mbox to Maildir format, and then used mbsync to upload the messages to the new provider. This worked.

In order to prevent lock-in in the future, I used a custom domain. My go-to registrar is Namecheap and I have no complaints. I went with Gandi, as they are based in France and I read good things about them.

To be able to migrate at my own pace, I setup a forward and delete filter rule in Gmail. I had hundred of accounts using my email address as username. Thankfully, my password manager knows about those and I changed the ones I use more often. Every time I get a newsletter or notification, I take the chance to unsubscribe, and check the To: field and update my profile, or delete that account.

I replaced the GMail mobile application with FairMail. The build is not free as in beer ($), but it is Open-Source (GPL). Paying to get a working binary and some support is worth it.

Google Search

I switched to DuckDuckGo a long time ago already. I tried Quant (based in France) but for some reason it takes seconds to connect. I can’t believe they fail in this obvious detail.

Migrating Google Photos & Google Drive

My usual workflow has been to download photos locally and upload to Google Photos. The lack of a good sync mechanism resulted in glitches over time. Some albums were present locally and some existed only in Google Photos.

I used both gohotos-sync and Google Takeout to get two full copies of albums and the photo stream. gphotos-sync has a useful flag --compare-folder which hels comparing albums in Google Photos with the local version of those, creating symlinks for local and remote missing files.

I then used exiftool to sort pictures further. If you don’t know this tool, I highly recommned you learn it.

I selected Hetzner Storage Share, an affordable Nextcloud based service hosted in Germany. Nextcloud is intended to replace Google Drive, which means it allows to share files via eg. public links. You can, however, install many applications in it, including a very simple photo gallery.

The feature I will miss the most is to be able to do AI based search on my photos. I search often by keywords and concepts.

I setup the Linux and mobile clients. The Linux client syncs a part of my Pictures folder that is ready and organized. I configured Instant-Upload on my phone which auto-uploads photos I take with the camera. The upload is unidirectional, but as they land on a folder I have configured to be synced with my computer, they reach my laptop to be further organized. I can delete the camera files without risk of losing what has been uploaded.

I still depend on Drive for sharing files with my band. I relegated Drive to its own Firefox Container, this way I am not permanently logged into the Google Account as I browse the Web, but do not need to log-in again to use Drive.

Google Keep

For personal notes, I use org-mode on a synced folder. I sync the folder to my Nextcloud instance. Orgzly provides a TODO widget and access the files via WebDAV. Orgro gives you a more sophisticated viewer.

I do share a shopping list with my family in Keep and I haven’t yet solved that problem. I have thought about a Keep-like view for Orgzly -it is open-source-, by transforming each headline into a card.

Google Fit

I track my runs in Fit. My ideal solution would be to store tracks directly as a file in a NextCloud folder. An alternative is to store them in a internal database and do an Export from time to time.

Google Takeout allows you to export tracks in TCX format, with summaries as CSV files. I ended with 300+ TCX files.

I evaluated many apps that required no Cloud service. RunnerUp, ForRunners and FitoTrack are also Open-Source, where Sportractive is not.

FitoTrack and Sportractive where the most promissing ones. In both apps I could not import more than one file at a time so I contacted the authors asking for tips how to import my data. Sportractive author mentioned this was not possible. FitoTrack author found this a simple addition, implemented it and pointed me to the next release. Due to a glitch, took longer to show up in the Play Store, but I built the app from source and started experimenting with this feature.

To convert the TCX files to GPX I used gpsbabel. FitoTrack has trouble with Fit multiple laps/tracks. The pack option in gpsbabel merges them.

for fn in ../*.tcx; do gpsbabel -i gtrnctr -f "$fn" -x track,pack -o gpx -F $(basename $fn .tcx).gpx; done

I had now 300+ files with names like 2018-04-15T00_44_22+02_00_PT38M17.962S_Running.gpx, no description and no metadata specifying it was “Running”.

I hacked this script which finds the starting point, does reverse geolocation to find the place name, cleans it up and then renames the file. It also sets the description to something like “Run in Madrid, Spain”.

import os
import time
import unidecode

import gpxpy
import gpxpy.gpx
from geopy.geocoders import Nominatim

geolocator = Nominatim(user_agent="JustATestScript")

for filename in os.listdir("."):
    if not filename.endswith(".gpx"):
        continue

    print("Current: {}".format(filename))
    gpx_file = open(filename, "r")
    gpx = gpxpy.parse(gpx_file)

    # get first point
    point = None
    try:
        point = gpx.tracks[0].segments[0].points[0]
    except Exception:
        print(" `-> No point 0")
        continue

    location = geolocator.reverse(
        (point.latitude, point.longitude),
        language="en",
        addressdetails=True,
    )
    country = unidecode.unidecode(location.raw["address"]["country"])
    # City is not so easy. Fallback until we get something
    city = None
    for place in ["city", "village", "suburb", "town"]:
        if place not in location.raw["address"]:
            continue
        import re
        city = re.sub(r".+/\s+", "", location.raw["address"][place])
        city = unidecode.unidecode(city)
        break
    if not city:
        raise Exception("No place in address: {}".format(location.raw))
    newname = "{}-Running-{}_{}.gpx".format(
        point.time.strftime("%Y-%m-%d_T%H_%m"),
        city.replace(" ", "_"),
        country.replace(" ", "_"),
    )
    print(" `-> new name: {}".format(newname))
    gpx.tracks[0].name = "Run in {}, {}".format(city, country)
    gpx.tracks[0].description = None
    # FIXME: does not serialize. Fix with xmlstarlet
    gpx.tracks[0].type = "running"
    with open(filename, "w") as out:
        out.write(gpx.to_xml())
    try:
        os.rename(filename, newname)
    except Exception as e:
        print(location.raw)
        raise e
    # do not call the API too fast
    time.sleep(1)

The result was:

2018-07-29_T08_07-Munich_Germany.gpx
2019-08-10_T14_08-Barcelone_Spain.gpx
2020-08-22_T07_08-Warsaw_Poland.gpx
2020-07-11_T15_07-Nuremberg_Germany.gpx
2020-08-20_T06_08-Valencia_Spain.gpx
2018-06-03_T10_06-Stuttgart_Germany.gpx
...

(city names and dates are not the real ones)

Setting the sport type in the metadata did not get serialized back, so I fix it with xmlstarlet:

xmlstarlet ed --inplace -N x="http://www.topografix.com/GPX/1/0" -s /x:gpx/x:trk -t elem -n type -v "running" *.gpx

Then, mass import into FitoTrack and I got all my activities with nice descriptions and the right “Running” icon.

Conclusions

My new mail setup is working for some weeks already without problems. I miss some Photos features, but that’s it. I was not expecting Fit to take that much effort.

In general, I am happy with the results. I regained control of my data and I got to use more open-source.

This is a mostly useless blathering but since I got a good laugh out of it, I wanted to index this bit of fun and share it because that is what you do, right? Share nonsense on the Internet? Isn’t that why they invented the thing? I was watching “Adrian’s Digital Basement” on YouTube and … Continue reading Rickroll in the Terminal →