The initial articles about librsvg's conversion to Rust are in my old blog, so they may be a bit hard to find from this new blog. Here is a list of those posts, just so they are easier to find:

• Librsvg gets Rusty
• Porting a few C functions to Rust
• Bézier curves, markers, and SVG's concept of directionality
• Refactoring C to make Rustification easier
• Exposing Rust objects to C code
• Debugging Rust code inside a C library
• Reproducible font rendering for librsvg's tests
• Algebraic data types in Rust, and basic parsing
• How librsvg exports reference-counted objects from Rust to C
• Griping about parsers and shitty specifications
• Porting librsvg's tree of nodes to Rust
• gboolean is not Rust bool

Within this new blog, you can look for articles with the librsvg tag.

Before continuing with the glib-rs architecture, let's take a detour and look at GObject Introspection. Although it can seem like an obscure part of the GNOME platform, it is an absolutely vital part of it: it is what lets people write GNOME applications in any language.

Let's start with a bit of history.

Brief history of language bindings in GNOME

When we started GNOME in 1997, we didn't want to write all of it in C. We had some inspiration from elsewhere.

Prehistory: GIMP and the Procedural Database

There was already good precedent for software written in a combination of programming languages. Emacs, the flagship text editor of the GNU project, was written with a relatively small core in C, and the majority of the program in Emacs Lisp.

In similar fashion, we were very influenced by the design of the GIMP, which was very innovative at that time. The GIMP has a large core written in C. However, it supports plug-ins or scripts written in a variety of languages. Initially the only scripting language available for the GIMP was Scheme.

The GIMP's plug-ins and scripts run as separate processes, so they don't have immediate access to the data of the image being edited, or to the core functions of the program like "paint with a brush at this location". To let plug-ins and scripts access these data and these functions, the GIMP has what it calls a Procedural Database (PDB). This is a list of functions that the core program or plug-ins wish to export. For example, there are functions like gimp-scale-image and gimp-move-layer. Once these functions are registered in the PDB, any part of the program or plug-ins can call them. Scripts are often written to automate common tasks — for example, when one wants to adjust the contrast of photos and scale them in bulk. Scripts can call functions in the PDB easily, irrespective of the programming language they are written in.

We wanted to write GNOME's core libraries in C, and write a similar Procedural Database to allow those libraries to be called from any programming language. Eventually it turned out that a PDB was not necessary, and there were better ways to go about enabling different programming languages.

Enabling sane memory management

GTK+ started out with a very simple scheme for memory management: a container owned its child widgets, and so on recursively. When you freed a container, it would be responsible for freeing its children.

However, consider what happens when a widget needs to hold a reference to another widget that is not one of its children. For example, a GtkLabel with an underlined mnemonic ("_N_ame:") needs to have a reference to the GtkEntry that should be focused when you press Alt-N. In the very earliest versions of GTK+, how to do this was undefined: C programmers were already used to having shared pointers everywhere, and they were used to being responsible for managing their memory.

Of course, this was prone to bugs. If you have something like

typedef struct {
    GtkWidget parent;

    char *label_string;
    GtkWidget *widget_to_focus;
} GtkLabel;

then if you are writing the destructor, you may simply want to

static void
gtk_label_free (GtkLabel *label)
{
    g_free (label_string);
    gtk_widget_free (widget_to_focus);          /* oops, we don't own this */

    free_parent_instance (&label->parent);
}

Say you have a GtkBox with the label and its associated GtkEntry. Then, freeing the GtkBox would recursively free the label with that gtk_label_free(), and then the entry with its own function. But by the time the entry gets freed, the line gtk_widget_free (widget_to_focus) has already freed the entry, and we get a double-free bug!

Madness!

That is, we had no idea what we were doing. Or rather, our understanding of widgets had not evolved to the point of acknowledging that a widget tree is not a simply tree, but rather a directed graph of container-child relationships, plus random-widget-to-random-widget relationships. And of course, other parts of the program which are not even widget implementations may need to keep references to widgets and free them or not as appropriate.

I think Marius Vollmer was the first person to start formalizing this. He came from the world of GNU Guile, a Scheme interpreter, and so he already knew how garbage collection and seas of shared references ought to work.

Marius implemented reference-counting for GTK+ — that's where gtk_object_ref() and gtk_object_unref() come from; they eventually got moved to the base GObject class, so we now have g_object_ref() and g_object_unref() and a host of functions to have weak references, notification of destruction, and all the things required to keep garbage collectors happy.

The first language bindings

The very first language bindings were written by hand. The GTK+ API was small, and it seemed feasible to take

void gtk_widget_show (GtkWidget *widget);
void gtk_widget_hide (GtkWidget *widget);

void gtk_container_add (GtkContainer *container, GtkWidget *child);
void gtk_container_remove (GtkContainer *container, GtkWidget *child);

and just wrap those functions in various languages, by hand, on an as-needed basis.

Of course, there is a lot of duplication when doing things that way. As the C API grows, one needs to do more and more manual work to keep up with it.

Also, C structs with public fields are problematic. If we had

typedef struct {
    guchar r;
    guchar g;
    guchar b;
} GdkColor;

and we expect program code to fill in a GdkColor by hand and pass it to a drawing function like

void gdk_set_foreground_color (GdkDrawingContext *gc, GdkColor *color);

then it is no problem to do that in C:

GdkColor magenta = { 255, 0, 255 };

gdk_set_foreground_color (gc, &magenta);

But to do that in a high level language? You don't have access to C struct fields! And back then, libffi wasn't generally available.

Authors of language bindings had to write some glue code, in C, by hand, to let people access a C struct and then pass it on to GTK+. For example, for Python, they would need to write something like

PyObject *
make_wrapped_gdk_color (PyObject *args, PyObject *kwargs)
{
    GdkColor *g_color;
    PyObject *py_color;

    g_color = g_new (GdkColor, 1);
    /* ... fill in g_color->r, g, b from the Python args */

    py_color = wrap_g_color (g_color);
    return py_color;
}

Writing that by hand is an incredible amount of drudgery.

What language bindings needed was a description of the API in a machine-readable format, so that the glue code could be written by a code generator.

The first API descriptions

I don't remember if it was the GNU Guile people, or the PyGTK people, who started to write descriptions of the GNOME API by hand. For ease of parsing, it was done in a Scheme-like dialect. A description may look like

(class GtkWidget
       ;;; void gtk_widget_show (GtkWidget *widget);
       (method show
               (args nil)
               (retval nil))

       ;;; void gtk_widget_hide (GtkWidget *widget);
       (method hide
               (args nil)
               (retval nil)))

(class GtkContainer
       ;;; void gtk_container_add (GtkContainer *container, GtkWidget *child);
       (method add
               (args GtkWidget)
               (retval nil)))

(struct GdkColor
        (field r (type 'guchar))
        (field g (type 'guchar))
        (field b (type 'guchar))) 

Again, writing those descriptions by hand (and keeping up with the C API) was a lot of work, but the glue code to implement the binding could be done mostly automatically. The generated code may need subsequent tweaks by hand to deal with details that the Scheme-like descriptions didn't contemplate, but it was better than writing everything by hand.

Glib gets a real type system

Tim Janik took over the parts of Glib that implement objects/signals/types, and added a lot of things to create a good type system for C. This is where things like GType, GValue, GParamSpec, and fundamental types come from.

For example, a GType is an identifier for a type, and a GValue is a type plus, well, a value of that type. You can ask a GValue, "are you an int? are you a GObject?".

You can register new types: for example, there would be code in Gdk that registers a new GType for GdkColor, so you can ask a value, "are you a color?".

Registering a type involves telling the GObject system things like how to copy values of that type, and how to free them. For GdkColor this may be just g_new() / g_free(); for reference-counted objects it may be g_object_ref() / g_object_unref().

Objects can be queried about some of their properties

A widget can tell you when you press a mouse button mouse on it: it will emit the button-press-event signal. When GtkWidget's implementation registers this signal, it calls something like

    g_signal_new ("button-press-event",
        gtk_widget_get_type(), /* type of object for which this signal is being created */
        ...
        G_TYPE_BOOLEAN,  /* type of return value */
        1,               /* number of arguments */
        GDK_TYPE_EVENT); /* type of first and only argument */

This tells GObject that GtkWidget will have a signal called button-press-event, with a return type of G_TYPE_BOOLEAN, and with a single argument of type GDK_TYPE_EVENT. This lets GObject do the appropriate marshalling of arguments when the signal is emitted.

But also! You can query the signal for its argument types! You can run g_signal_query(), which will then tell you all the details of the signal: its name, return type, argument types, etc. A language binding could run g_signal_query() and generate a description of the signal automatically to the Scheme-like description language. And then generate the binding from that.

Not all of an object's properties can be queried

Unfortunately, although GObject signals and properties can be queried, methods can't be. C doesn't have classes with methods, and GObject does not really have any provisions to implement them.

Conventionally, for a static method one would just do

void
gtk_widget_set_flags (GtkWidget *widget, GtkWidgetFlags flags)
{
    /* modify a struct field within "widget" or whatever */
    /* repaint or something */
}

And for a virtual method one would put a function pointer in the class structure, and provide a convenient way to call it:

typedef struct {
    GtkObjectClass parent_class;

    void (* draw) (GtkWidget *widget, cairo_t *cr);
} GtkWidgetClass;

void
gtk_widget_draw (GtkWidget *widget, cairo_t *cr)
{
    GtkWidgetClass *klass = find_widget_class (widget);

    (* klass->draw) (widget, cr);
}

And GObject has no idea about this method — there is no way to query it; it just exists in C-space.

Now, historically, GTK+'s header files have been written in a very consistent style. It is quite possible to write a tool that will take a header file like

/* gtkwidget.h */
typedef struct {
    GtkObject parent_class;

    void (* draw) (GtkWidget *widget, cairo_t *cr);
} GtkWidgetClass;

void gtk_widget_set_flags (GtkWidget *widget, GtkWidgetFlags flags);
void gtk_widget_draw (GtkWidget *widget, cairo_t *cr);

and parse it, even if it is with a simple parser that does not completely understand the C language, and have heuristics like

• Is there a class_name_foo() function prototype with no corresponding foo field in the Class structure? It's probably a static method.

• Is there a class_name_bar() function with a bar field in the Class structure? It's probably a virtual method.

• Etc.

And in fact, that's what we had. C header files would get parsed with those heuristics, and the Scheme-like description files would get generated.

Scheme-like descriptions get reused, kind of

Language binding authors started reusing the Scheme-like descriptions. Sometimes they would cannibalize the descriptions from PyGTK, or Guile (again, I don't remember where the canonical version was maintained) and use them as they were.

Other times they would copy the files, modify them by hand some more, and then use them to generate their language binding.

C being hostile

From just reading/parsing a C function prototype, you cannot know certain things. If one function argument is of type Foo *, does it mean:

• the function gets a pointer to something which it should not modify ("in" parameter)

• the function gets a pointer to uninitialized data which it will set ("out" parameter)

• the function gets a pointer to initialized data which it will use and modify ("inout" parameter)

• the function will copy that pointer and hold a reference to the pointed data, and not free it when it's done

• the function will take over the ownership of the pointed data, and free it when it's done

• etc.

Sometimes people would include these annotations in the Scheme-like description language. But wouldn't it be better if those annotations came from the C code itself?

GObject Introspection appears

For GNOME 3, we wanted a unified solution for language bindings:

• Have a single way to extract the machine-readable descriptions of the C API.

• Have every language binding be automatically generated from those descriptions.

• In the descriptions, have all the information necessary to generate a correct language binding...

• ... including documentation.

We had to do a lot of work to accomplish this. For example:

• Remove C-isms from the public API. Varargs functions, those that have foo (int x, ...), can't be easily described and called from other languages. Instead, have something like foov (int x, int num_args, GValue *args_array) that can be easily consumed by other languages.

• Add annotations throughout the code so that the ad-hoc C parser can know about in/out/inout arguments, and whether pointer arguments are borrowed references or a full transfership of ownership.

• Take the in-line documentation comments and store them as part of the machine-readable description of the API.

• When compiling a library, automatically do all the things like g_signal_query() and spit out machine-readable descriptions of those parts of the API.

So, GObject Introspection is all of those things.

Annotations

If you have looked at the C code for a GNOME library, you may have seen something like this:

/**
 * gtk_widget_get_parent:
 * @widget: a #GtkWidget
 *
 * Returns the parent container of @widget.
 *
 * Returns: (transfer none) (nullable): the parent container of @widget, or %NULL
 **/
GtkWidget *
gtk_widget_get_parent (GtkWidget *widget)
{
    ...
}

See that "(transfer none) (nullable)" in the documentation comments? The (transfer none) means that the return value is a pointer whose ownership does not get transferred to the caller, i.e. the widget retains ownership. Finally, the (nullable) indicates that the function can return NULL, when the widget has no parent.

A language binding will then use this information as follows:

• It will not unref() the parent widget when it is done with it.

• It will deal with a NULL pointer in a special way, instead of assuming that references are not null.

Every now and then someone discovers a public function which is lacking an annotation of that sort — for GNOME's purposes this is a bug; fortunately, it is easy to add that annotation to the C sources and regenerate the machine-readable descriptions.

Machine-readable descriptions, or repository files

So, what do those machine-readable descriptions actually look like? They moved away from a Scheme-like language and got turned into XML, because early XXIst century.

The machine-readable descriptions are called GObject Introspection Repository files, or GIR for short.

Let's look at some parts of Gtk-3.0.gir, which your distro may put in /usr/share/gir-1.0/Gtk-3.0.gir.

<repository version="1.2" ...>

  <namespace name="Gtk"
             version="3.0"
             shared-library="libgtk-3.so.0,libgdk-3.so.0"
             c:identifier-prefixes="Gtk"
             c:symbol-prefixes="gtk">

For the toplevel "Gtk" namespace, this is what the .so library is called. All identifiers have "Gtk" or "gtk" prefixes.

A class with methods and a signal

Let's look at the description for GtkEntry...

    <class name="Entry"
           c:symbol-prefix="entry"
           c:type="GtkEntry"
           parent="Widget"
           glib:type-name="GtkEntry"
           glib:get-type="gtk_entry_get_type"
           glib:type-struct="EntryClass">

      <doc xml:space="preserve">The #GtkEntry widget is a single line text entry
widget. A fairly large set of key bindings are supported
by default. If the entered text is longer than the allocation
...
       </doc>

This is the start of the description for GtkEntry. We already know that everything is prefixed with "Gtk", so the name is just given as "Entry". Its parent class is Widget and the function which registers it against the GObject type system is gtk_entry_get_type.

Also, there are the toplevel documentation comments for the Entry class.

Onwards!

      <implements name="Atk.ImplementorIface"/>
      <implements name="Buildable"/>
      <implements name="CellEditable"/>
      <implements name="Editable"/>

GObject classes can implement various interfaces; this is the list that GtkEntry supports.

Next, let's look at a single method:

      <method name="get_text" c:identifier="gtk_entry_get_text">
        <doc xml:space="preserve">Retrieves the contents of the entry widget. ... </doc>

        <return-value transfer-ownership="none">
          <type name="utf8" c:type="const gchar*"/>
        </return-value>

        <parameters>
          <instance-parameter name="entry" transfer-ownership="none">
            <type name="Entry" c:type="GtkEntry*"/>
          </instance-parameter>
        </parameters>
      </method>

The method get_text and its corresponding C symbol. Its return value is an UTF-8 encoded string, and ownership of the memory for that string is not transferred to the caller.

The method takes a single parameter which is the entry instance itself.

Now, let's look at a signal:

      <glib:signal name="activate" when="last" action="1">
        <doc xml:space="preserve">The ::activate signal is emitted when the user hits
the Enter key. ...</doc>

        <return-value transfer-ownership="none">
          <type name="none" c:type="void"/>
        </return-value>
      </glib:signal>

    </class>

The "activate" signal takes no arguments, and has a return value of type void, i.e. no return value.

A struct with public fields

The following comes from Gdk-3.0.gir; it's the description for GdkRectangle.

    <record name="Rectangle"
            c:type="GdkRectangle"
            glib:type-name="GdkRectangle"
            glib:get-type="gdk_rectangle_get_type"
            c:symbol-prefix="rectangle">

      <field name="x" writable="1">
        <type name="gint" c:type="int"/>
      </field>
      <field name="y" writable="1">
        <type name="gint" c:type="int"/>
      </field>
      <field name="width" writable="1">
        <type name="gint" c:type="int"/>
      </field>
      <field name="height" writable="1">
        <type name="gint" c:type="int"/>
      </field>

    </record>

So that's the x/y/width/height fields in the struct, in the same order as they are defined in the C code.

And so on. The idea is for the whole API exported by a GObject library to be describable by that format. If something can't be described, it's a bug in the library, or a bug in the format.

Making language bindings start up quickly: typelib files

As we saw, the GIR files are the XML descriptions of GObject APIs. Dynamic languages like Python would prefer to generate the language binding on the fly, as needed, instead of pre-generating a huge binding.

However, GTK+ is a big API: Gtk-3.0.gir is 7 MB of XML. Parsing all of that just to be able to generate gtk_widget_show() on the fly would be too slow. Also, there are GTK+'s dependencies: Atk, Gdk, Cairo, etc. You don't want to parse everything just to start up!

So, we have an extra step that compiles the GIR files down to binary .typelib files. For example, /usr/lib64/girepository-1.0/Gtk-3.0.typelib is about 600 KB on my machine. Those files get mmap()ed for fast access, and can be shared between processes.

How dynamic language bindings use typelib files

GObject Introspection comes with a library that language binding implementors can use to consume those .typelib files. The libgirepository library has functions like "list all the classes available in this namespace", or "call this function with these values for arguments, and give me back the return value here".

Internally, libgirepository uses libffi to actually call the C functions in the dynamically-linked libraries.

So, when you write foo.py and do

import gi
gi.require_version('Gtk', '3.0')
from gi.repository import Gtk
win = Gtk.Window()

what happens is that pygobject calls libgirepository to mmap() the .typelib, and sees that the constructor for Gtk.Window is a C function called gtk_window_new(). After seeing how that function wants to be called, it calls the function using libffi, wraps the result with a PyObject, and that's what you get on the Python side.

Static languages

A static language like Rust prefers to have the whole language binding pre-generated. This is what the various crates in gtk-rs do.

The gir crate takes a .gir file (i.e. the XML descriptions) and does two things:

• Reconstructs the C function prototypes and C struct declarations, but in a way Rust can understand them. This gets output to the sys crate.

• Creates idiomatic Rust code for the language binding. This gets output to the various crates; for example, the gtk one.

When reconstructing the C structs and prototypes, we get stuff like

#[repr(C)]
pub struct GtkWidget {
    pub parent_instance: gobject::GInitiallyUnowned,
    pub priv_: *mut GtkWidgetPrivate,
}

extern "C" {
    pub fn gtk_entry_new() -> *mut GtkWidget;
}

And the idiomatic bindings? Stay tuned!

Virtualization systems like KVM, Xen, or z/VM offer the possibility to move running guests from one physical server to another one without service interruption. This is a cool feature for the system administrator because he can service a virtualization host without shutting down any workload on his cluster.

Before doing such a migration, the system perform a number of checks. Especially z/VM is very strict with this, but also gives high confidence that nothing bad happens with the workload. Unfortunately, the default system you get when running linux on z/VM has a number of devices attached, that prevent z/VM from relocating that guest to a different node. A typical message would look like this:

VMRELO TEST LINUX001 ZVM002
HCPRLH1940E LINUX001 is not relocatable for the following reason(s):
...
HCPRLI1996I LINUX001: Virtual machine device 0191 is a link to a local minidisk
...

For some of the devices it is obvious to the experienced z/VM admin that he can detach them. However some of the devices might also be in use by Linux, and it would definitly confuse the system when just removing the device. Therefore, the z/VM admin has to ask the person reponsible for Linux if it is ok to remove that device. When talking about 10 guests, this might be ok, but when talking about lots and lots of servers and many different Stakeholders, this can get quite painful.

Starting with SLES12 SP2, a new service called “virtsetup” sneaked into the system that can ease this task a lot. When enabled, it removes all the unneeded CMS disks from the guest and thus prepares the guest for live guest relocation.

How to run this service:
# systemctl enable virtsetup
# systemctl start virtsetup

Thats basically everything you have to do for a default setup. If you want some specific disk untouched, just have a look at “/etc/sysconfig/virtsetup”. This is the file where this service is configured.

Enabling this service is not a big deal for the single machine, but it makes a big difference for the z/VM admin. When this is enabled, most machines will simply be eligible for relocation without further action and thus allowing for continuous operation during service of a z/VM Node.

Image from https://events.opensuse.org

Yesterday, on September 3rd. Mr. Fuminobu Takeyama has contacted me about my proposal in openSUSE.Asia Summit 2017. I’m so happy when I read the email my proposal was accepted with openSUSE.Asia Summit committees. Furthermore, my happiness didn’t stop there. Two out of my 3 proposals got a High Score! There are “Implement Active Directory Server for Single Sign-on Login Using openSUSE” and “Have Fun Claim Control your Docker Images with Portus on openSUSE Leap“.

openSUSE.Asia Summit itself is one of the great events for openSUSE community (i.e., both contributors, and users) in Asia. Those who usually communicate online can get together from all over the world, talk face to face, and have fun. Members of the community will share their most recent knowledge, experiences, and learn FLOSS technologies surrounding openSUSE.

This is the fourth of openSUSE Asia Summit. It held every year in different countries. Beforehand, it held in Beijing on 2014, Taipei on 2015, Yogyakarta on 2016 and in this years, openSUSE Asia Summit will be held in Chofu, Tokyo, Japan.

So, Mr. Takeyama asks me about which talk do I wanna choose. Indeed, both of the proposals got a high score but I have to choose one of them. Then, I choose Portus as my talk in openSUSE.Asia Summit 2017. Because he said to me, many Japan Engineers interested with Docker.

In this trip, I will fly away to Japan with different airlines. For departure, i will use Garuda Indonesia and Return with AirAsia and stay in a homestay with my friends from Indonesia. Mas Moko, Mas Ary, Mas Kukuh, Mbak Alin and Mbak Umul.

The main aim for me is to share with people how we can have a lot of fun with openSUSE. Get acquainted with a new people, contributor in Asia and many others. Finding a new idea for openSUSE Indonesia and recharging a spirit. I think this event will be awesome. I can’t wait for attending this event as a speaker since last years I’m only become to be a volunteer in openSUSE Asia Summit 2016, Yogyakarta. This will be a new amazing experience for me.

After taking a long process. Finally, I’m so happy my proposal has been accepted. Thank you openSUSE Indonesia, Om Edwin Zakaria, as one of the committees for openSUSE.Asia Summit for struggling Indonesian people papers and everyone who gives me for motivation. I realized my contribution in openSUSE is not enough, but i will give my best for the things what i love. I will give the best for openSUSE and openSUSE.Asia Summit 2017.

If you want to attend it, prepare your self. Registration will be open soon. Stay tune in https://events.opensuse.org and See you in Tokyo!

The post openSUSE.Asia Summit 2017 Tokyo, Japan. I’m Coming! appeared first on dhenandi.com.

As the 4.13 release has now happened, the merge window for the 4.14 kernel release is now open. I mentioned this many weeks ago, but as the word doesn’t seem to have gotten very far based on various emails I’ve had recently, I figured I need to say it here as well.

So, here it is officially, 4.14 should be the next LTS kernel that I’ll be supporting with stable kernel patch backports for at least two years, unless it really is a horrid release and has major problems. If so, I reserve the right to pick a different kernel, but odds are, given just how well our development cycle has been going, that shouldn’t be a problem (although I guess I just doomed it now…)

Eigentlich ist es ja gewünscht wenn das System einem regelmäßig die temporären Dateien wegräumt. Löscht es einem dabei aber auch Dateien welche man gerade erst dort abgelegt hatte, dann steht wohl irgendwas in der Konfiguration schief.

Ich nutze tatsächlich häufig das Verzeichnis /tmp. Gerade beim Schreiben kleiner Skripte ist es ganz angenehm sich im Fehlerfall einfach darauf zu verlassen, dass übrig gebliebene temporäre Dateien schon wieder verschwinden werden. Auch meine virtuellen Python-Umgebungen lege ich, seit mich Holger Peters mal auf diese Idee gebracht hat, stets dort ab. Seit einiger Zeit löschte mir meine openSUSE meine gerade angelegten Dateien aber immer kurz nach dem Systemstart unter der Nase weg.

Da das Problem immer kurz nach dem Systemstart auftrat war schnell klar, dass da wohl der Aufräumcronjob schuld sein muss. Auch wenn dies natürlich inzwischen gar kein Cronjob mehr ist sondern ein Systemd-Timer.

marix@eddie:~> systemctl list-timers
NEXT                         LEFT        LAST                         PASSED       UNIT                         ACTIVATES
Di 2017-09-05 21:10:12 CEST  22h left    Mo 2017-09-04 21:10:12 CEST  1h 9min ago  systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.serv
Mo 2017-09-11 00:00:00 CEST  6 days left Mo 2017-09-04 20:55:51 CEST  1h 23min ago fstrim.timer                 fstrim.service

Der alte Cronjob hatte eine Konfigurationsdatei in /etc/sysconfig in der man die Schonzeit für Dateien in /tmp und /var/tmp konfigurieren konnte. Beim Systemd-Job übernehmen die Konfigurationsdateien von systemd-tmpfiles diesen Job. Die dazugehörige Manpage ist tmpfiles.d (5).

Die Standardatei /usr/lib/tmpfiles.d/tmp.conf erklärt auch ganz brav diese Verzeichnisse in Ruhe zu lassen. Das ist zwar nicht ganz was ich möchte, aber auch nicht Grund meines Problemes.

# Clear tmp directories separately, to make them easier to override
# SUSE policy: we don't clean those directories
q /tmp 1777 root root -
q /var/tmp 1777 root root -

Die schuldige Konfiguration fand sich in /etc/tmpfiles.d/tmp.conf. Hier muss, bei einem der vielen Versionsupgrades welche mein System hinter sich hat, ein kleiner Übertragungsfehler passiert sein.

d /tmp 1777 root root 0d
d /var/tmp 1777 root root -
x /tmp/* - - - - root

Mit dem Alter Null Tage wird systemd-tmpfiles angewiesen alle Dateien zu löschen, egal wann sie zuletzt modifiziert wurden. Auch wenn dies die meißten Anwendungen in meinem System klaglos hinnahmen sorgte es natürlich für einiges WTF was meine eigenen Skripte und die die virtuellen Python-Umgebungen anging. Die korrektur des Wertes auf einen Tag hat mein Problem erfolgreich gelöst.

d /tmp 1777 root root 1d
d /var/tmp 1777 root root -
x /tmp/* - - - - root

Und das schöne an Systemd ist, dass man so eine Änderung dann auch gleich sehr leicht testen kann.

sudo systemctl start systemd-tmpfiles-clean.service

Eine Anwendung welche mit dem agressiven Aufräumen der temporären Dateien gar nicht klar kommt ist übrigens meine früher so geliebter Browser Opera. Nachdem die temporären Dateien weggräumt wurden findet dieser seine bereits laufende Instanz nicht mehr und fliegt dann beim Versuch ein zweites mal das gleiche Profil zu öffnen ordendlich auf die Fresse. Dies hat mich hinreichend genervt, so dass ich kürzlich zum Firefox gewechselt bin. Seit ich kurz darauf die hier beschriebene Fehlkonfiguration behoben habe funktioniert aber auch der, inzwischen von mir nicht mehr wirklich genutzte, Opera wieder tadellos.

Today I released librsvg 2.41.1, and it's a big release! Apart from all the Rust goodness, and the large number of bug fixes, I am very happy with the way the build system works these days. I've found it invaluable to have good examples of Autotools incantations to copy&paste, so hopefully this will be useful to someone else.

There are some subtleties that a "good" autotools setup demands, and so far I think librsvg is doing well:

• The configure script checks for cargo and rustc.

• "make distcheck" works. This means that the build can be performed with builddir != srcdir, and also that make check runs the available tests and they all pass.

• The rsvg_internals library is built with Rust, and our Makefile.am calls cargo build with the correct options. It is able to handle debug and release builds.

• "make clean" cleans up the Rust build directories as well.

• If you change a .rs file and type make, only the necessary stuff gets rebuilt.

• Etcetera. I think librsvg feels like a normal autotool'ed library. Let's see how this is done.

Librsvg's basic autotools setup

Librsvg started out with a fairly traditional autotools setup with a configure.ac and Makefile.am. For historical reasons the .[ch] source files live in the toplevel librsvg/ directory, not in a src subdirectory or something like that.

librsvg
├ configure.ac
├ Makefile.am
├ *.[ch]
├ src/
├ doc/
├ tests/
└ win32/

Adding Rust to the build

The Rust source code lives in librsvg/rust; that's where Cargo.toml lives, and of course there is the conventional src subdirectory with the *.rs files.

librsvg
├ configure.ac
├ Makefile.am
├ *.[ch]
├ src/
├ rust/         <--- this is new!
│ ├ Cargo.toml
│ └ src/
├ doc/
├ tests/
└ win32/

Detecting the presence of cargo and rustc in configure.ac

This goes in configure.ac:

AC_CHECK_PROG(CARGO, [cargo], [yes], [no])
AS_IF(test x$CARGO = xno,
    AC_MSG_ERROR([cargo is required.  Please install the Rust toolchain from https://www.rust-lang.org/])
)
AC_CHECK_PROG(RUSTC, [rustc], [yes], [no])
AS_IF(test x$RUSTC = xno,
    AC_MSG_ERROR([rustc is required.  Please install the Rust toolchain from https://www.rust-lang.org/])
)

These two try to execute cargo and rustc, respectively, and abort with an error message if they are not present.

Supporting debug or release mode for the Rust build

One can call cargo like "cargo build --release" to turn on expensive optimizations, or normally like just "cargo build" to build with debug information. That is, the latter is the default: if you don't pass any options, cargo does a debug build.

Autotools and C compilers normally work a bit differently; one must call the configure script like "CFLAGS='-g -O0' ./configure" for a debug build, or "CFLAGS='-O2 -fomit-frame-pointer' ./configure" for a release build.

Linux distros already have all the infrastructure to pass the appropriate CFLAGS to configure. We need to be able to pass the appropriate flag to Cargo. My main requirement for this was:

• Distros shouldn't have to substantially change their RPM specfiles (or whatever) to accomodate the Rust build.
• I assume that distros will want to make release builds by default.
• I as a developer am comfortable with passing extra options to make debug builds on my machine.

The scheme in librsvg lets you run "configure --enable-debug" to make it call a plain cargo build, or a plain "configure" to make it use cargo build --release instead. The CFLAGS are passed as usual through an environment variable. This way, distros don't have to change their packaging to keep on making release builds as usual.

This goes in configure.ac:

dnl Specify --enable-debug to make a development release.  By default,
dnl we build in public release mode.

AC_ARG_ENABLE(debug,
              AC_HELP_STRING([--enable-debug],
                             [Build Rust code with debugging information [default=no]]),
              [debug_release=$enableval],
              [debug_release=no])

AC_MSG_CHECKING(whether to build Rust code with debugging information)
if test "x$debug_release" = "xyes" ; then
    AC_MSG_RESULT(yes)
    RUST_TARGET_SUBDIR=debug
else
    AC_MSG_RESULT(no)
    RUST_TARGET_SUBDIR=release
fi
AM_CONDITIONAL([DEBUG_RELEASE], [test "x$debug_release" = "xyes"])

AC_SUBST([RUST_TARGET_SUBDIR])

This defines an Automake conditional called DEBUG_RELEASE, which we will use in Makefile.am later.

It also causes @RUST_TARGET_SUBDIR@ to be substituted in Makefile.am with either debug or release; we will see what these are about.

Adding Rust source files

The librsvg/rust/src directory has all the *.rs files, and cargo tracks their dependencies and whether they need to be rebuilt if one changes. However, since that directory is not tracked by make, it won't rebuild things if a Rust source file changes! So, we need to tell our Makefile.am about those files:

RUST_SOURCES =                   \
        rust/build.rs            \
        rust/Cargo.toml          \
        rust/src/aspect_ratio.rs \
        rust/src/bbox.rs         \
        rust/src/cnode.rs        \
        rust/src/color.rs        \
        ...

RUST_EXTRA =                     \
        rust/Cargo.lock

EXTRA_DIST += $(RUST_SOURCES) $(RUST_EXTRA)

It's a bit unfortunate that the change tracking is duplicated in the Makefile, but we are already used to listing all the C source files in there, anyway.

Most notably, the rust subdirectory is not listed in the SUBDIRS in Makefile.am, since there is no rust/Makefile at all!

Cargo release or debug build?

if DEBUG_RELEASE
CARGO_RELEASE_ARGS=
else
CARGO_RELEASE_ARGS=--release
endif

We will call cargo build with that argument later.

Verbose or quiet build?

Librsvg uses AM_SILENT_RULES([yes]) in configure.ac. This lets you just run "make" for a quiet build, or "make V=1" to get the full command lines passed to the compiler. Cargo supports something similar, so let's add it to Makefile.am:

CARGO_VERBOSE = $(cargo_verbose_$(V))
cargo_verbose_ = $(cargo_verbose_$(AM_DEFAULT_VERBOSITY))
cargo_verbose_0 =
cargo_verbose_1 = --verbose

This expands the V variable to empty, 0, or 1. The result of expanding that gives us the final command-line argument in the CARGO_VERBOSE variable.

What's the filename of the library we are building?

RUST_LIB=@abs_top_builddir@/rust/target/@RUST_TARGET_SUBDIR@/librsvg_internals.a

Remember our @RUST_TARGET_SUBDIR@ from configure.ac? If you call plain "cargo build", it will put the binaries in rust/target/debug. But if you call "cargo build --release", it will put the binaries in rust/target/release.

With the bit above, the RUST_LIB variable now has the correct path for the built library. The @abs_top_builddir@ makes it work when the build directory is not the same as the source directory.

Okay, so how do we call cargo?

@abs_top_builddir@/rust/target/@RUST_TARGET_SUBDIR@/librsvg_internals.a: $(RUST_SOURCES)
    cd $(top_srcdir)/rust && \
    CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo build $(CARGO_VERBOSE) $(CARGO_RELEASE_ARGS)

We make the funky library filename depend on $(RUST_SOURCES). That's what will cause make to rebuild the Rust library if one of the Rust source files changes.

We override the CARGO_TARGET_DIR with Automake's preference, and call cargo build with the correct arguments.

Linking into the main C library

librsvg_@RSVG_API_MAJOR_VERSION@_la_LIBADD = \
        $(LIBRSVG_LIBS)                      \
        $(LIBM)                              \
        $(RUST_LIB)

This expands our $(RUST_LIB) from above into our linker line, along with librsvg's other dependencies.

make check

This is our hook so that make check will cause cargo test to run:

check-local:
        cd $(srcdir)/rust && \
        CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo test

make clean

Same thing for make clean and cargo clean:

clean-local:
        cd $(top_srcdir)/rust && \
        CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo clean

Vendoring dependencies

Linux distros probably want Rust packages to come bundled with their dependencies, so that they can replace them later with newer/patched versions.

Here is a hook so that make dist will cause cargo vendor to be run before making the tarball. That command will creates a rust/vendor directory with a copy of all the Rust crates that librsvg depends on.

RUST_EXTRA += rust/cargo-vendor-config

dist-hook:
    (cd $(distdir)/rust && \
    cargo vendor -q && \
    mkdir .cargo && \
    cp cargo-vendor-config .cargo/config)

The tarball needs to have a rust/.cargo/config to know where to find the vendored sources (i.e. the embedded dependencies), but we don't want that in our development source tree. Instead, we generate it from a rust/cargo-vendor-config file in our source tree:

# This is used after `cargo vendor` is run from `make dist`.
#
# In the distributed tarball, this file should end up in
# rust/.cargo/config

[source.crates-io]
registry = 'https://github.com/rust-lang/crates.io-index'
replace-with = 'vendored-sources'

[source.vendored-sources]
directory = './vendor'

One last thing

If you put this in your Cargo.toml, release binaries will be a lot smaller. This turns on link-time optimizations (LTO), which removes unused functions from the binary.

[profile.release]
lto = true

Summary and thanks

I think the above is some good boilerplate that you can put in your configure.ac / Makefile.am to integrate a Rust sub-library into your C code. It handles make-y things like make clean and make check; debug and release builds; verbose and quiet builds; builddir != srcdir; all the goodies.

I think the only thing I'm missing is to check for the cargo-vendor binary. I'm not sure how to only check for that if I'm the one making tarballs... maybe an --enable-maintainer-mode flag?

This would definitely not have been possible without prior work. Thanks to everyone who figured out Autotools before me, so I could cut&paste your goodies:

Update 2017/Nov/11: Fixed the initialization of RUST_EXTRA; thanks to Tobias Mueller for catching this.

• Hubert Figuière's "Rust and Automake"
• Luke Nukem's "Autotools and Rust"
• OStree's incantation for cargo vendor
• Havoc's "Cargo.lock should be in version control"

We had a nice weekend at FrOSCon with a lot of fun. This atmosphere has gone over to our neighbours, so some Fedora Ambassadors wanted to change to openSUSE. That was the last time at the Fedora booth for them and their booth became green.

You can see here a Fedora Ambassador who wants to have openSUSE marketing material for students of the university Marburg. He has green glasses as a signal for his change. He’ll give Linux workshops with openSUSE and wants to become a openSUSE Hero.

We had many visitors the first day. Our release party took place at our booth at 5 o’clock. We were surprised about so many people. The cake was away after a quarter hour. It wasn’t enough for all interested guests. All were happy and toasted the new Leap release with the champagne.

After that we had our first tombola with a big chameleon. What for a surprise! Last year a family of LPI won 2 chameleons. This year a small LPI girl won the first one again. That shows us the partnership between LPI and openSUSE.

Sunday I went to some interesting presentations. We shared our service at the openSUSE booth. Additional to that we spoke about the OpenRheinRuhr organization, what we want to improve and how we can realize all with new German Advocates. Second day we had a second tombola. This chameleon went to invis server.

Debian and Ubuntu didn’t have any booth. Some Debian users asked us for Debian Contributors. I sent them to Open Office. After this visit they came back and talked with us about openSUSE and what is new. They were really interested.

That was a successful weekend for openSUSE with a lot of fun. Thanks for all the sponsoring at FrOSCon!

I have not been actively participating in openSUSE project for some time now, as a result there has not been much to blog about on openSUSE Lizards blog, there is a new blog at https://cyberorg.wordpress.com to blog about what I have been and will be up to with Li-f-e: Linux for Education project among other things. I am also now “Member Emeritus” of the openSUSE community due to lack of participation, so cyberorg@opensuse.org email address will no longer work, please use @cyberorg.info if you need to get in touch with me.

After almost a decade of bringing you Li-f-e: Linux for Education based on openSUSE, it is now based on Ubuntu MATE LTS releases. I hope to provide the same excellent user experience that you have come to expect. Download it from here. Reason for this change is mentioned in previous post and it’s discussion(lack of interest/time/skills by anyone for maintaining live installer). You can of course easily enable Education Build Service repository to install packages on standard openSUSE or use susestudio to create your own spin with Education packages.

To new beginnings…

(First part of the series, with index to all the articles)

In the first part, we saw how glib-rs provides the FromGlib and ToGlib traits to let Rust code convert from/to Glib's simple types, like to convert from a Glib gboolean to a Rust bool and vice-versa. We also saw the special needs of strings; since they are passed by reference and are not copied as simple values, we can use FromGlibPtrNone and FromGlibPtrFull depending on what kind of ownership transfer we want, none for "just make it look like we are using a borrowed reference", or full for "I'll take over the data and free it when I'm done". Going the other way around, we can use ToGlibPtr and its methods to pass things from Rust to Glib.

In this part, we'll see the tools that glib-rs provides to do conversions of more complex data types. We'll look at two cases:

• Passing null-terminated arrays of strings from Glib to Rust

• Passing GLists to Rust

And one final case just in passing:

• Passing containers from Rust to Glib

Passing arrays from Glib to Rust

We'll look at the case for transferring null-terminated arrays of strings, since it's an interesting one. There are other traits to convert from Glib arrays whose length is known, not implied with a NULL element, but for now we'll only look at arrays of strings.

Null-terminated arrays of strings

Look at this function for GtkAboutDialog:

/**
 * gtk_about_dialog_add_credit_section:
 * @about: A #GtkAboutDialog
 * @section_name: The name of the section
 * @people: (array zero-terminated=1): The people who belong to that section
 * ...
 */
void
gtk_about_dialog_add_credit_section (GtkAboutDialog  *about,
                                     const gchar     *section_name,
                                     const gchar    **people)

You would use this like

const gchar *translators[] = {
    "Alice <alice@example.com>",
    "Bob <bob@example.com>",
    "Clara <clara@example.com>",
    NULL
};

gtk_about_dialog_add_credit_section (my_about_dialog, _("Translators"), translators);

The function expects an array of gchar *, where the last element is a NULL. Instead of passing an explicit length for the array, it's done implicitly by requiring a NULL pointer after the last element. The gtk-doc annotation says (array zero-terminated=1). When we generate information for the GObject-Introspection Repository (GIR), this is what comes out:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

<method name="add_credit_section"
        c:identifier="gtk_about_dialog_add_credit_section"
        version="3.4">
  ..
    <parameter name="people" transfer-ownership="none">
      <doc xml:space="preserve">The people who belong to that section</doc>
      <array c:type="gchar**">
        <type name="utf8" c:type="gchar*"/>
      </array>
    </parameter>

You can see the transfer-ownership="none" in line 5. This means that the function will not take ownership of the passed array; it will make its own copy instead. By convention, GIR assumes that arrays of strings are NULL-terminated, so there is no special annotation for that here. If we were implementing this function in Rust, how would we read that C array of UTF-8 strings and turn it into a Rust Vec<String> or something? Easy:

let c_char_array: *mut *mut c_char = ...; // comes from Glib
let rust_translators = FromGlibPtrContainer::from_glib_none(c_char_array);
// rust_translators is a Vec<String>

Let's look at how this bad boy is implemented.

First stage: impl FromGlibPtrContainer for Vec<T>

We want to go from a "*mut *mut c_char" (in C parlance, a "gchar **") to a Vec<String>. Indeed, there is an implementation of the FromGlibPtrContainer trait for Vecs here. These are the first few lines:

impl <P: Ptr, PP: Ptr, T: FromGlibPtrArrayContainerAsVec<P, PP>> FromGlibPtrContainer<P, PP> for Vec<T> {
    unsafe fn from_glib_none(ptr: PP) -> Vec<T> {
        FromGlibPtrArrayContainerAsVec::from_glib_none_as_vec(ptr)
    }

So... that from_glib_none() will return a Vec<T>, which is what we want. Let's look at the first few lines of FromGlibPtrArrayContainerAsVec:

1
2
3
4

    impl FromGlibPtrArrayContainerAsVec<$ffi_name, *mut $ffi_name> for $name {
        unsafe fn from_glib_none_as_vec(ptr: *mut $ffi_name) -> Vec<Self> {
            FromGlibContainerAsVec::from_glib_none_num_as_vec(ptr, c_ptr_array_len(ptr))
        }

Aha! This is inside a macro, thus the $ffi_name garbage. It's done like that so the same trait can be implemented for const and mut pointers to c_char.

See the call to c_ptr_array_len() in line 3? That's what figures out where the NULL pointer is at the end of the array: it figures out the array's length.

Second stage: impl FromGlibContainerAsVec::from_glib_none_num_as_vec()

Now that the length of the array is known, the implementation calls FromGlibContainerAsVec::from_glib_none_num_as_vec()

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

    impl FromGlibContainerAsVec<$ffi_name, *const $ffi_name> for $name {
        unsafe fn from_glib_none_num_as_vec(ptr: *const $ffi_name, num: usize) -> Vec<Self> {
            if num == 0 || ptr.is_null() {
                return Vec::new();
            }

            let mut res = Vec::with_capacity(num);
            for i in 0..num {
                res.push(from_glib_none(ptr::read(ptr.offset(i as isize)) as $ffi_name));
            }
            res
        }

Lines 3/4: If the number of elements is zero, or the array is NULL, return an empty Vec.

Line 7: Allocate a Vec of suitable size.

Lines 8/9: For each of the pointers in the C array, call from_glib_none() to convert it from a *const c_char to a String, like we saw in the first part.

Done! We started with a *mut *mut c_char or a *const *const c_char and ended up with a Vec<String>, which is what we wanted.

Passing GLists to Rust

Some functions don't give you an array; they give you a GList or GSList. There is an implementation of FromGlibPtrArrayContainerAsVec that understands GList:

impl<T> FromGlibPtrArrayContainerAsVec<<T as GlibPtrDefault>::GlibType, *mut glib_ffi::GList> for T
where T: GlibPtrDefault + FromGlibPtrNone<<T as GlibPtrDefault>::GlibType> + FromGlibPtrFull<<T as GlibPtrDefault>::GlibType> {

    unsafe fn from_glib_none_as_vec(ptr: *mut glib_ffi::GList) -> Vec<T> {
        let num = glib_ffi::g_list_length(ptr) as usize;
        FromGlibContainer::from_glib_none_num(ptr, num)
    }

The impl declaration is pretty horrible, so just look at the method: from_glib_none_as_vec() takes in a GList, then calls g_list_length() on it, and finally calls FromGlibContainer::from_glib_none_num() with the length it computed.

I have a Glib container and its length

In turn, that from_glib_none_num() goes here:

impl <P, PP: Ptr, T: FromGlibContainerAsVec<P, PP>> FromGlibContainer<P, PP> for Vec<T> {
    unsafe fn from_glib_none_num(ptr: PP, num: usize) -> Vec<T> {
        FromGlibContainerAsVec::from_glib_none_num_as_vec(ptr, num)
    }

Okay, getting closer to the actual implementation.

Give me a vector already

Finally, we get to the function that walks the GList:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

impl<T> FromGlibContainerAsVec<<T as GlibPtrDefault>::GlibType, *mut glib_ffi::GList> for T
where T: GlibPtrDefault + FromGlibPtrNone<<T as GlibPtrDefault>::GlibType> + FromGlibPtrFull<<T as GlibPtrDefault>::GlibType> {

    unsafe fn from_glib_none_num_as_vec(mut ptr: *mut glib_ffi::GList, num: usize) -> Vec<T> {
        if num == 0 || ptr.is_null() {
            return Vec::new()
        }
        let mut res = Vec::with_capacity(num);
        for _ in 0..num {
            let item_ptr: <T as GlibPtrDefault>::GlibType = Ptr::from((*ptr).data);
            if !item_ptr.is_null() {
                res.push(from_glib_none(item_ptr));
            }
            ptr = (*ptr).next;
        }
        res
    }

Again, ignore the horrible impl declaration and just look at from_glib_none_num_as_vec().

Line 4: that function takes in a ptr to a GList, and a num with the list's length, which we already computed above.

Line 5: Return an empty vector if we have an empty list.

Line 8: Allocate a vector of suitable capacity.

Line 9: For each element, convert it with from_glib_none() and push it to the array.

Line 14: Walk to the next element in the list.

Passing containers from Rust to Glib

This post is getting a bit long, so I'll just mention this briefly. There is a trait ToGlibContainerFromSlice that takes a Rust slice, and can convert it to various Glib types.

• To GSlist and GList. These have methods like to_glib_none_from_slice() and to_glib_full_from_slice()

• To an array of fundamental types. Here, you can choose between to_glib_none_from_slice(), which gives you a Stash like we saw the last time. Or, you can use to_glib_full_from_slice(), which gives you back a g_malloc()ed array with copied items. Finally, to_glib_container_from_slice() gives you back a g_malloc()ed array of pointers to values rather than plain values themselves. Which function you choose depends on which C API you want to call.

I hope this post gives you enough practice to be able to "follow the traits" for each of those if you want to look at the implementations.

Next up

Passing boxed types, like public structs.

Passing reference-counted types.

How glib-rs wraps GObjects.