Challenge:

Root-cause:

Resolution:

This post has been migrated to my new blog that you can find here:

https://pureooze.com/blog/posts/2018-05-21-unit-testing-browser-extensions/

In April I became the maintainer of Saka, a browser extension that allows users to search through their tabs, bookmarks and history. The original goal of Saka was to provide an elegant tab search but this soon evolved to include recently closed tabs, bookmarks and history when the original maintainer eejdoowad recognized that users search for tabs the same way they search bookmarks and history. This was an important insight and it has helped make Saka a valuable productivity tool.

When I became the maintainer I was surprised at the absence of tests in the project. There were several components with complicated logic but no tests to be found anywhere. One of the most important things I have learned as a developer is that tests are the easiest ways to write reliable, easy to refactor code. Was the old maintainer just lazy? Did he simply not care about the quality of his code? No. The opposite in fact, he cared a lot.

The issue is that the lack of documentation on the topic means that almost no one is able to test their extension. Having no confidence in my ability to make changes without breaking the code, this was a big problem. But as fate would have it after trying a dozen different approaches I ended up finding a solution.

Why We Test

As developers we want to be sure that the code we write today is not going to become a burden to maintain later in the lifetime of the application. One way we avoid creating these burdens is by writing tests. The great thing about tests is that outside of just verifying the behavior of functions, tests allow us to provide documentation for future developers. For example by creating unit tests we declare the valid inputs and outputs for a given function. This makes it easier to refactor code because we can have confidence that our code is working correctly when all our tests pass.

The Testing Approach

This post will focus on setting up the environment and writing some basic unit tests. I have a separate post at this link which you can use to perform integration testing.

To read the rest of this post it can be found on my new blog here:

https://pureooze.com/blog/posts/2018-05-21-unit-testing-browser-extensions/

Unit Testing Browser Extensions was originally published in Information & Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.

We always enjoy that new people join openSUSE community and help them in their first steps. Because of that, openSUSE participates again in GSoC, an international program in which stipends are awarded to students who hack on open source projects during the summer. We are really excited to announce that this year four students will learn about open source development while hacking on openSUSE projects. The coding period started last week, so our students are already busy hacking and they have written some nice articles about their projects. :blush:

Matheus de Sousa Bernardo

Matheus is a Software Engineering student in the University of Brasília. He will be working in Trollolo, together with his mentors Ana Martínez (me :stuck_out_tongue_winking_eye:) and Cornelius Schumacher. Trollolo is a cli-tool which helps teams using Trello to organize their work. Matheus’ project includes improving Trollolo’s API and workflow. If you want to know more about him and his project, check his beginning blog post: https://matheussbernardo.me/gsoc/2018/04/24/hello-internet-gsoc

Xu Liana

Xu is a Software Engineering student in the Chongqing Normal University, China. She will work, together with her mentors ZhaoQiang, epico and Hillwood Yang, in integrating Cloud Pinyin (the most popular input method in China) on ibus-libpinyin. Check her beginning blog post with more details about her and the project: https://liana.hillwoodhome.net/2018/05/20/the-blog-for-the-first-week-during-gsoc-libpinyin-project-coding

Ankush Malik

Ankush is pursuing a Bachelor of Technology in Computer Science in the Maharaja Agrasen Institute of Technology, India. He will be working on improving people collaboration in the Hackweek tool, together with his mentors David Kang and Stella Rouzi. He just released his GSoC Journey blog post which you can find here: https://medium.com/@ankushmalik631/my-gsoc-journey-4f02818fdb8d

Asad Syed

Asad is from Pakistan and is studying a Masters degree in Informatics at the Technical University of Munich, Germany. He will develop a container-based backend for openQA with the help of his mentors Santiago Zarate, Ettore Di Giacinto and Stephan Kulow. Read here about his experiences with openSUSE and past participations in Google Summer of Code: https://medium.com/asadpiz/google-summer-of-code-2018-with-opensuse-part-1-9f514ac2e7ae

As you see, there is a lot of great work coming in from our new talented contributors. Stay tuned because they will keep blogging about their GSoC experience! :wink:

I am incredibly happy because of three big things that are going on in librsvg right now:

1. Paolo Borelli finished porting all the CSS properties to Rust. What was once a gigantic RsvgState struct in C is totally gone, along with all the janky C code to parse individual properties. The process of porting RsvgState to Rust has been going on since about two months ago, and has involved many multi-commit merge requests and refactorings. This is a tremendous amount of really good work! The result is all in Rust now in a State struct, which is opaque from C's viewpoint. The only places in C that still require accessors to the State are in the filter effects code. Which brings me to...

2. Ivan Molodetskikh, my Summer of Code student, submitted his first merge request and it's merged to master now. This ports the bookkeeping infrastructure for SVG filters to Rust, and also the feOffset filter is ported now. Right now the code doesn't do anything fancy to iterate over the pixels of Cairo image surfaces; that will come later. I am very happy that filters, which were a huge barrier, are now starting to get chipped away into nicer code.

3. I have started to move librsvg's old representation of CSS properties into something that can really represent properties that are not specified, or explicitly set to inherit from an SVG element's parent, or set to a normal value. Librsvg never had a representation of property values that actually matched the SVG/CSS specs; it just knew whether a property was specified or not for an element. This worked fine for properties which the spec mandates that they should inherit automatically, but those that don't, were handled through special hacks. The new code makes this a lot cleaner. It should also make it easier to copy Servo's idioms for property inheritance.

We already mentioned Instantiated Services a few times. However, there is one question we did not yet cover:

How to hook up those instances with our package manager

For a normal service file this is pretty easy. Such a service can be handled as follow in the spec file:

%pre
%service_add_pre superduper.service

%post
%service_add_post superduper.service

%preun
%service_del_preun superduper.service

%postun
%service_del_postun superduper.service

That is it. It will do all the things we want.

Our graphics stack that deals with images has evolved a lot over the years.

In ye olden days

In the context of GIMP/GNOME, the only thing that knew how to draw RGB images to X11 windows (doing palette mapping for 256-color graphics cards and dithering if necessary) was the GIMP. Later, when GTK+ was written, it exported a GtkPreview widget, which could take an RGB image buffer supplied by the application and render it to an X window — this was what GIMP plug-ins could use in their user interface to show, well, previews of what they were about to do with the user's images. Later we got some obscure magic in a GdkColorContext object, which helped allocate X11 colors for the X drawing primitives. In turn, GdkColorContext came from the port that Miguel and I did of XmHTML's color context object (and for those that remember, XmHTML became the first version of GtkHtml; later it was rewritten as a port of KDE's HTML widget). Thankfully all that stuff is gone now; we can now assume that video cards are 24-bit RGB or better everywhere, and there is no need to worry about limited color palettes and color allocation.

Later, we started using the Imlib library, from the Enlightenment project, as an easy API to load images — the APIs from libungif, libjpeg, libpng, etc. were not something one really wanted to use directly — and also to keep images in memory with a uniform representation. Unfortunately, Imlib's memory management was peculiar, as it was tied to Enlightenment's model for caching and rendering loaded/scaled images.

A bunch of people worked to write GdkPixbuf: it kept Imlib's concepts of a unified representation for image data, and an easy API to load various image formats. It added support for an alpha channel (we only had 1-bit masks before), and it put memory management in the hands of the calling application, in the form of reference counting. GdkPixbuf obtained some high-quality scaling functions, mainly for use by Eye Of Gnome (our image viewer) and by applications that just needed scaling instead of arbitrary transformations.

Later, we got libart, the first library in GNOME to do antialiased vector rendering and affine transformations. Libart was more or less compatible with GdkPixbuf: they both had the same internal representation for pixel data, but one had to pass the pixels/width/height/rowstride around by hand.

Mea culpa

Back then I didn't understand premultiplied alpha, which is now ubiquitous. The GIMP made the decision to use non-premultiplied alpha when it introduced layers with transparency, probably to "avoid losing data" from transparent pixels. GdkPixbuf follows the same scheme.

(Now that the GIMP uses GEGL for its internal representation of images... I have no idea what it does with respect to alpha.)

Cairo and afterwards

Some time after the libart days, we got Cairo and pixman. Cairo had a different representation of images than GdkPixbuf's, and it supported more pixel formats and color models.

GTK2 got patched to use Cairo in the toplevel API. We still had a dichotomy between Cairo's image surfaces, which are ARGB premultiplied data in memory, and GdkPixbufs, which are RGBA non-premultiplied. There are utilities in GTK+ to do these translations, but they are inconvenient: every time a program loads an image with GdkPixbuf's easy API, a translation has to happen from non-premul RGBA to premul ARGB.

Having two formats means that we inevitably do translations back and forth of practically the same data. For example, when one embeds a JPEG inside an SVG, librsvg will read that JPEG using GdkPixbuf, translate it to Cairo's representation, composite it with Cairo onto the final result, and finally translate the whole thing back to a GdkPixbuf... if someone uses librsvg's legacy APIs to output pixbufs instead of rendering directly to a Cairo surface.

Who uses that legacy API? GTK+, of course! GTK+ loads scalable SVG icons with GdkPixbuf's loader API, which dynamically links librsvg at runtime: in effect, GTK+ doesn't use librsvg directly. And the SVG pixbuf loader uses the "gimme a pixbuf" API in librsvg.

GPUs

Then, we got GPUs everywhere. Each GPU has its own preferred pixel format. Image data has to be copied to the GPU at some point. Cairo's ARGB needs to be translated to the GPU's preferred format and alignment.

Summary so far

• Libraries that load images from standard formats have different output formats. Generally they can be coaxed into spitting ARGB or RGBA, but we don't expect them to support any random representation that a GPU may want.

• GdkPixbuf uses non-premultiplied RGBA data, always in that order.

• Cairo uses premultiplied ARGB in platform-endian 32-bit chunks: if each pixel is 0xaarrggbb, then the bytes are shuffled around depending on whether the platform is little-endian or big-endian.

• Cairo internally uses a subset of the formats supported by pixman.

• GPUs use whatever they damn well please.

• Hilarity ensues.

What would we like to do?

We would like to reduce the number of translations between image formats along the loading-processing-display pipeline. Here is a plan:

• Make sure Cairo/pixman support the image formats that GPUs generally prefer. Have them do the necessary conversions if the rest of the program passes an unsupported format. Ensure that a Cairo image surface can be created with the GPU's preferred format.

• Make GdkPixbuf just be a wrapper around a Cairo image surface. GdkPixbuf is already an opaque structure, and it already knows how to copy pixel data in case the calling code requests it, or wants to turn a pixbuf from immutable to mutable.

• Provide GdkPixbuf APIs that deal with Cairo image surfaces. For example, deprecate gdk_pixbuf_new() and gdk_pixbuf_new_from_data(), in favor of a new gdk_pixbuf_new_from_cairo_image_surface(). Instead of gdk_pixbuf_get_pixels() and related functions, have gdk_pixbuf_get_cairo_image_surface(). Mark the "give me the pixel data" functions as highly discouraged, and only for use really by applications that want to use GdkPixbuf as an image loader and little else.

• Remove calls in GTK+ that cause image conversions; make them use Cairo image surfaces directly, from GdkTexture up.

• Audit applications to remove calls that cause image conversions. Generally, look for where they use GdkPixbuf's deprecated APIs and update them.

Is this really a performance problem?

This is in the "excess work" category of performance issues. All those conversions are not really slow (they don't make up for the biggest part of profiles), but they are nevertheless things that we could avoid doing. We may get some speedups, but it's probably more interesting to look at things like power consumption.

Right now I'm seeing this as a cool, minor optimization, but more as a way to gradually modernize our image API.

We seem to change imaging models every N years (X11 -> libart -> Cairo -> render trees in GPUs -> ???). It is very hard to change applications to use different APIs. In the meantime, we can provide a more linear path for image data, instead of doing unnecessary conversions everywhere.

Code

I have a use-cairo-surface-internally branch in gdk-pixbuf, which I'll be working on this week. Meanwhile, you may be interested in the ongoing Performance Hackfest in Cambridge!