Hackweek 22 took place last week. During this week all the SUSE employees are free to hack on whatever they want. This one of the perks of working at SUSE 😎.

This time my personal project has been about building a unikernel that runs WebAssembly.

I wanted this blog post to contain all the details about this journey. However I realized this would have been too much for a single post. I hence decided to split everything into smaller chunks. I’ll update this section to keep track of all the posts.

In the meantime, you can find the code of the POC here.

Why

There are multiple reasons why I did that, but I don’t want to repeat what I wrote inside of the project description. Learning and fun goals aside, I think there’s actually a good reason to mix unikernels and WebAssembly.

From the application developer POV, porting/writing an application to the unikernel is not an easy task. The application and all its dependencies have to support the target unikernel. Some patching might be required inside of the whole application stack to make it work.

From the unikernel maintainers POV, they have to invest quite some energies to ensure any kind of application can run in a seamless way on top of their platform. They don’t know which kind of system primitives the user applications will leverage, this makes everything harder.

On the other hand, when targeting a WebAssembly platform (think of Spin or Spiderlightning), the application has a clear set of capabilities that have to be provided by the WebAssembly runtime.

If you look at the Spiderlightning scenario, an application might be requiring Key/Value store capabilities at runtime. However, how these capabilities are implemented on the host side is not relevant to the application. That means the same .wasm module can be run by a runtime that implements the K/V store using Redis or using Azure Cosmos DB. That would be totally transparent to the end user application.

You might see where I’m going with all that…

If we write a unikernel application that runs WebAssembly modules and supports a set of Spiderlightning APIs, then the same Spiderlightning application could be run both on top of the regular slight runtime and of this unikernel.

All of that without any additional work from the application developer. The Wasm module wouldn’t even realize that. The complexity would fall only on the unikernel developer who, whoever, would have a clear set of functionalities to implement (as opposed to “let’s try to make any kind of application work”).

How

Sometimes ago I stumbled over the RustyHermit project, this is a unikernel written in Rust. I decided to use it as the foundation to write my unikernel application.

Building a RustyHermit application is pretty straightforward. Their documentation, even though is a bit scattered, is good and their examples help a lot.

The cool thing is that RustyHermit is part of Rust nightly, which makes the whole developer experience great. It feels like writing a regular Rust application.

Obviously you cannot expect all kind of Rust crates to just work with RustyHermit. You will see how that influenced the development of the POC.

The next sections go over some of the major challenges I faced during the last week. I’ll share more details inside of the upcoming blog posts (see the disclaimer section at the top of the page).

The WebAssembly runtime

Unfortunately Wasmtime, my favorite WebAssembly runtime, does not build on top of RustyHermit. Many of its dependencies expect libc or other low level libraries to be around. The same applies to wasmer.

I thought about using something like WebAssembly Micro Runtime (WAMR), but I preferred to stick with something written in Rust and have the “full RustyHermit experience”.

After some searching I found wasmi a pure Rust WebAssembly runtime. This works fine on top of RustyHermit, plus its design is inspired by the one of Wasmtime, which allowed me to reuse a lot of my previous knowledge.

WebAssembly Component Model

Spiderlightning leverages the WebAssembly Component Model proposal to offer capabilities to the WebAssembly guests and to allow the host to consume capabilities offered by the WebAssembly guest.

The communication between the host and the guest happens using types defined with the Wasm Interface Type.

To give some concrete examples, the demo I’m going to run leverages the WebAssembly Component Model in these ways:

• The guest asks the host to start a HTTP server. When doing that, the guest informs the host about the HTTP routes that have to be registered, plus the names of its internal handlers (the functions that have to be executed). This is done by using the http-server types. In this case it’s the guest that leverages capabilities offered by the host.
• The host handles the incoming HTTP requests using the routing information provided by the guest. The http handlers mentioned before are functions exposes by the WebAssembly guest. The server is now consuming capabilities offered by the guest. The communication is done using the http-handler types.
• Some of the http handlers defined by the guest are also interacting with a Key/Value store. Also in this case the guest is leveraging a set of capabilities offered by the host. These are defined using the keyvalue types.

As you can see there are many WIT types involved. For each one of them we need code both inside of the guest (a SDK basically) and on the host (the code that implements the guest SDK). This code can be scaffolded by a cli tool called wit-bindgen, which generates host/guest code starting from a .wit file.

In this case I only had to implement the host side of these interfaces inside of the unikernel.

The code generated by wit-bindgen is doing low level operations using the WebAssembly runtime. The code to be scaffolded depends on the programming language and on the WebAssembly runtime used on the host side.

Obviously the wasmi WebAssembly runtime was not supported by wit-bindgen, hence I had to extend wit-bindgen to handle it. The code can be found inside of this fork, under the wasmi branch.

With all of that in place, I scaffolded the host side of the Key/Value capability and I made a simple implementation of the host traits. The host code was just emitting some debug information. I was then able run the vanilla keyvalue-demo from the Spiderlightning project. 🥳

Summary

You made to the bottom of this long post, kudos! I think you deserve a prize for that, so here we go…

This is a recording of the unikernel application running the Spiderlightning http-server demo.

I hope you enjoyed the reading. Stay tuned for the next part of the journey. This will cover Rust async, Redis and some weird errors.

This is the sixth part of my syslog-ng tutorial. Last time, we learned about syslog-ng source definitions and how to check the syslog-ng version. Today, we learn about syslog-ng destinations and the log path. At the end of the session, we will also perform a quick syntax check.

You can watch the video on YouTube:

Or you can read the rest the tutorial as a blog at: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-101-part-6-destinations-and-log-path

Back in 2012, I received a box of eight hundred openSUSE 12.1 promo DVDs, which I then set out to distribute to local Linux users’ groups, tech conferences, other SUSE crew in Australia, and so forth. I didn’t manage to shift all 800 DVDs at the time, and I recently rediscovered the remaining three hundred and eighty four while installing some new shelves. As openSUSE 12.1 went end of life in May 2013, it seemed likely the DVDs were now useless, but I couldn’t bring myself to toss them in landfill. Instead, given last week was Hack Week, I decided to use them for an art project. Here’s the end result:

Making that mosaic was extremely fiddly. It’s possibly the most annoying Hack Week project I’ve ever done, but I’m very happy with the outcome

The backing is a piece of 900mm x 600mm x 6mm plywood, primed with some leftover kitchen and bathroom undercoat, then spray pained black. I’d forgotten how bad spray paint smells, but it makes for a nice finish. To get the Geeko shape, I took the official openSUSE logo, then turned it into an outline in Inkscape, saved that as a PNG, opened it in GIMP, and cut it into nine 300mm x 200mm pieces which I then printed on A4 paper, stuck together with tape, and cut out to make a stencil. Of course, the first time I did that, nothing quite lined up, so I had to reprint it but with “Ignore page margins” turned off and “Draw crop marks” turned on, then cut the pages down along the crop marks before sticking them together the second time. Then I placed the stencil on the backing, glued the eye down (that just had to be made from the centre of a DVD!) and started laying out cut up DVD shards.

I initially tried cutting the DVDs with tin snips, which is easy on the hands, but had a tendency to sometimes warp the DVD pieces and/or cause them to delaminate, so I reverted to a large pair of scissors which was more effort but ultimately less problematic.

After placing the pieces that made up the head, tail, feet and spine, and deciding I was happy with how they looked, I glued each piece down with superglue. Think: carefully pick up DVD shard without moving too many other shards, turn over, dab on a few tiny globs of superglue, lower into place, press for a few seconds, move to next piece. Do not get any superglue on your fingers, or you’ll risk sticking your fingers together and/or make a gluey mess on the shiny visible side of the DVD shards.

It was another three sessions of layout-then-glue-down to fill in the body. I think I stuck my fingers together about six, or eight, or maybe twenty times. Also, despite my best efforts to get superglue absolutely nowhere near the stencil at all, when I removed the stencil, it had stuck to the backing in several places. I managed to scrape/cut that off with a combination of fingernails, tweezers, and the very sharp knife in my SLE 12 commemorative Leatherman tool, then touched up the remaining white bits with a fine point black Sharpie.

Judging from the leftover DVD centre pieces, this mosaic used about 12 DVDs in all, which isn’t very many considering my initial stash. I had a few other ideas for the remainder, mostly involving hanging them up somehow, which I messed around with earlier on while waiting for the paint to dry on the plywood.

One (failed) idea was to use a cutting wheel on my Dremel tool to slice half way through a few DVDs, then slot them into each other to make a hanging thingy that would spin in the wind. I was unable to make a smooth/straight enough cut for this to work, and superglue doesn’t bridge gaps. You can maybe get an idea of what I was aiming at from this photo:

My wife had an idea for a better way to do this, which is to take a piece of dowel, cut slots in the sides, and glue DVD halves into the slots using Araldite (that’s an epoxy resin, in case you didn’t grow up with that brand name). I didn’t get around to trying this, but I reckon she’s onto something. Next time I’m at the hardware store, I’ll try to remember to pick up some suitably sized dowel.

I did make one somewhat simpler hanging thingy, which I call “Geeko’s Tail (Uncurled)”. It’s just DVDs superglued together on the flat, hanging from fishing line, but I think it’s kinda cool:

Also, I’ve discovered that Officeworks has an e-waste recycling program, so any DVDs I don’t use in future projects needn’t go to landfill.

Update 2023-02-20: For photos of the mosaic, plus wallpapers made from the photos, see https://github.com/tserong/hackweek22

Our reference server stopped sending out email notifications on February 3th, 2023. In the lines below you will find a detailed explanation of what happened. Impact Our reference server wasn’t sending out email notifications (or only partially) to the users starting Feburary 3th, 2023 around 11:37PM until February 6th, 2023 13:06PM. Root Causes The problem was caused by an exception thrown in the SendEventEmailsJob due to not properly querying “hidden projects”, which made the job...

You can read it in English

Olá Geekos.

Estamos animados com o nosso novo fórum em português. Esperamos ter a sua ajuda para construir uma comunidade envolvente no nosso idioma e para a nossa querida distribuição Linux, o openSUSE.

O fórum em português está disponível em: https://forums.opensuse.org/c/portugues-portuguese/130

Com este novo canal, queremos unir a comunidade portuguesa de usuários do openSUSE e melhorar o apoio aos nossos usuários. Crie uma conta e comece fazer perguntas ou ajudar a responder dúvidas de outras pessoas.

Esse novo meio de comunicação é um complemento aos canais do Telegram e Matrix, e ainda continuaremos com o suporte nos diversos meios. O fórum oferece algumas vantagens em relação às mensagens instantâneas, como, por exemplo, uma melhor ferramenta de busca e armazenamento a longo termo.

Esperamos que o fórum cresça e melhore gradativamente com a sua ajuda.

Divirta-se!

---

#English

Hello Geekos.

We are excited to bring you our new Portuguese forum. We look forward to build, with your help, an engaging community in our language for our beloved Linux distribution, openSUSE.

The Portuguese forum is available at: https://forums.opensuse.org/c/portugues-portuguese/130

With this new place, we want to unite the Portuguese openSUSE community and continue to provide mutual support to our users. Do not hesitate to create an account, ask questions and help other users.

This new communication channel complements our Telegram and Matrix channels, and we will continue to provide support on each platform. The forum offers a few advantages compared to instant messaging, such as a better search mechanism and better long-term data retention.

We hope that the Portuguese forum will gradually grow and improve with your help.

Have a lot of fun!

You can read it in English

Olá Geekos.

Estamos animados com o nosso novo fórum em português. Esperamos ter a sua ajuda para construir uma comunidade envolvente no nosso idioma e para a nossa querida distribuição Linux, o openSUSE.

O fórum em português está disponível em: https://forums.opensuse.org/c/portugues-portuguese/130

Com este novo canal, queremos unir a comunidade portuguesa de usuários do openSUSE e melhorar o apoio aos nossos usuários. Crie uma conta e comece fazer perguntas ou ajudar a responder dúvidas de outras pessoas.

Esse novo meio de comunicação é um complemento aos canais do Telegram e Matrix, e ainda continuaremos com o suporte nos diversos meios. O fórum oferece algumas vantagens em relação às mensagens instantâneas, como, por exemplo, uma melhor ferramenta de busca e armazenamento a longo prazo.

Esperamos que o fórum cresça e melhore gradativamente com a sua ajuda.

Divirta-se!

---

#English

Hello Geekos.

We are excited to bring you our new Portuguese forum. We look forward to build, with your help, an engaging community in our language for our beloved Linux distribution, openSUSE.

The Portuguese forum is available at: https://forums.opensuse.org/c/portugues-portuguese/130

With this new place, we want to unite the Portuguese openSUSE community and continue to provide mutual support to our users. Do not hesitate to create an account, ask questions and help other users.

This new communication channel complements our Telegram and Matrix channels, and we will continue to provide support on each platform. The forum offers a few advantages compared to instant messaging, such as a better search mechanism and better long-term data retention.

We hope that the Portuguese forum will gradually grow and improve with your help.

Have a lot of fun!

This night of Linux Saloon chat was some of my most favorite I have had since this started. I would say that the show went completely off the rails but that would imply that it was a disaster. I would say that the conversation grew and flowered from the fertile ground tilled by the various […]

Dear Tumbleweed users and hackers,

During this week, many developers took part in HackWeek, resulting in resources being deviated from regular distro maintenance to other areas of interest. I’m certain to see some great outcomes of this hackweek over the next weeks/months (see for example the thread on Creating a Leap replacement based on ALP). Of course, Tumbleweed has been keeping up with all the changes and supports everybody in getting their results delivered to users. And it did so by delivering the usual 7 snapshots in a week (0126…0201)

The main changes found in those 7 snapshots were:

• Node.JS 19.5.0
• Mesa 22.3.4
• pipewire 0.3.65
• btrfsprogs 6.1.3
• Systemd 252.5
• XTerm 378
• libnvme 1.3 and nvme-cli 2.3
• Boost 1.81.0

The next few snapshots might be really interesting though: Snapshot 0202, which is currently building, will be the first to have switched the default openssl implementation from openssl 1.1 to 3.0. This was a major project spanning quite a long period, but it finally ended.

Here is an overview of what the next week’s snapshot promise to deliver:

• Switched openssl by default to the 3.0 branch (currently 3.0.7)
• Mozilla Thunderbird 102.7.1
• GStreamer 1.22.0
• KDE Gear 22.12.2
• Rust 1.67
• KDE Plasma 5.27 beta (5.26.90)
• Binutils 2.40
• Enabling of python311 modules (keeping python 3.10 as the default interpreter in a first step)
• Staging:H still tests ruby 3.2 as the new default (yast2-packager is the only failing package left)
• Staging:L holds some packages breaking others stuff taking more time, like gpg2, and ant
• Staging:Gcc7 tests the impact of using GCC 13 as the default compiler

This hackweek I’ve been playing a bit around with my desktop computer which has AMD Radeon 6600 XT graphics card which is based on the RDNA2 architecture. The idea was to find a way to utilize it for Stable Diffusion Version 2 latent text-to-image diffusion model without invading the host too much with randomly downloaded modules, but still using the GPU for computing. The graphics card has “only” 8GB RAM which is apparently only a starter amount in this field, so I needed to also check if that’s enough.

Shortly, I found out that while the code is open source, the model data is unfortunately not as once again new licenses have been developed (OpenRAIL license family) by people who have not fully understood or wanted to understand the wisdom in The Open Source Definition (or free software definition either). So ultimately this is just about studying and using these models for fun, not for serious use. Hopefully open source models will be also developed at some point in the future. I just fear this will only happen a long time later, after the effects of having vague ethical points in a copyright license are felt and “this is not what we intended, how could we have anticipated these problems?” said by the people creating and utilizing the data. (continued hopefully with “hmm, how could we re-license all of this to CC-BY-SA?”)

Since my Hackweek time is more limited than intended, and I also ended up battling broken pypi modules and other things, I’ll just leave here a Docker container git tree and a sample image generated below. To put it short, it worked like a breeze until it broke, thanks pip/pypi/numpy/something. Anyway, when it works, it initializes InvokeAI based web UI for inputting to Stable Diffusion. And yes, the ROCm stack works nicely on my desktop computer - I downloaded and used stable-diffusion-2.1-768 model data only, disabled nsfw filter to save VRAM, and created 768x768 images - the VRAM use was around 6.5GB out of 8GB available according to radeontop, and it worked like a charm!

https://github.com/tjyrinki/sd-rocm

Many of the dockerfiles around were both woefully outdated and unlicensed so I could not use those other than for inspiration - these are MIT licensed.

Here is also image of the UI running in web browser (you can also use just Python CLI):

The shakiness of pypi installation has ended after yesterday now, and this time I’ll commit the final docker container result for later use.

As start let’s sort it out what is dependency and what is reverse dependency.

Dependencies and reverse dependencies in Linux distributions are important concepts to understand. A package dependency means that another package relies on it in order to function. For example, if package B requires package A to be installed in order to work, then package B is dependent on package A and is considered a reverse dependency of package A.