I am happy to release Kraft 0.51 today. Kraft is the KDE solution to handle daily business documents like quotes and invoices in your small business.

This is a bugfix release which brings a handful of useful fixes against bugs which were reported by Kraft users since the last release.

In the catalog view, now drag and drop is working to sort templates. Removing of sub chapters is also working now. A bug in the unit handling was fixed that picked wrong units in some cases. The path to document templates is not utf8 save.

As a new feature the address of the own company now can be picked from Kraft’s settings dialog also after first setup routine.

A source tarball can be downloaded from the Sourceforge Project, binary packages are on the way. Please also report bugs on SF.

Thanks for your interest and contribution to Kraft. If you want to support Kraft, please give feedback, spread the word or buy cool stuff.

Warning: Viewer discretion is advised.

Where should I begin?

I guess I should start by saying that I am obsessed with MIME and, in particular, MIME parsers. No, really. I am obsessed. Don't believe me? I've written and/or worked on several MIME parsers at this point. It started off in my college days working on Spruce which had a horrendously bad MIME parser, and so as you read farther along in my rant about shitty MIME parsers, keep in mind: I've been there, I've written a shitty MIME parser.

As a handful of people are aware, I've recently started implementing a C# MIME parser called MimeKit. As I work on this, I've been searching around on GitHub and Google to see what other MIME parsers exist out there to find out what sort of APIs they provide. I thought perhaps I'll find one that offers a well-designed API that will inspire me. Perhaps, by some miracle, I'd find one that was actually pretty good that I could just contribute to instead of writing my own from scratch (yea, wishful thinking). Instead, all I have found are poorly designed and implemented MIME parsers, many probably belong on the front page of the Daily WTF.

I guess I'll start with some softballs.

First, there's the fact that every single one of them were written as System.String parsers. Don't be fooled by the ones claiming to be "stream parsers", because all any of those did was to slap a TextReader on top of the byte stream and start using reader.ReadLine(). What's so bad about that, you ask? For those not familiar with MIME, I'd like for you to take a look at the raw email sources in your inboxes particularly if you have correspondence with anyone outside of the US. Hopefully most of your friends and colleagues are using more-or-less MIME compliant email clients, but I guarantee you'll find at least a few emails with raw 8bit text.

Now, if the language they were using was C or C++, they might be able to get away with doing this because they'd technically be operating on byte arrays, but with Java and C#, a 'string' is a unicode string. Tell me: how does one get a unicode string from a raw byte array?

Bingo. You need to know the charset before you can convert those bytes into unicode characters.

To be fair, there's really no good way of handling raw 8bit text in message headers, but by using a TextReader approach, you are really limiting the possibilities.

Next up is the ReadLine() approach. One of the 2 early parsers in GMime (pan-mime-parser.c back in the version 0.7 days) used a ReadLine() approach, so I understand the thinking behind this. And really, there's nothing wrong with this approach as far as correctness goes, it's more of a "this can never be fast" complaint. Of the two early parsers in GMime, the pan-mime-parser.c backend was horribly slow compared to the in-memory parser. Of course, that's not very surprising. More surprising to me at the time was that when I wrote GMime's current generation of parser (sometime between v0.7 and v1.0), it was just as fast as the in-memory parser ever was and only ever had up to 4k in a read buffer at any given time. My point is, there are far better approaches than ReadLine() if you want your parser to be reasonably performant... and why wouldn't you want that? Your users definitely want that.

Okay, now come the more serious problems that I encountered in nearly all of the mime parser libraries I found.

I think that every single mime parser I've found so far uses the "String.Split()" approach for parsing address headers and/or for parsing parameter lists on headers such as Content-Type and Content-Disposition.

Here's an example from one C# MIME parser:

string[] emails = addressHeader.Split(',');

Here's how this same parser decodes encoded-word tokens:

private static void DecodeHeaders(NameValueCollection headers)
{
    ArrayList tmpKeys = new ArrayList(headers.Keys);

    foreach (string key in headers.AllKeys)
    {
        //strip qp encoding information from the header if present
        headers[key] = Regex.Replace(headers[key].ToString(), @"=\?.*?\?Q\?(.*?)\?=",
            new MatchEvaluator(MyMatchEvaluator), RegexOptions.IgnoreCase | RegexOptions.Multiline);
        headers[key] = Regex.Replace(headers[key].ToString(), @"=\?.*?\?B\?(.*?)\?=",
            new MatchEvaluator(MyMatchEvaluatorBase64), RegexOptions.IgnoreCase | RegexOptions.Multiline);
    }
}

private static string MyMatchEvaluator(Match m)
{
    return DecodeQP(m.Groups[1].Value);
}

private static string MyMatchEvaluatorBase64(Match m)
{
    System.Text.Encoding enc = System.Text.Encoding.UTF7;
    return enc.GetString(Convert.FromBase64String(m.Groups[1].Value));
}

Excuse my language, but what the fuck? It completely throws away the charset in each of those encoded-word tokens. In the case of quoted-printable tokens, it assumes they are all ASCII (actually, latin1 may work as well?) and in the case of base64 encoded-word tokens, it assumes they are all in UTF-7!?!? Where in the world did he get that idea? I can't begin to imagine his code working on any base64 encoded-word tokens in the real world. If anything is deserving of a double facepalm, this is it.

I'd just like to point out that this is what this project's description states:

A small, efficient, and working mime parser library written in c#.
...
I've used several open source mime parsers before, but they all either
fail on one kind of encoding or the other, or miss some crucial
information. That's why I decided to finally have a go at the problem
myself.

I'll grant you that his MIME parser is small, but I'd have to take issue with the "efficient" and "working" adjectives. With the heavy use of string allocations and regex matching, it could hardly be considered "efficient". And as the code pointed out above illustrates, "working" is a bit of an overstatement.

Folks... this is what you get when you opt for a "lightweight" MIME parser because you think that parsers like GMime are "bloated".

On to parser #2... I like to call this the "Humpty Dumpty" approach:

public static StringDictionary parseHeaderFieldBody ( String field, String fieldbody ) {
    if ( fieldbody==null )
        return null;
    // FIXME: rewrite parseHeaderFieldBody to being regexp based.
    fieldbody = SharpMimeTools.uncommentString (fieldbody);
    StringDictionary fieldbodycol = new StringDictionary ();
    String[] words = fieldbody.Split(new Char[]{';'});
    if ( words.Length>0 ) {
        fieldbodycol.Add (field.ToLower(), words[0].ToLower().Trim());
        for (int i=1; i<words.Length; i++ ) {
            String[] param = words[i].Trim(new Char[]{' ', '\t'}).Split(new Char[]{'='}, 2);
            if ( param.Length==2 ) {
                param[0] = param[0].Trim(new Char[]{' ', '\t'});
                param[1] = param[1].Trim(new Char[]{' ', '\t'});
                if ( param[1].StartsWith("\"") && !param[1].EndsWith("\"")) {
                    do {
                        param[1] += ";" + words[++i];
                    } while ( !words[i].EndsWith("\"") && i<words.Length);
                }
                fieldbodycol.Add ( param[0], SharpMimeTools.parserfc2047Header (param[1].TrimEnd(';').Trim('\"', ' ')) );
            }
        }
    }
    return fieldbodycol;
}

I'll give this guy some credit, at least he saw that his String.Split() approach was flawed and so tried to compensate by piecing Humpty Dumpty back together again. Of course, with his String.Trim()ing, he just won't be able to put him back together again with any level of certainty. The white space in those quoted tokens may have significant meaning.

Many of the C# MIME parsers out there like to use Regex all over the place. Here's a snippet from one parser that is entirely written in Regex (yea, have fun maintaining that...):

if (m_EncodedWordPattern.RegularExpression.IsMatch(field.Body))
{
    string charset = m_CharsetPattern.RegularExpression.Match(field.Body).Value;
    string text = m_EncodedTextPattern.RegularExpression.Match(field.Body).Value;
    string encoding = m_EncodingPattern.RegularExpression.Match(field.Body).Value;

    Encoding enc = Encoding.GetEncoding(charset);

    byte[] bar;

    if (encoding.ToLower().Equals("q"))
    {
        bar = m_QPDecoder.Decode(ref text);
    }
    else
    {
        bar = m_B64decoder.Decode(ref text);
    }                    
    text = enc.GetString(bar);

    field.Body = Regex.Replace(field.Body,
        m_EncodedWordPattern.TextPattern, text);
    field.Body = field.Body.Replace('_', ' ');
}

Let's pretend that the regex pattern strings are correct in their definitions (because they are god-awful to read and I can't be bothered to double-check them), the replacing of '_' with a space is wrong (it should only be done in the "q" case) and the Regex.Replace() is just evil. Not to mention that there could be multiple encoded-words per field.Body which this code utterly fails to handle.

Guys. I know you love regular expressions and that they are very very useful, but they are no substitute for writing a real tokenizer. This is especially true if you want to be lenient in what you accept (and in the case of MIME, you really need to be).

I was at an Agile Roundtable not too long ago and someone was talking up Strengths Finder 2.0, so I decided to pick up the book off of Amazon:

The book has a short introduction, which is a quick read, details about the strengths, and a code to take the test on their website. The test is timed and probably takes about 35 minutes. After the series of questions are answered it calculates your strengths and gives you your top five strengths. Once you have your five strengths the site and the book gives you an explanation of the strength as well as “Ideas for Action” for that strength.

According to the test these are my top five strengths.

1. Adaptability
2. Input
3. Learner
4. Communication
5. Achiever

What are yours?

It has long been a challenge to use MS Silverlight based websites on linux systems. Especially in The Netherlands this is a big hurdle as many (>80%) of the secondary school websites that pupils must use to communicate with their school (for homework, marks, etc) are equipped with Silverlight. Yes, really…

Fortunately at the end of August 2013 I discovered pipelight, a very smart idea to use MS Silverlight based website natively on Linux. The problem was however to find a working pipelight package for openSUSE. As there was none, I decided to build one myself using the incredible openSUSE Build Service. It was quite a quest to obtain a working package, but due to very good cooperation with the pipelight developers, I’m now able to present a working pipelight package to the openSUSE community. Oh, and while working on the package I reported a bug via the bug report system, that was solved and published via an rpm package within 1 hour after reporting it (that was during out of office hours). Indeed within 1 hour after reporting the problem it was; accepted, investigated, analysed, fixed, tested, handed over to me, packaged, tested and published! The amazing world of Open Source Software!

Pipelight works okay for the following sites (among many others): arte, LOVEFiLM, Netflix, Magister based NL schoolwebsites, WATCHEVER, etc. View the complete list on the pipelight website.

The installation instructions are on the pipelight website. Be aware though, that pipelight requires the wine package that is provided via the home:rbos:pipelight repository. With any other wine package, pipelight will (very likely) not work. If you rely on your currently installed wine package and installed MS applications and are unsure that the wine package provided via the home:rbos:pipelight repository will leave your currently in use MS applications untouched: don’t install pipelight (or only after making very good backups). You can always start by installing pipelight in a virtual machine.

Have fun with pipelight.

Summer is ending soon (at least for those living in the northern hemisphere) and while usually cleaning is done during spring, the KDE team decided to do what I’d call… autumn cleaning of repositories.

You may know that the KDE presence in openSUSE, aside being the default desktop, is quite a long one. In the past years different repositories were created by the members of the openSUSE KDE team (at the time mostly made up by KDE people hired by Novell) in order to review and test packages, like newer Qt versions, KDE software, and so on. Fast forward to the present: nowadays the members of the KDE team are almost completely from the openSUSE community, and quite a number of changes went by the repositories as well. For example, newer releases of KDE software are submitted as maintenance updates for the latest available version of the distribution, and there are the KDE:Release:xy repositories for those who want the latest and greatest KDE software.

That also meant that a lot of repositories were unused, and were left bitrotting (and consuming the OBS’s precious build power). But no more! Recently, thanks to the input from Raymond (tittiatcoke on IRC), a rather large cleaning of repositories is taking place.

The following repositories are going to be deleted:

• KDE:Qt45

• KDE:Qt46

• KDE:Qt47

• KDE:Qt:Stable

• KDE:Netbook

• KDE:Qt50

In the (unlikely) case you are using them, you should remove them ASAP.

This repository instead will be **moved **(thanks kdepepo for reminding me):

• KDE:Frameworks → KDE:Unstable:Frameworks

As with repository cleaning, there was also a reorganization of the maintainership, because a number of former maintainers had moved on. In practice this will mean that notifications and reports will get to the right people instead of just clogging the mailboxes of unrelated people. ;) Of course, the present KDE team stands on the shoulders of giants, and is extremely thankful for the work done by those people in the past.

That’s all, now we’re back to our regularly scheduled programs.

Sometimes you don’t have access to vim’s wonderful xxd tool, and you need to use it to generate some .c code based on a binary file. This happened to me recently when packaging up the EFI signing tools for Gentoo. Adding a build requirement of vim for a single autogenerated file was not an option for some users, so I created a perl version of the xxd -i command line tool.

This works because everyone has perl in their build systems, whether they like it or not. Instead of burying it in the efitools package, here’s a copy of it for others to use if they want/need it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

#!/usr/bin/env perl
#
# xxdi.pl - perl implementation of 'xxd -i' mode
#
# Copyright 2013 Greg Kroah-Hartman <gregkh@linuxfoundation.org>
# Copyright 2013 Linux Foundation
#
# Released under the GPLv2.
#
# Implements the "basic" functionality of 'xxd -i' in perl to keep build
# systems from having to build/install/rely on vim-core, which not all
# distros want to do.  But everyone has perl, so use it instead.

use strict;
use warnings;
use File::Slurp qw(slurp);

my $indata = slurp(@ARGV ? $ARGV[0] : \*STDIN);
my $len_data = length($indata);
my $num_digits_per_line = 12;
my $var_name;
my $outdata;

# Use the variable name of the file we read from, converting '/' and '.
# to '_', or, if this is stdin, just use "stdin" as the name.
if (@ARGV) {
        $var_name = $ARGV[0];
        $var_name =~ s/\//_/g;
        $var_name =~ s/\./_/g;
} else {
        $var_name = "stdin";
}

$outdata .= "unsigned char $var_name\[] = {";

# trailing ',' is acceptable, so instead of duplicating the logic for
# just the last character, live with the extra ','.
for (my $key= 0; $key < $len_data; $key++) {
        if ($key % $num_digits_per_line == 0) {
                $outdata .= "\n\t";
        }
        $outdata .= sprintf("0x%.2x, ", ord(substr($indata, $key, 1)));
}

$outdata .= "\n};\nunsigned int $var_name\_len = $len_data;\n";

binmode STDOUT;
print {*STDOUT} $outdata;

Yes, I know I write perl code like a C programmer, that’s not an insult to me.

Sometimes you don’t have access to vim’s wonderful xxd tool, and you need to use it to generate some .c code based on a binary file. This happened to me recently when packaging up the EFI signing tools for Gentoo. Adding a build requirement of vim for a single autogenerated file was not an option for some users, so I created a perl version of the xxd -i command line tool.

This works because everyone has perl in their build systems, whether they like it or not. Instead of burying it in the efitools package, here’s a copy of it for others to use if they want/need it.

You might have heard, ownCloud Client 1.4.0 was released last week. It is available from our sync clients page for all major desktop platforms, investigate the Changelog.

Danimos Visual Guide has outlined the new stuff in the release already, so no need to repeat it here. You should install and try it, that seems to be the opinion of many people who tried it.

Also people who shared their critical view on the client very publically in the past are much more pleased now with 1.4.0. One example is a recent blog post on BITBlokes. It is a blog about all kind of topics around FOSS. I regularly read it and often share its opinions. He concludes very positively about the 1.4.0 client.

It is good to see the positive feedback overall. That shows a couple of things from my engineering point of view: The concentrated work we continously do on all parts of ownCloud pays off. That is obvious of course, but still nice to see. And our (also obvious) actions to improve code quality such as the consequent use of continous integration, code reviews and such helps to improve quality.

“People are always excited if releases come with GUI changes!” I heard people saying. Well, maybe, but that’s not the whole truth. It also proves for me again is how important UI design and UX is. Me as a knee-deep-developer have an interesting relationship to all UX topics: I always have an opinion. Often a strong opinion. But the results coming out of that have not always been the, well, the most optimal. Very fortunate on the client we work together with our UX guy Jan and the positive feedback also shows how good that is for the software.

But enough of release pride. There is more work to do: The bug tracker is still not empty, the list of feature ideas is long. We will continue to focus on correctness, stability and robustness of syncing, performance and useful features and work on a version 1.5 for you.

These are a couple of concrete points we’re focussing on for 1.5:

1. we already merged the client code on the new upstream sync version in git.
2. performace improvements through further reduction of the number of requests and more efficiency in database operations on the client.
3. we are working on a new propagator component that allows us to do the changes mentioned in 2 more easily.
4. File manager integration, which means havingn icons in Explorer, Dolphin and friends.

A more detailed list can be found at github.

Thank you for all your help and support. It’s big fun!

First let me explain why openSUSE Factory and not in fact Gentoo.

Factory points:

• + Much faster install/update
• + Better out of the box experience – no need to fiddle with everything to get it running
• + I can throw stuff to OBS and get it back fast as a binary when I am on battery
• – It has more issues than Gentoo stable in some areas
• – Unable to remove things like pulseaudio easily

Gentoo points:

• + More stable on stable – this will change a lot in near future if everything goes right for openSUSE
• + Faster system (matter of few percents)
• + Possibility to remove stuff you really don’t want
• – Compilation of everything – I update/install on battery often
• – Smaller chance to fix anything while I am traveling as I have to compile it on the box

So how should I install it? (Break your machine in 3 easy steps)

We will need ISO that is easy to grab from our build service.
Sometimes when we do fancy TM stuff on Yast side the ISO might not work, YAY, but fear less you can still grab latest released version and start from there.

Installation

You really don’t expect me to give you guide on how to install it right? Just get it to state where it reboots for the first login.

Actual migration

Check the repositories and verify that they are pointing on the Factory target, depends on media we went from but it is not hard to change.
Just remove the repositories installer provided us during the installation and put the following ones in (or edit them if you want) the yast:

mosquito:~ # zypper lr -u
# | Alias              | Name                     | Enabled | Refresh | URI                                                    
--+--------------------+--------------------------+---------+---------+--------------------------------------------------------
1 | repo-debug         | openSUSE-Factory-Debug   | No      | Yes     | http://download.opensuse.org/factory/repo/debug/       
2 | repo-non-oss       | openSUSE-Factory-Non-Oss | Yes     | Yes     | http://download.opensuse.org/factory/repo/non-oss/     
3 | repo-oss           | openSUSE-Factory-Oss     | Yes     | Yes     | http://download.opensuse.org/factory/repo/oss/         
4 | repo-source        | openSUSE-Factory-Source  | No      | Yes     | http://download.opensuse.org/factory/repo/src-oss/

Q: How should I add them in the zypper from cli?
A: zypper ar -f -n openSUSE-Factory-Oss http://download.opensuse.org/factory/repo/oss/

After this change we just migrate to the Factory by updating the whole distribution (even if nothing changed as the ISOs sometimes won’t build for a bit).
I recommend running this from CLI and not having anything running at the time, even if it won’t hurt much it is just safer :-)

zypper dup

As a note keep in mind that you should always run zypper dup on factory as we actually sometimes even downgrade and so on, so forget the zypper up there.

I did it. Now what?

Well mate, now you have rolling based binary distribution that is quite usable and working even despite all the openSUSE contributors trying to break it every day ;-)

If you find any problems during the usage just drop by on respective development IRC channel (#opensuse-kde/…) for the package that might be causing the trouble for you and ask the members if they already have fix and if yes if it is on its way to Factory. Then sit back for a while and in ~1 day enjoy your updated/fixed package.

If there is no fix, or nobody else willing to fix your issue (as that might happen as we all see our priorities differently), just try to file a bug or do even better. Hack on it and fix it yourself and get some cool stuff like irc cloak and email alias for being contributor (that does not come with the first fix obviously :P).

What are your current issues with the Factory?

Actually it is running quite well and if I find some issue I can annoy somebody/fix it pretty fast but let me list some things I am currently indiferent or are not annoying enough to fix:

• Czech translation of SUSE internals is sometimes funny
• Disk decrypt password dialog shows the disk name the way it overflow the screen, not a biggie
• Artwork for obvious reasons is moving target so sometimes it says 12.3 sometimes it changes every day and so on, that is not bug that is expected
• Akonadi is PITA, not a problem of Factory at all but it is biggest issue I have on all my machines
• Kernel updated too often – that was my bad as I approved the 3.11 too early and then I had to put in the fixes so you guys do not suffer :-)

As endnote I have to mention I udpated bit our wiki to explain who we are and what we do. It is more of work in progress but it should give at least some explanation how we achieve the Factory and what are the issues we face. We are always looking for contributors that want to work on fixing the current factory state or improving our tools. If you are bored help us out so 13.1 is awesome release for everyone…

Mandatory screenshot

Excuse the Nexus 4 camera, too lazy to reach out for normal one :-)