You want to be an Open Source developer? Want to hack up some nasty code. Make everyone obey your order and take over the world. I was young back when I entered these shallow waters and how green I was back then.. oh boy!

My first app

I been coding long time maybe too long. First I was using Pascal but it was too high level for me and not cool at all. When I started using Linux KDE 1 was Koolest desktop environment on earth and CDE was de-facto environment on for the big boys. Soon after KDE 2 was released I started using KDE PIM suite because KMail is still neat application and Korganizer was way better than Evolution. I realized I like to format my happenings in list which wasn’t supported the way I liked. I thought, ‘Hey what If I write console application for that. I know how to code C and Java so C++ can’t that hard?’.

It was possible. QT2 was really great GUI library for writing applications. That time QT licensing was insane but today it’s much easier to understand. Writing applications with KDE libraries wasn’t all that hard. Application was all main-function and soon as I got it working I mailed to KDE mailing list. I don’t have that mail any more and can’t find it from the net but it was something like: ‘Hello, I’m the best QT-coder ever and I have this app called KonsoleKalendar‘. I got very friendly feedback and it got included into CVS. I though now I’m greatest coder ever lived!

Actually I maintained KonsoleKalendar only short time and as I said I wasn’t happy about licensing of QT2 (It didn’t help that it was badly written application like ever). Most wonderful and bizarre thing is that KonsoleKalendar this exists in KDE5 and it’s in much better shape than when I left it. Afterwards this was the main learning point about collaboration in Open Source project for me. In start of 2000 there weren’t Git nor there where any fancy GUIs for sending patches. People mailed each other and tried to cope with CVS/Subversion and KDE still is very friendly community if you compare it to many others.

Getting along the communities

If you ever are going to cope the Open Source world try to get along with community. There is as many communities as there is project and they can be friendly, neutral, unknown or hostile. There are several nearly or really hostile projects where bug reports and patches are rejected with making fun of you body organs or mom. Hostile projects seems to have same pattern. There one master of universe mega alpha coder that dictators everything and then people who needs that project or are somehow contributed something that coder number one things that they can exists. If you cross this kind of project you should have very nice shielding or some precious code gems to take as bounty. Remember many very successful project also have that mega dictator which have some urge to make things happen. problem is that most of these dictators only understand code and can’t speak anything else.

Unknown projects are strangest ones. Common thing is that plenty of people use them and commit actively bug reports. Only few people commit changes to version control but they don’t pay any attention to bug reports or mailing-lists. libSDL is this kind of project. People share their patches on bug database but it’s nearly impossible to get them to code base or I just don’t understand how SDL development goes.

Neutral communities are nice. They have good management and clean orders how to contribute. You can get you patch or bug report in if it’s good enough. Neutral communities are somehow uneasy to enter but if you prove to make good contribution they let you make your thing. So what is difference with neutral and friendly. In friendly community you can ask stupid questions and someone answers you nicely not just blank silence or some kind RTFM answers.

How to contribute?

I tried to open situation little bit above but I give you example from Mixxx community which I’m most active these days. Most of people pop in mailing list and they have the best idea ever but they haven’t looked Mixxx code so they don’t know if it’s possible or not. If their ideas are reasonable and that human being is ready to do work it’s mostly greeted with some advises and notes how it should be made. After hiatus that developer commits Pull Request or not.

If Pull request been made then rough ride starts. Reviewing code is not a bad thing and people who are making these code reviews in Mixxx knows application well and only wants best code in.  For green contributor it can be very frustrating. Basicly you have to have good code quality to get into Mixxx code base and you have to sign contributor paper.

How fast this happens? It really depends size of you contribution and how badly it’s done. People are doing code review in their spare time so it can be slow. If you just get out of blue with Pull request in Github you most probably won’t get nothing in. In Mixxx everything gets in with Pull Request (if it’s trivial then it’s just LGTM and merge style stuff).

If you didn’t read anything else this is what I wanted to say

What I have learned are in these three things: Know community you are going to work with (it takes time and motivation), Know how to contribute (what are rules) and try to cope with some level of frustration (They can be very hard on you if you ask stupid questions because most of them are stupid in Open Source world). If you just stop development because project things your code is pile of sh*t and you have to work on it more. Understand it’s pile of sh*t until they are happy and you have to make it to their standard.  Every community it is always Dystopia of commiters. They decide what goes in and what doesn’t. If it’s your project you can choose but if you are not in enough you just have to cope with it. You can Fork code and start new project but believe most time it’s more progressive to stay in same project and try to change that. If it’s not possible then just Fork it but you can end up like FFmpeg and AVConv situation.

Working in Open Source project is about communication. So talk in mailing list, work on bug reports, write documentation and review code. If you are silent then you don’t exist and remember if you can’t code but you like to something there is always plenty to do. If you like to contribute learn: Git (Gitbub), Mercurial (Bitbucket), Subversion, Bug reporting (Mantis, Bugzilla), code structure of project and debugging/reading others code or if you are sysadmin, web designer or something else there always something to do. Remember if you think you are correct and everyone else if incorrect you are the one who have to prove them incorrect. Flaming and trolling is nice and fun but not going to get project forward.

I end here and remember these are my own notes.

Many things can be modelled as finite state machines. Particularly things where you’d naturally use “state” in the name e.g. the current state of an order, or delivery status. We often model these as enums.

enum OrderStatus {
    Pending,
    CheckingOut,
    Purchased,
    Shipped,
    Cancelled,
    Delivered,
    Failed,
    Refunded
}

Enums are great for restricting our order status to only valid states. However, usually there are only certain transitions that are valid. We can’t go from Delivered to Failed. Nor would we go straight from Pending to Delivered. Maybe we can transition from Purchased to either Shipped or Cancelled.

Using enum values we cannot restrict to the transitions to only those that we desire. It would be nice to also let the compiler help us out by not letting us choose invalid transitions in our code.

We can, however, achieve this if we use a class hierarchy to represent our states instead, and it can still be fairly concise. There are other reasons for using regular classes, they allow us to store and even capture state from the surrounding context.

Here’s a way we could model the above enum as a class heirarchy with the valid transitions.

interface OrderStatus extends State {}
static class Pending     implements OrderStatus, BiTransitionTo {}
static class CheckingOut implements OrderStatus, BiTransitionTo {}
static class Purchased   implements OrderStatus, BiTransitionTo {}
static class Shipped     implements OrderStatus, BiTransitionTo {}
static class Delivered   implements OrderStatus, TransitionTo {}
static class Cancelled   implements OrderStatus {}
static class Failed      implements OrderStatus {}
static class Refunded    implements OrderStatus {}

We’ve declared an OrderStatus interface and then created implementations of OrderStatus for each valid state. We’ve then encoded the valid transitions as other interface implementations. There’s a TransitionTo<State> and BiTransitionTo<State1,State2>, or TriTransitionTo<State1,State2,State3> depending on the number of valid transitions from that state. We need differently named interfaces for different numbers of transitions because Java doesn’t support variance on the number of generic type parameters.

Compile-time checking valid transitions

Now we can create the TransitionTo/BiTransitionTo interfaces, which can give us the functionality to transition to a new state (but only if it is valid)

We might imagine an api like this where we can choose which state to transition to

new Pending()
    .transitionTo(CheckingOut.class)
    .transitionTo(Purchased.class)
    .transitionTo(Refunded.class) // <-- can we make this line fail to compile?

This turns out to be a little tricky, but not impossible, due to type erasure.

Let's try to implement BiTransitionTo interface with the two valid transition.

public interface BiTransitionTo {
    default T transitionTo(Class type) { ... }
    default U transitionTo(Class type) { ... }
}

Both of these transitionTo methods have the same erasure. So we can't do it quite like this. However, if we can encourage the consumer of our API to pass a lambda, there is a way to work around this same erasure problem.

So how about this API, where instead of passing class literals we pass constructor references. It looks similarly clean, but constructor references are basically lambdas so we can avoid type erasure.

new Pending()
    .transition(CheckingOut::new)
    .transition(Purchased::new)
    .transition(Refunded::new) // <-- Now we can make this fail to compile

In order to make this work the trick is to create a new interface type for each valid transition within our BiTransitionTo interface

public interface BiTransitionTo {
    interface OneTransition extends Supplier { }
    default T transition(OneTransition constructor) { ... }
    interface TwoTransition extends Supplier { }
    default U transition(TwoTransition constructor) { ... }
}

Supplier<T> is a functional interface in the java.util.function that is equivalent to a no-args constructor reference. By creating two interfaces that extend this we can overload the transition() method twice, allowing both methods to be passed a constructor reference without the two methods having the same erasure.

Runtime checking

Sometimes we might not be able to know at compile-time what state our statemachine is in. Perhaps a Customer has a field of type OrderStatus that we serialize to a database. We would need to be able to try a transition at runtime, and fail in some manner if the transition is not valid.

This is also possible using the TransitionTo<NewState> approach outlined above. Since supertype parameters are available at runtime, we can implement a tryTransition() method that uses reflection to check which transitions are permitted.

First we'll need a way of finding the valid transition types. We'll add it to our State base interface.

default List> validTransitionTypes() {
    return asList(getClass().getGenericInterfaces())
        .stream()
        .filter(type -> type instanceof ParameterizedType)
        .map(type -> (ParameterizedType) type)
        .filter(TransitionTo::isTransition) 
        .flatMap(type -> asList(type.getActualTypeArguments()).stream())
        .map(type -> (Class>) type)
        .collect(toList());
}

Note the isTransition filter. Since we have multiple transition interfaces - TransitionTo<T>, BiTransitionTo<T,U>, TriTransitionTo<T,U,V> etc, we need a way of marking them as all specifying transitions. I've used an annotation

@Retention(RUNTIME)
@Target(ElementType.TYPE)
public @interface Transition {

}
static boolean isTransition(ParameterizedType type) {
     Class> cls = (Class>)type.getRawType();
     return cls.getAnnotationsByType(Transition.class).length > 0;
}

@Transition
public interface TriTransitionTo...

Once we have validTransitionTypes() we can find which transitions are valid at runtime.

static class Pending implements OrderStatus, BiTransitionTo {}
@Test
public void finding_valid_transitions_at_runtime() {
    Pending pending = new Pending();
    assertEquals(
        asList(CheckingOut.class, Cancelled.class),
        pending.validTransitionTypes()
    );
}

Now that we have the valid types, tryTransition() needs to check whether the requested transition is to one of those types.

This is a little tricky, but since we're passing a lambda we make it a lambda-type-token and use reflection to find the type parameter of the lambda.

Our implementation then looks something like

interface NextState extends Supplier, MethodFinder {
    default Class type() {
        return (Class) getContainingClass();
    }
}
default  T tryTransition(NextState desired) {
    if (validTransitionTypes.contains(desired.type())) {
        return desired.get();
    }

    throw new IllegalStateTransitionException();
}

We can make it a bit nicer by allowing the caller to specify the exception to throw on error, like an Optional's orElseThrow. We can also allow the caller to ignore failed transitions.

@Test
public void runtime_checked_transition() {
    OrderStatus state = new Pending();
    assertTrue(state instanceof Pending);
    state = state
        .tryTransition(CheckingOut::new)
        .unchecked();
    assertTrue(state instanceof CheckingOut);
}

Since we've transitioned into a known state (or thrown an exception) with tryTransition we could then chain compile-time checked transitions on the end.

@Test
public void runtime_checked_transition() {
    OrderStatus state = new Pending();
    assertTrue(state instanceof Pending);
    state = state
        .tryTransition(CheckingOut::new)
        .unchecked()
        .transition(Purchased::new); // This will be permitted if the tryTransition succeeds.
    assertTrue(state instanceof CheckingOut);
}

We can even let people ignore transition failures if they wish, just by catching the exception and returning the original value.

@Test
public void runtime_checked_transition_ignoring_failure() {
    OrderStatus state = new Pending();
    assertTrue(state instanceof Pending);
    state = state
        .tryTransition(Refunded::new)
        .ignoreIfInvalid();
    assertFalse(state instanceof Refunded);
    assertTrue(state instanceof Pending);
}

Adding Behaviour

Since our states are classes, we can add behaviour to them.

For instance we could add a notifyProgress() method to our OrderStatus, with different implementations in each state.

interface OrderStatus extends State {
    default void notifyProgress(Customer customer, EmailSender sender) {}
}
static class Purchased implements OrderStatus, BiTransitionTo {
    public void notifyProgress(Customer customer, EmailSender emailSender) {
        emailSender.sendEmail("fulfillment@mycompany.com", "Customer order pending");
        emailSender.sendEmail(customer.email(), "Your order is on its way");
    }
}
...
OrderStatus status = new Pending();
status.notifyProgress(customer, sender); // Does nothing
status = status
    .tryTransition(CheckingOut::new)
    .unchecked()
    .transition(Purchased::new);
status.notifyProgress(customer, sender) ; // sends emails

Then we can call notifyProgress on any OrderStatus instance and it will notify differently depending on which implementation is active.

Internal Transitions

One of the ways to make most use of the typechecked transitions is to have the transitions internally within the state. e.g. in a state machine for the Regex "A+B" the A state can transition either

• Back to A
• To B
• To a match failure state

If we do this we can make them typechecked even though we don't know what the string we're matching in advance is.

static class A implements APlusB, TriTransitionTo {
    public APlusB match(String s) {
        if (s.length() < 1) return transition(NoMatch::new);
        if (s.charAt(0) == 'A') return transition(A::new).match(s.substring(1));
        if (s.charAt(0) == 'B') return transition(B::new).match(s.substring(1));
        return transition(NoMatch::new);
    }
}

Full example here

Capturing State

If we use non-static classes we could also capture state from the enclosing class. Supposing these OrderStatuses are contained within an Order class that already has an EmailSender available, we'd no longer need to pass in the emailSender and the customer to the notifyProgress() method.

class Order {
    EmailSender emailSender;
    Customer customer;
    class Purchased implements OrderStatus, BiTransitionTo {
        public void notifyProgress() {
            emailSender.sendEmail("fulfillment@mycompany.com", "Customer order pending");
            emailSender.sendEmail(customer.email(), "Your order is on its way");
        }
    }
}

Guards

Another feature we might want is the ability to execute some code before transitioning into a new state or after transitioning into a new state. This is something we can add to our base State interface. Let's add two methods beforeTransition() and afterTransition()

interface State {
    default void afterTransition(T from) {}
    default void beforeTransition(T to) {}
}

We can then update our transition implementation to invoke these guard methods before and after a transition occurs.

We could use this to log all transitions into the Failure state.

class Failed implements OrderStatus {
    @Override
    public void afterTransition(OrderStatus from) {
        failureLog.warning("Oh bother! failed from " + from.getClass().getSimpleName());
    }
}

We could also combine state capturing and guard methods to build a stateful-state machine that updates its state on transition instead of just returning the new state. Here's an example where we use a guard method to mutate the state of lightSwitch after each transition.

class LightExample {
    Switch lightSwitch = new Off();

    public class Switch implements State {
        @Override
        public void afterTransition(Switch from) {
            LightExample.this.lightSwitch = Switch.this;
        }
    }
    public class On extends Switch implements TransitionTo {}
    public class Off extends Switch implements TransitionTo {}

    @Test
    public void stateful_switch() {
        assertTrue(lightSwitch instanceof Off);
        lightSwitch.tryTransition(On::new).ignoreIfInvalid();
        assertTrue(lightSwitch instanceof On);
        lightSwitch.tryTransition(Off::new).ignoreIfInvalid();
        assertTrue(lightSwitch instanceof Off);
    }
}

Show me the code

The code is on github if you'd like to play with it/see full executable examples

Another use of lambda parameter reflection could be to write html inline in Java. It allows us to create builders like this, in Java, where we’d previously have to use a language like Kotlin and a library like Kara.

String doc =
    html(
        head(
            meta(charset -> "utf-8"),
            link(rel->stylesheet, type->css, href->"/my.css"),
            script(type->javascript, src -> "/some.js")
        ),
        body(
            h1("Hello World", style->"font-size:200%;"),
            article(
                p("Here is an interesting paragraph"),
                p(
                    "And another",
                    small("small")
                ),
                ul(
                    li("An"),
                    li("unordered"),
                    li("list")
                )
            )
        )
    ).asString();

Which generates html like

  
    


    
  

  
    
Hello World

    
Here is an interesting paragraph

    
And anothersmall

    

• An
• unordered
• list

Code Generation

Why would you do this? Well we could do code generation. e.g. we can programmatically generate paragraphs.

body(
    asList("one","two","three")
        .stream()
        .map(number -> "Paragraph " + number)
        .map(content -> p(content))
)

Help from the Type System

We can also use the Java type system to help us write valid code.

It will be a compile time error to specify an invalid attribute for link rel.

It’s a compile time error to omit a mandatory tag

It’s also a compile time error to have a body tag inside a p tag, because body is not phrasing content.

We can also ensure that image sizes are in pixels

Safety

We can also help reduce injection attacks when inserting content from users into our markup, by having the DSL html-encoding any content passed in.

e.g.

assertEquals(
    "
<script src="attack.js"></script>
", 
    p("").asString()
);

How does it work?

See this previous blogpost that shows how to get lambda parameter names with reflection. This allows us to specify the key value pairs for html attributes quite cleanly.

I’ve created an Attribute type that converts a lambda to a html attribute.

public interface Attribute extends NamedValue {
    default String asString() {
        return name() + "=\"" + value()+"\"";
    }
}

For the tags themselves we declare an interface per tag, with a heirarchy to allow certain tags in certain contexts. For example Small is PhrasingContent and can be inside a P tag.

public interface Small extends PhrasingContent {
    default Small small(String content) {
        return () -> tag("small", content);
    }
}

To make it easy to have all the tag names available in the context without having to static import lots of things, we can create a “mixin” interface that combines all the tags.

public interface HtmlDsl extends
        Html,
        Head,
        Body,
        Link,
        Meta,
        P,
        Script,
        H1,
        Li,
        Ul,
        Article,
        Small,
        Img
        ...

Then where we want to write html we just make our class implement HtmlDsl (Or we could staticly import the methods instead.

We can place restrictions on which tags are valid using overloaded methods for the tag names. e.g. HTML

public interface Html extends NoAttributes {
    default Html html(Head head, Body body) { 
    ...

and restrict the types of attributes using enums or other wrapper types. Here Img tag can only have measurements in pixels

public interface Img extends NoChildren {
    default Img img(Attribute src, Attribute dim1, Attribute dim2) {
    ...

All the code is available on github to play with. Have a look at this test for executable examples. n.b. it’s just a proof of concept at this point. Only sufficient code exists to illustrate the examples in this blog post.

What other creative uses can you find for parameter reflection?

This was supposed to be survival guide to open source and free software world but I realized I’m not that good citizen of open source world that I can give any advises to others. What I’m giving are hint’s what I have learn along the years. So why I’m not very good open source citizen? I read several projects mail lists but only topics that I like and make contributions but not with rage but when I feel like it. I answer few mails that I receive about open source in limited time frame that I have (which sometimes can be too long) and use many projects with out giving anything back. I prefer license to steal and freedom as value not as in beer.

What is license for?

Wear your tin-hats and make securing spells because here we go. In modern world everything is for sale and everything you can image will be stolen and in-incorporated in nuclear bomb or used in mass destruction of human beings. This ain’t new feature in human society. For example fire have been probably one man thing for long time (and I can just imagine how many jokes you can make about that poor fellow and that even poorer soul that first roasted something) after a while it widespread all over the world and same goes with wheel. They were invented somewhere and someone took them in use without giving a dime to the inventor. Is it fair? No it’s not. Bit harsh and unfair to this original guy but again this is how ball is played.

What this is related to licensing and what is the big deal of open source and free software anyway if this the Status Quo? What are the licenses for? Believe or not they are important agreements! It’s not secret that most of the Github repositories are still without proper license. That is why they launch: http://choosealicense.com/. Take time and study a bit or if you don’t have a time I’ll tell how I see things.

Free software

There is Free Software movement which is constructed around GNU project and FSF (Free Software Foundation). Most significant person in Free software is Richard Stallman (Yes that hansom guy with a beard) . If you want more history please read it from FSF site they know it more better than me. Main principals in Free software are Freedom, Freedom to share and make sure that everyone else have that freedom also.

Free Software licenses are commonly known as Copyleft licenses and most know is Gnu Public license (Actually version 3.0 ain’t that popular). All these licenses share a same thing. You will always have 4 freedoms:

• Freedom 0 – the freedom to use the work
• Freedom 1 – the freedom to study the work
• Freedom 2 – the freedom to copy and share the work with others
• Freedom 3 – the freedom to modify the work, and the freedom to distribute modified and therefore derivative works.

What this means (and I’m not a lawyer so don’t blame me if you get sued) is that you can make your changes to code, used it in nuclear bomb but if you release your bomb to big public (or make it available only in machine readable form) you must release also the code changes you have made to original code. There is eternal fight do these changes have to be delivered to upstream project and do they have to be suitable to attach into original code base.

This why every distribution is releasing their source packages (which ain’t bad thing at all) because GPL demands it. Copyleft tries to make sure you will get source code of binary if you demand it but it can be made available only for those who demand it and even in printed out form as A4 papers.

Copyleft there different opinions how these license articles really apply and there is plenty of violations like Allwinner. Still most of them a settled out of the court. One of the biggest GPL (which wasn’t about GPL license at all) that get in the court was (or is it still?) Linux kernel vs SCO. It was only possible because code was freely available and everyone could study it.

Because you didn’t read the anything above. Remember one thing Copyleft is VERY restrictive license. If you are using some library which is using Copyleft license of any form and you doing in-house development make sure you apply license demands before releasing your work or it could get real nasty. Main thing is: you have the right to use the source but same time you have to provide everyone else the same rights and no this doesn’t mean you have to have version control or bug tracker.

Puppy projects: Linux, Libreoffice, Blender and GIMP

Open source

Open Source licenses are widespread and there is plenty more of them than there is Copyleft licenses. How they compare Copyleft? Open Soure licenses tend to give you all rights so they are more liberal. Most popular licenses are MIT, Apache License 2.0 and BSD license. Why they are popular? It’s because they are simple and all these three gives you right to do what ever you want with these files and choose you want to contribute back. If you choose some not popular license you have to make sure you are compatible with GPL without that there is no game.

Why even have license then if you don’t care what people do with your stuff? Choosing liberate Open Source license is not letting everyone ‘steal’ your work it’s about making sure that they know what they can do with it and you are the owner of the rights. Project without any license or release as Public domain is most dangerous ever because you don’t know is this some kind of bomb project that author is waiting to spread and then he or she is making demand on court with statement ‘hey I have this code on Internet and this my new EULA! GIVE ME YOU €€€ (or $$$ sometimes £££) YOU FEALTY ROBBERS!’. These licenses most cases tell what kind of warranty you have and every time it’s next to nothing.

Why then choose Open Source license and not the Free Software? It’s about the attitude: Freedom makes freedom happen. With Free Software you are forced to be free and with Open Source you can choose what is your freedom level and of course you need something to fight about.

Puppy projects: Docker, FreeBSD, Apache HTTP server

What about something else that is not code

Back in days creative works was a weak spot of licensing. Free Software and Open Source license are very fitting to code but they are not very well fitting to creative work. This is why Creative Commons (commonly known as CC) was created. There is suitable licenses for sharing you images, writings or what ever. They have Copyleft style licenses and then more liberal ones. Take you time and find what fits to your project.

Lengthy post but nothing much to said

I think I’ll rest my case here. Next time I think I’ll post about contributing code and remember these are my OWN observations. If they are incorrect please let me know or if you hate me because I like Systemd and liberal licenses you can tell that too. Remember it’s your project and you can choose any license in the world you like.

I am very late with this, but I still wanted to write a few words about Akademy 2015.

First of all: It was an awesome conference! Meeting all the great people involved with KDE and seeing who I am working with (as in: face to face, not via email) was awesome. We had some very interesting discussions on a wide variety of topics, and also enjoyed quite some beer together. Particularly important to me was of course talking to Daniel Vrátil who is working on XdgApp, and Aleix Pol of Muon Discover fame.

Also meeting with the other Kubuntu members was awesome – I haven’t seen some of them for about 3 years, and also met many cool people for the first time.

My talk on AppStream/Limba went well, except that I got slightly confused by the timer showing that I had only 2 minutes left, after I had just completed the first half of my talk. It turned out that the timer was wrong

Another really nice aspect was to be able to get an insight into areas where I am usually not involved with, like visual design. It was really interesting to learn about the great work others are doing and to talk to people about their work – and I also managed to scratch an itch in the Systemsettings application, where three categories had shown the same icon. Now Systemsettings looks like it is supposed to be, finally

The only thing I could maybe complain about was the weather, which was more Scotland/Wales like than Spanish – but that didn’t stop us at all, not even at the social event outside. So I actually don’t complain

We also managed to discuss some new technical stuff, like AppStream for Kubuntu, and plenty of other things that I’ll write about in a separate blog post.

Generally, I got so many new impressions from this year’s Akademy, that I could write a 10-pages long blogpost about it while still having to leave out things.

Kudos to the organizers of this Akademy, you did a fantastic job! I also want to thank the Ubuntu community for funding my trip, and the Kubuntu community for pushing me a little to attend :-).

KDE is a great community with people driving the development of Free Software forward. The diversity of people, projects and ideas in KDE is a pleasure, and I am very happy to be part of this community.

KDE has recently released the newest Release Candidate of the Applications 15.08 release. Among the new features and changes of this release, there is a technology preview of the new KF5-based KDE PIM suite (including reworked, faster Akonadi internals), new applications ported to KF5 (the most notable ones being Dolphin and Ark). After some consideration and thinking on how to allow users to test this release without affecting their setups too much, the openSUSE community KDE team is happy to bring this latest RC to openSUSE Tumbleweed and openSUSE 13.2¹.

To install this new release, add the KDE:Applications repository either using YaST or zypper. One special mention is the PIM suite: as upstream KDE labels it as a technology preview, we decided to allow installation only with an explicit choice of the user. To do so, one should install the kmail5 package and the akonadi-server package (other components of the PIM suite are also there, with the 5 suffix): this operation will uninstall the 4.14 packages (but not remove any local data) and install the new version of mail and PIM. To go back, install akonadi-runtime and the respective packages without the 5 suffix (e.g., kmail, korganizer).

It is essential for upstream KDE to have proper bug reports, in particular for PIM, so please report any issues you find. If instead you find a bug in the packaging, turn over to openSUSE’s Bugzilla.

At litl we use Docker images to package and deploy our Room for More services, using our Galaxy deployment platform. This week I spent some time looking into how we might reduce the size of our images and speed up container deployments.

Most of our services are in Go, and thanks to the fact that compiled Go binaries are mostly-statically linked by default, it’s possible to create containers with very few files within. It’s surely possible to use these techniques to create tighter containers for other languages that need more runtime support, but for this post I’m only focusing on Go apps.

The old way

We built images in a very traditional way, using a base image built on top of Ubuntu with Go 1.4.2 installed. For my examples I’ll use something similar.

Here’s a Dockerfile:

FROM golang:1.4.2
EXPOSE 1717

RUN go get github.com/joeshaw/qotd

# Don't run network servers as root in Docker
USER nobody

CMD qotd

The golang:1.4.2 base image is built on top of Debian Jessie. Let’s build this bad boy and see how big it is.

$ docker build -t qotd .
...
Successfully built ae761b93e656

$ docker images qotd
REPOSITORY     TAG         IMAGE ID          CREATED           VIRTUAL SIZE
qotd           latest      ae761b93e656      3 minutes ago     520.3 MB

Yikes. Half a gigabyte. Ok, what leads us to a container this size?

$ docker history qotd
IMAGE               CREATED BY                                      SIZE
ae761b93e656        /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "qotd"]   0 B
b77d0ca3c501        /bin/sh -c #(nop) USER [nobody]                 0 B
a4b2a01d3e42        /bin/sh -c go get github.com/joeshaw/qotd       3.021 MB
c24802660bfa        /bin/sh -c #(nop) EXPOSE 1717/tcp               0 B
124e2127157f        /bin/sh -c #(nop) COPY file:56695ddefe9b0bd83   2.481 kB
69c177f0c117        /bin/sh -c #(nop) WORKDIR /go                   0 B
141b650c3281        /bin/sh -c #(nop) ENV PATH=/go/bin:/usr/src/g   0 B
8fb45e60e014        /bin/sh -c #(nop) ENV GOPATH=/go                0 B
63e9d2557cd7        /bin/sh -c mkdir -p /go/src /go/bin && chmod    0 B
b279b4aae826        /bin/sh -c #(nop) ENV PATH=/usr/src/go/bin:/u   0 B
d86979befb72        /bin/sh -c cd /usr/src/go/src && ./make.bash    97.4 MB
8ddc08289e1a        /bin/sh -c curl -sSL https://golang.org/dl/go   39.69 MB
8d38711ccc0d        /bin/sh -c #(nop) ENV GOLANG_VERSION=1.4.2      0 B
0f5121dd42a6        /bin/sh -c apt-get update && apt-get install    88.32 MB
607e965985c1        /bin/sh -c apt-get update && apt-get install    122.3 MB
1ff9f26f09fb        /bin/sh -c apt-get update && apt-get install    44.36 MB
9a61b6b1315e        /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
902b87aaaec9        /bin/sh -c #(nop) ADD file:e1dd18493a216ecd0c   125.2 MB

This is not a very lean container, with a lot of intermediate layers. To reduce the size of our containers, we did two additional steps:

(1) Every repo has a clean.sh script that is run inside the container after it is initially built. Here’s part of a script for one of our Ubuntu-based Go images:

apt-get purge -y software-properties-common byobu curl git htop man unzip vim \
python-dev python-pip python-virtualenv python-dev python-pip python-virtualenv \
python2.7 python2.7 libpython2.7-stdlib:amd64 libpython2.7-minimal:amd64 \
libgcc-4.8-dev:amd64 cpp-4.8 libruby1.9.1 perl-modules vim-runtime \
vim-common vim-tiny libpython3.4-stdlib:amd64 python3.4-minimal xkb-data \
xml-core libx11-data fonts-dejavu-core groff-base eject python3 locales \
python-software-properties supervisor git-core make wget cmake gcc bzr mercurial \
libglib2.0-0:amd64 libxml2:amd64

apt-get clean autoclean
apt-get autoremove -y

rm -rf /usr/local/go
rm -rf /usr/local/go1.*.linux-amd64.tar.gz
rm -rf /var/lib/{apt,dpkg,cache,log}/
rm -rf /var/{cache,log}

(2) We run Jason Wilder’s excellent docker-squash tool. It is especially helpful when combined with the clean.sh script above.

These steps are time intensive. Cleaning and squashing take minutes and dominate the overall build and deploy time.

In the end, we have built a mostly-statically linked Go binary sitting alongside an entire Debian or Ubuntu operating system. We can do better.

Separating containers for building and running

There have been a handful of good blog posts about how to do this in the past, including one by Atlassian this week. Here’s another one from Xebia, and another from Codeship.

However, all these posts focus on building a completely static Go binary. This means you eschew cgo by setting CGO_ENABLED=0 and the benefits that go along with it. On OS X, you lose access to the system’s SSL root CA certificates. On Linux, user.Current() from the os/user package no longer works. And in both cases you must use the Go DNS resolver rather than the one provided by the operating system. If you are not testing your application with CGO_ENABLED=0 prior to building a Docker container with it then you are not testing the code you ship.

We can use a few purpose-built base Docker images and the tricks from Jamie McCrindle’s Dockerception to build two separate Docker containers: one larger container to build our software and another smaller one to run it.

The builder

We create a Dockerfile.build, which is responsible for initializing the build environment and building the software:

FROM golang:1.4.2

RUN go get github.com/joeshaw/qotd
COPY / Dockerfile.run

# This command outputs a tarball which can be piped into
# `docker build -f Dockerfile.run -`
CMD tar -cf - -C / Dockerfile.run -C $GOPATH/bin qotd

This container, when run, will output a tarball to standard out, containing only our qotd binary and Dockerfile.run, used to build the runner.

Dynamically linked binary

Notice that we did not set CGO_ENABLED=0 here, so our binary is still dynamically linked against GNU libc:

$ ldd $GOPATH/bin/qotd
	linux-vdso.so.1 (0x00007ffea6b8a000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6e76e50000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6e76aa7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6e7706d000)

We need to run this binary in an environment that has glibc available to us. That means we cannot use stock BusyBox (which uses uClibc) or Alpine (which uses musl). However, the BusyBox distribution that ships with Ubuntu is linked against glibc, and that’ll be the foundation for our running container.

The busybox:ubuntu-14.04 image only has a root user, but you should never run network-facing servers as root, even in a container. Use my joeshaw/busybox-nonroot image — which adds a nobody user with UID 1 — instead.

The runner

Now we create a Dockerfile.run, which is responsible for creating the environment in which to run our app:

FROM joeshaw/busybox-nonroot
EXPOSE 1717

COPY qotd /bin/qotd

USER nobody
CMD qotd

Putting them together

First, create the builder image:

docker build -t qotd-builder -f Dockerfile.build .

Next, run the builder container, piping its output into the creation of the runner container:

docker run --rm qotd-builder | docker build -t qotd -f Dockerfile.run -

Now we have a qotd container which has the basic BusyBox environment, plus our qotd binary. The size?

$ docker images qotd
REPOSITORY     TAG         IMAGE ID          CREATED           VIRTUAL SIZE
qotd           latest      92e7def8f105      3 minutes ago     8.611 MB

Under 9 MB. Much improved. Better still, it doesn’t require squashing, which saves us a lot of time.

Conclusion

In this example, we were able to go from a 500 MB image built from golang:1.4.2 and containing a whole Debian installation down to a 9 MB image of just BusyBox and our binary. That’s a 98% reduction in size.

For one of our real services at litl, we reduced the image size from 300 MB (squashed) to 25 MB and the time to build and deploy the container from 8 minutes to 2. That time is now dominated by building the container and software, and not by cleaning and squashing the resulting image. We didn’t have to give up on using cgo and glibc, as some of its features are essential to us. If you’re using Docker to deploy services written in Go, this approach can save you a lot of time and disk space. Good luck!

In this article I will show you how to use text-based shell build into libgreattao(interface generation library). Libgreattao can works in one of three modes: normal mode(UI), shell mode(CLI), network mode(using tao protocol). In  each mode way libgreattao works is different. In this article I will focus on shell mode.

Libgreattao allows applications written in libgreattao to creating windows, adding abstracts(for this article purpose I will call it event path) to windows, setting abstract attributes__(poprawić: chodzi o skojarzenia)__. Examples of abstract attribute are: description, name, icon. In shell window class don’t do anything. It was used only to doing operations on windows. Access to window are possible by window path. Window path looks like this:

window_class_name[instance_number]

In network mode it’s also no matter about window class, but in UI(normal)  mode window class are very important. In normal mode window classes are associated with class file. Class file contains GUI elements definitions and process elements, which translates abstracts and abstract’s attributes. I attached example below:

void *window = tao_new_window("/desktop/dialogs/question_dialog");

tao_add_handler(window, "/message", NULL, NULL);
tao_add_handler(window, "/actions/exit", &exit_callback, value_for_callback);
tao_add_handler(window, "/actions/retry", &retry_callback, value_for_callbask2;

tao_set_hint(window, "/message", HINT_NAME, "Critical error");
tao_set_hint(window, "/message", HINT_DESCRIPTION "We cannot open file required by application");
tao_set_hint(window, "/action/exit", HINT_DESCRIPTION "Shutdown application");
tao_set_hint(window, "/action/retry", HINT_DESCRIPTION "Will retry");

In above example we’ve used three abstracts and one window. First abstract is related to message. Next two abstracts are related to actions user can perform. In case of /message abstract third and last parameter doesn’t have meaning. Why we attached only description to buttons(ehmmm… .actions)? The answer is name are not needed here. In shell mode we have using abstract by window path and abstract path/name(second argument) and window class file perhaps contains  rules for automatically attaching labels for buttons using part of abstract path/name. There’s no icons in shell mode.

Running application in shell mode

Libgreattao supports two switches to run libgreattao in shell mode. This it tao-shell-execute-file and tao-na-shell-interactive. In second parameter we have used string -na-, because we don’t supply any value for parameter. It is added, because maybe in future I would like add possibility to specify a value to an parameter.

Both of those switches makes libgreattao to run in shell mode. First of it will start script given as an argument. Second will start interactive mode. You can connect both of it to execute script first and run interactive mode once script ends of it execution.

You can exit from interactive mode by type command exit exit_code. If application doesn’t define exit function, application will ends with given exit code. In other case application will decide  what do with exit code.

About language

When creating this language I wish to follow lower price, but also by introducing many exciting features. By following this goal I decided to:

1. Integers are only strings – you cannot create integer variable
2. Other result of this decision are syntax of language – it looks like assember

Second point means there’s no operation marks – we have instructions, like =add, =text, etc. I attached example below:

=add variable_name value1 value2

In opposite with making my libgreattao language as simple as possible,  I added exciting features to it, like closures, dictionaries or variables expanding. If you want to have arrays support, you can include file called types in shell-utils directory of libgreattao sources directory. Arrays are kind of dictionaries, so I have used dictionaries to implement arrays.

Language build in libgreattao don’t have name still, but I created two names: “goto languiage” and “scope switcher”. You can suggest your’s name too. Command interpreter have about 6500 lines of code with function declarations, macrodefinitions and others.

Limitations

• There’s no possibility to use varnames as functions/code blocks names.
• Module names cannot start with ‘@’ or ‘.’ characters
• Name of variables cannot start with dots – only names of build in variables can start with dot
• Labels cannot start with dot – only special labels can start with dot
• Maximum command length can have 1024 characters

Typing commands and about commands

In interactive mode, commands should been typing with accepting each by enter. There’s exceptions from this rule. In this cases, command interpreter will wait for rest of command. This happens, when:

• Before newline character we put backslash
• You don’t close quatation marks
• Count of close parenthesis characters are not equal of count of open parenthesis characters

Attention: forwarding an character with backshalsh character makes forwarded character no special character. By putting enter in case shell awaits something different you can force shell to put newline character in place of pressing enter.

Parameters

Before you place parameter in code, excepting first, you should place command default separator. Inverted commas and apostrophe are normal characters, when you place before or after it different character than command default separator. For example:

echo a"

Will display

a"

For other example:

echo "a"

Will display

a

Variables substitution

For use variable from current context, you should use dollar sign. By placing variable name with forwarding dollar sign into inverted commas, apostrophe or parenthesis, or by forwarding dollar sign by backslash disallow to put variable value in place of name.
You can use many dollar signs to get value of variable visible on some level(it depends how many dollar signs you have use). For example one dollar sign means variable visible on current level. Two dollar signs means variable visible at one level bellow. You must be aware, while using multiple dollar sign, because mechanism for accessing variable in different level can be used only with special commands. One command which allow to use multiple dollar signs is scopelevel. There’s an example:

function a
=text a 1
function b
=text a 3
scopelevel -1 =add a $$a $a
endblock b
b
echo $a
endblock a
a

This code will display 4. Scope switching mechanism allow me to avoid unnecessary problems with implementing return statement and references, etc. Functions can return any number of results into any levels.

Typing blocks of code

After met keyword function or block and ascending name, shell will wait for instructions. Functions/blocks of code must be closed with keyword endblock – programmer can pass block/function name after enblock keyword to make code cleaner and to allow interpreter to detects errors.

Parenthesis

Parenthesis are normal strings, but starting with ( and ending with ). It have special purpose only, when using as command. In this case content of parenthesis are parset as normal command(without start and end characters). Parenthesis, which contains command should have default separator character after ( character. Example of usage of parenthesis are displayed below:

(\n echo
This is a string)

Code placed above will display “This is a string”. Another example to display “This is a string” below:

echo "This is a string"

(; echo;This is a string)

Empty parameter

To use empty parameter, you can do one of below:

• Place empty inverted commas or apostrophe
• Place two default separators for current command near, but only if it’s not space

For example:

echo ""

(\n echo

)

(In above example are placed two newline characters)

(; echo;;)
<h2>Selected commands</h2>
Libgreattao's shell are created to allow to control of application in shell way, so base command is command to run action associated with some abstract. It's run. As first parameter it required window and as second event path(abstract name). This is example for główny.o program, which will ran editor.o and exits from główny.o in next step. Sources of both programs are included in source of libgreattao.

run /desktop/dialogs/question_dialog[1] /actions/Button3
run /desktop/dialogs/question_dialog[1] /actions/exit

You can use variable to remember window in it:

=window main_window /desktop/dialogs/question_dialog[1]
run $main_window /actions/Button3
run $main_window /actions/exit

Using window path as first argument will make shell try to found window. This operation can fail. Errors are generated, when you give unexpected value, like below:

=add i $a "We cause error"

=Window command

=window variable_name window_path[window_number_with_this_path]

It will cause search for variable with given name in current block of code. If variable doesn't exist in current block of code, it will be created. In situation variable with given name exist with current block of code, but it has other type, it will cause error.
Commands starting with equal character are exposed to change variable value or creating new variables. This commands will write to variable with name given as second argument(first argument is command name). Commands starting with exclamation mark causes iteration over some values. For example, command below will show all abstracts for given window:

!event /desktop/dialogs/question_dialog[1] a "echo $a" 

Variables

Libgreattao's shell supports those kind of variables:

• String
• Window
• Dictionary

Two additional types are only other strings, but for commands requested some type, shell will check value can be converted into requested type. These two types are:

• Integer(signed)
• Natural number(unsigned)

For removing variable from current block of code, you should use unset.

=text text "This is sample text"
unset text
echo $text

This will cause error.

Expanding variables

Imagine that you have string contains two parameters, like window path and abstract name.We will now run the run command with those two parameters. You can achieve this by expanding this variable. Variable is expanded, when you pass .:~ after last dollar sign before first character of variable name. This is example:

=text parameters "/desktop/dialogs/question_dialog[1] /actions/Button3"
run $.:~parameters

Assigning and removing variables

When you change value of variable it looks like it was removed. I presented below list of action performed by shell, when you delete/assign variable:

• Deleting string - value are removed from memory; Creating string - value is copied
• Deleting window - decreasing reference count for some window - window will be removed once reference counter drops to 0 and program marks window to be removed; Creating window - increase reference count for window
• Deleting dictionary - value are removed from memory; Creating dictionary - all elements of dictionary are copied

Dictionaries

This article introduces how to convert string into window reference. I must also tell how to convert string into dictionary. String, which representing dictionary, should looks like:

[key1 = value1 [key2 = value2 ... [keyN = valueN]]]

For example:

window = /desktop/dialogs/question_dialog[1] action = /actions/exit

Spaces around equal sign are necessary. Also spaces between each element are necessary. You cannot use variables to create dictionary. To add value of variable to dictionary, you should create empty dictionary and use appendtodict. Example are below:

=dictionary name ""
appendtodict name key value

To obtain value from a dictionary you should use =fromdict, for example:

=fromdict a $dictionary key_name

Dollar sign before dictionary name is put specially, because this command take dictionary instead of dictionary name . In place of $dictionary you can put string that represents an dictionary and get value from it.
To remove element from dictionary, you should use removefromdict:

removefromdict dict_name key_name

You can expand dictionary in the same way as string. Order of putting values of elements into command line depends on order of modify/create elements in dictionary.

Blocks of code and functions

Libgreattao supports two kind of code blocks(but there's way to use additional code block using parenthesis and execute-all command), but these two blocks of code can realize the same tasks as blocks of code in structural paradigm. Each code block supports commands such like continue, break, return. I will describe these commands, when I will describing conditional statements. You can achieve break label or continue label by using scopelevel or scopebyname.
Which kind of code blocks are supported? Normal block of code and function. How each blocks looks like I show you before. What's a difference? Normal block of code is executed, when after an currently executed instruction interpreter meets block of code and we didn't use call, callonce, goto, throwerror, return, break or conditional instruction. Function are executed in invoke time and there can be recursion of function. Another think you need to know is that before function is executed, there's created dictionary $.unnamed_params and positional arguments are added to this dictionary. Additional difference is that only inside function you can use commands functionuse and usemodule.
Each function and code block must contains name, but name don't must be unique.

Labels and goto

Labels and goto instructions can be used to change program execution. You can use labels only inside functions or code blocks. It is dedicated need to reduce memory consumption and simplify interpreter code.
I put two examples of infinite loop below:

block infinity_loop
echo Started
=text message "Iteration "
=text i 1
: iteration
=cat how_many $message $i
echo $how_many
goto iteration
endblock infinity_loop

block infinite_loop
 
function next
scopelevel -1 goto iteration
endblock next
 
echo Started
=text message "Iteration "
=text i 1
: iteration
=cat how_many $message $i
echo $how_many
next
endblock infinite_loop

block iteration
=cat how_many $messages $i
echo $how_many
continue
endblock iteration
endblock infinity_loop

Return and break

Return and break are not the same. The meaning are very low intuitive. Break not necessary cause return from current block. There's way to call label in current block - you can use call and callonce. Difference is in break return from current invocation, but return will return from block. This is used, when interpreter calling .initialize label.

Function parameters

In tao's shell function have two kind of parameters. First is named parameters and second is positioned parameters. Positioned parameters should be placed after function name, separated by space. This is an example:

Function_name [parameter1 [parameter2 ... [parameterN]]]

You can pass named parameters using example below:

appendtodict .named_params name_of_first_parameter value1
appendtodict .named_params name_of_second_parameter value2
Function_name

It will works only, when variable .named_params already exist. If not, you may create it:

=dictionary .named_params ""

Variables with this name is exception for rule disallowing to put dot at beginning of variable name, when changing value or creating. Named parameters could been used to simulate this from object programming, because dictionary of named parameters would been used between many calls.
Programmer can deliver both named and positioned parameters.
You could use both named and positioned parameters.
How to use parameters inside function? First way is variable $.@. This variable contains what you write in command line after function name. This way isn't ideal. Substitution variable $.@ as function parameter will causes variable $.@ inside invoked function will have $.@ substring. To solve this problem, you should use function expandvariablesandcallfunction from shell-utils/functions file from libgreattao source. This is an example:

expandvariablesandcallfunction function_name $.@

You can use =shellparsestring and =shellgettokencount. This example will show how to display each element passes in command line:

function echo_all
=shellgettokencount i $.@
=text j 0
block process
ifintless $j $i
=add j $j 1
=shellparsestring wynik $j $.@
echo $wynik
continue
:  .false
endblock process
endblock echo_all

I will write about conditional instructions later.
How to access named and unnamed parameters directly? Function have access to two dictionaries. First is for named params($.named_params) and next is for unnamed params($.unnamed_params). Named params isn't in current context, but unnamed params is in current context. $.unnamed_params contains elements counted starting from 1. Function have access also to two variables: $.unnamed_param_count and $.named_param_count. You can enjoy with these variables, but I've created simpler method - paramrequires function. This function is defined in shell-utils/function in directory with libgreattao sources. In next example I will show definition of two run function. Second of it gets two parameters: abstract name and window. In next step it calls build in run function. First function simply invoke run.

function run
paramrequires positioned_window window positioned_string  path
buildin run $window $path
endblock
 
function run
errordisplaymode void
paramrequires any_string path any_window window
errordisplaymode all
buildin run $window $path
break
:  .error
errordisplaymode all
next
endblock

We had use error support and next instruction here. About errors I will write later. Instruction next call next function with the same name in current context. Thanks to these two functions, we can invoke run or run . Why? Because once we call run ..., second function is called and when we call run , paramrequires will throw error and next instruction is invoked.
I will now explain paramrequires function. From now parameters meaning all parameters without function name. For paramrequires you may pass natural number divisible by 2. Each unpaired parameter contains source, underline and type. Each paired parameter contain name of newly created variable. When these rules are not adjusted, paramrequires will throw error. Supported sources are: any(named and positioned), positioned, named. Supported types are: string, window, unsigned, signed. Paramrequires will first search in named dictionary and next in positioned dictionary for each parameters with any source. Search process in unnamed dictionary start for each parameter for next natural number, starting from 1.
You can call paramrequires from code of block inside function, like this:

scopetype readvariable scopebyname function_name paramrequires source1_type1 name1 ...

For example:

function error
=text error_code ""
block if_two_parameters
ifinteq $.unnamed_param_count 2
varscopelevel -1 error_code
varscopelevel -1 message
scopetype readvariable scopevbyname error paramrequires positioned_unsigned error_code positioned_string message
break
: .false
varscopelevel -1 message
scopetype readvariable scopevbyname błąd paramrequires positioned_string message
endblock if_two_parameters
=cat error_code $error_code " "
=cat message $error_code $message
echo $message
endblock

varscopelevel works similar to scopelevel, but makes all access to specified variable are related to specified level.

Switch block

Switch block of code looks very similar to other programming language. It will works as libgreattao block, but also like switch block:

block our_switch
ghostmode
goto $i
: 1
block 1
echo "have selected 1"
endblock
break
: 2
block 2
echo "have selected 2"
endblock
break
: 3
block 3
echo "have selected 3"
endblock
break
: .error
echo Default
endblock

In case of missing label, interpreter was jump to .error label. Ghostmode instruction makes our_switch block are invisible, so on error condition in subblock shell won't display Default.

Conditions and loop

In shell of libgreattao, conditions are normal instructions, but started with if prefix. They must behaves in special way - on false they should jump one level higher to .false label. Thats all! You can write your own conditional instructions. You can use label .true and .intialize. Instruction endconditionlist will call .initialize label(if exist) at first execution and next true label. You won't perhaps use .true label, because conditions are normal instructions - if condition are true, next instruction will be executed. This makes:

• Conditions are connected with something like and operator
• Conditions are executed until any condition are not true(this behavior are similar to Python and modern C)

I put an example of negation below:

function not
execute $.@
scopelevel -1 goto .false
: .false
endblock not

Negation are implemented in file shell-utils/logic in directory with libgreattao source. There's also implementation of OR and AND in the same file.

Assertions

W pliku shell-utils/logic of directory with libgreattao source are implementation of assertions. Assertion does nothing, but return if condition are true and jump to .error one level higher on false. I must also tell if label is missing, then shell jumps to .error label. There's also assert_verbose, which also on false print command line.

Loop examples

While:

block while
condition1
condition2
...
conditionN
instructions
continue
: .false
endblock

do-while

block do_while
instructions
condition1
condition2
...
conditionN
continue
: .false
endblock

repeat-until:

block repeat_until
: .false
instructions
end_condition
endblock

For:

block for
inicjalization
goto first_iteration
: iteration
ex.increase_counter
: first_iteration
condition1
condition2
...
conditionN
instructions
goto iteration
endblock

Lovely condition formatting

Condition list in libgreattao looks very good. There's gap with OR and AND instructions. You could use it in this way:

OR "ifstreq a b" "ifstreq a c"

But this doesn't looks good. The solution is below:

(\n OR
ifstreq a b
ifstreq a c)

You can insert new line character before OR or AND as newline character is default separator. You can do the same with close parenthesis mark. You should remember that putting newline characters near means the same as putting empty parameter.

Closures

Closures can be used to return function from other function. Returned function can use selected variables from context, which return function. Returned function can used only remembered parameters.
For example I will show function, which returns function with assembled number to be powered with parameter.

function create_power
=fromdict power $.unnamed_params 1
scopelevel -1 function power
functionuse power
=fromdict number $.unnamed_params 1
scopelevel -1 =text power 1
block wykonaj
ifintless 1 $power
=sub power $power 1
scopelevel -2 =mul power $$power $number
continue
: .false
endblock
endblock power
endblock create_power

Most interesting thing is functionuse. It remembers value of variable with given name, when parsing(when function is returned) and restore value to context of called function, when executed.
Code below throws error:

function a
echo $a
=text a s
functionuse a
endblock
a

Error will be caused on echo instruction, because variable a isn't set.

Modules

In libgreattao shell you can include files in this way:

script ścieżka_do_pliku

You can also load files as modules:

loadmodule ścieżka_do_pliku wewętrzna_nazwa_modułu

Each file, which includes libgreattao instructions libgreattao can be included by first or second method. Second method makes each symbol, which doesn't be exported used scopebylevel or similar, private. To understood this you can download source of tao-file-manager in second version from libgreattao home page and look at libgreattao module of this program. Possibility to create custom module are not introduced only to allow user create custom scripts, but it's also for programmer to allow user do not write long and many commands. For example libgreattao module of tao-file-manager supports functions, like ls, copy, rm, cd, pwd.

Usemodule instruction

Usemodule instruction are introduced especially to use inside blocks of code. This instruction makes module symbols(context) are inserted one level below. When block of code didn't use loadmodule before, module are inserted, but in other case module are replaced. This helps to create public function and private variables. To create public function. you need to use scopelevel functionuse to remember module name and usemodule.

Instruction hascurrentfiledescriptionflag

This instruction can be used to check that structure representing current file description has set specified flag. While starting interactive, debug mode or loading file, libgreattao will create special structure contains some information. This help with detects file is loaded by script command or as module.

Exceptions

Libgreattao supports four behavior on error condition: throw, continue, exit, debug. First do jump to label .error in current block, but only when current context doesn't have set error flag(this flag is reset by default). In addition, throw set this flag. If this flag is set, throw jumps to first .error label below and set error flag for each context between current context and context with first .error label below. There's one exception to this rule. When checking block does not have throw behavior, there is no propagation of error, but error behavior for this block are invoked. Continue does nothing in error condition. Exit stops execution of script. Debug run interactive debug mode.
Current way to support debug mode can be selected by code below:

errorbehaviour name

There's no way to set errorbehavior to debug, when program wasn't started with --tao-na-shell-enable-debug. In case program wasn't invoked with this switch and script tries to change debug behavior into debug, old behavior take precedence. Default behavior of support errors is throw.
Error behavior for current block is shipped in variable $.error_handling_behavior. Before block of code is invoked, there's $.error_handling_behavior variable append to new context. Value from $.error_handling_behavior_for_new_context are assigned to newly created $.error_handling_behavior, but there's one exception, when $.error_handling_behavior_for_new-context is inherit. In case the value is inherit, value from current $.error_handling_behaviour is assigned.
Error handling describes also flag about displaying errors with backtrace. To activate this flag, you should use this code:

errordisplaymode all

To deactivate this flag, you should use this code:

errordisplaymode void

One word

That's all!

In case you missed the openSUSE images for Docker got suddenly smaller.

During the last week I worked together with Marcus Schäfer (the author of KIWI) to reduce their size.

We fixed some obvious mistakes (like avoiding to install man pages and documentation), but we also removed some useless packages.

These are the results of our work:

• openSUSE 13.2 image: from 254M down to 82M
• openSUSE Tumbleweed image: from 267M down to 87M

Just to make some comparisons, the Ubuntu image is around 188M while the Fedora one is about 186M. We cannot obviously compete with images like busybox or Alpine, but the situation definitely improved!

Needless to say, the new images are already on the DockerHub.

Have fun!