At litl we use Docker images to package and deploy our Room for More services, using our Galaxy deployment platform. This week I spent some time looking into how we might reduce the size of our images and speed up container deployments.

Most of our services are in Go, and thanks to the fact that compiled Go binaries are mostly-statically linked by default, it’s possible to create containers with very few files within. It’s surely possible to use these techniques to create tighter containers for other languages that need more runtime support, but for this post I’m only focusing on Go apps.

The old way

We built images in a very traditional way, using a base image built on top of Ubuntu with Go 1.4.2 installed. For my examples I’ll use something similar.

Here’s a Dockerfile:

FROM golang:1.4.2
EXPOSE 1717

RUN go get github.com/joeshaw/qotd

# Don't run network servers as root in Docker
USER nobody

CMD qotd

The golang:1.4.2 base image is built on top of Debian Jessie. Let’s build this bad boy and see how big it is.

$ docker build -t qotd .
...
Successfully built ae761b93e656

$ docker images qotd
REPOSITORY     TAG         IMAGE ID          CREATED           VIRTUAL SIZE
qotd           latest      ae761b93e656      3 minutes ago     520.3 MB

Yikes. Half a gigabyte. Ok, what leads us to a container this size?

$ docker history qotd
IMAGE               CREATED BY                                      SIZE
ae761b93e656        /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "qotd"]   0 B
b77d0ca3c501        /bin/sh -c #(nop) USER [nobody]                 0 B
a4b2a01d3e42        /bin/sh -c go get github.com/joeshaw/qotd       3.021 MB
c24802660bfa        /bin/sh -c #(nop) EXPOSE 1717/tcp               0 B
124e2127157f        /bin/sh -c #(nop) COPY file:56695ddefe9b0bd83   2.481 kB
69c177f0c117        /bin/sh -c #(nop) WORKDIR /go                   0 B
141b650c3281        /bin/sh -c #(nop) ENV PATH=/go/bin:/usr/src/g   0 B
8fb45e60e014        /bin/sh -c #(nop) ENV GOPATH=/go                0 B
63e9d2557cd7        /bin/sh -c mkdir -p /go/src /go/bin && chmod    0 B
b279b4aae826        /bin/sh -c #(nop) ENV PATH=/usr/src/go/bin:/u   0 B
d86979befb72        /bin/sh -c cd /usr/src/go/src && ./make.bash    97.4 MB
8ddc08289e1a        /bin/sh -c curl -sSL https://golang.org/dl/go   39.69 MB
8d38711ccc0d        /bin/sh -c #(nop) ENV GOLANG_VERSION=1.4.2      0 B
0f5121dd42a6        /bin/sh -c apt-get update && apt-get install    88.32 MB
607e965985c1        /bin/sh -c apt-get update && apt-get install    122.3 MB
1ff9f26f09fb        /bin/sh -c apt-get update && apt-get install    44.36 MB
9a61b6b1315e        /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
902b87aaaec9        /bin/sh -c #(nop) ADD file:e1dd18493a216ecd0c   125.2 MB

This is not a very lean container, with a lot of intermediate layers. To reduce the size of our containers, we did two additional steps:

(1) Every repo has a clean.sh script that is run inside the container after it is initially built. Here’s part of a script for one of our Ubuntu-based Go images:

apt-get purge -y software-properties-common byobu curl git htop man unzip vim \
python-dev python-pip python-virtualenv python-dev python-pip python-virtualenv \
python2.7 python2.7 libpython2.7-stdlib:amd64 libpython2.7-minimal:amd64 \
libgcc-4.8-dev:amd64 cpp-4.8 libruby1.9.1 perl-modules vim-runtime \
vim-common vim-tiny libpython3.4-stdlib:amd64 python3.4-minimal xkb-data \
xml-core libx11-data fonts-dejavu-core groff-base eject python3 locales \
python-software-properties supervisor git-core make wget cmake gcc bzr mercurial \
libglib2.0-0:amd64 libxml2:amd64

apt-get clean autoclean
apt-get autoremove -y

rm -rf /usr/local/go
rm -rf /usr/local/go1.*.linux-amd64.tar.gz
rm -rf /var/lib/{apt,dpkg,cache,log}/
rm -rf /var/{cache,log}

(2) We run Jason Wilder’s excellent docker-squash tool. It is especially helpful when combined with the clean.sh script above.

These steps are time intensive. Cleaning and squashing take minutes and dominate the overall build and deploy time.

In the end, we have built a mostly-statically linked Go binary sitting alongside an entire Debian or Ubuntu operating system. We can do better.

Separating containers for building and running

There have been a handful of good blog posts about how to do this in the past, including one by Atlassian this week. Here’s another one from Xebia, and another from Codeship.

However, all these posts focus on building a completely static Go binary. This means you eschew cgo by setting CGO_ENABLED=0 and the benefits that go along with it. On OS X, you lose access to the system’s SSL root CA certificates. On Linux, user.Current() from the os/user package no longer works. And in both cases you must use the Go DNS resolver rather than the one provided by the operating system. If you are not testing your application with CGO_ENABLED=0 prior to building a Docker container with it then you are not testing the code you ship.

We can use a few purpose-built base Docker images and the tricks from Jamie McCrindle’s Dockerception to build two separate Docker containers: one larger container to build our software and another smaller one to run it.

The builder

We create a Dockerfile.build, which is responsible for initializing the build environment and building the software:

FROM golang:1.4.2

RUN go get github.com/joeshaw/qotd
COPY / Dockerfile.run

# This command outputs a tarball which can be piped into
# `docker build -f Dockerfile.run -`
CMD tar -cf - -C / Dockerfile.run -C $GOPATH/bin qotd

This container, when run, will output a tarball to standard out, containing only our qotd binary and Dockerfile.run, used to build the runner.

Dynamically linked binary

Notice that we did not set CGO_ENABLED=0 here, so our binary is still dynamically linked against GNU libc:

$ ldd $GOPATH/bin/qotd
	linux-vdso.so.1 (0x00007ffea6b8a000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6e76e50000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6e76aa7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6e7706d000)

We need to run this binary in an environment that has glibc available to us. That means we cannot use stock BusyBox (which uses uClibc) or Alpine (which uses musl). However, the BusyBox distribution that ships with Ubuntu is linked against glibc, and that’ll be the foundation for our running container.

The busybox:ubuntu-14.04 image only has a root user, but you should never run network-facing servers as root, even in a container. Use my joeshaw/busybox-nonroot image — which adds a nobody user with UID 1 — instead.

The runner

Now we create a Dockerfile.run, which is responsible for creating the environment in which to run our app:

FROM joeshaw/busybox-nonroot
EXPOSE 1717

COPY qotd /bin/qotd

USER nobody
CMD qotd

Putting them together

First, create the builder image:

docker build -t qotd-builder -f Dockerfile.build .

Next, run the builder container, piping its output into the creation of the runner container:

docker run --rm qotd-builder | docker build -t qotd -f Dockerfile.run -

Now we have a qotd container which has the basic BusyBox environment, plus our qotd binary. The size?

$ docker images qotd
REPOSITORY     TAG         IMAGE ID          CREATED           VIRTUAL SIZE
qotd           latest      92e7def8f105      3 minutes ago     8.611 MB

Under 9 MB. Much improved. Better still, it doesn’t require squashing, which saves us a lot of time.

Conclusion

In this example, we were able to go from a 500 MB image built from golang:1.4.2 and containing a whole Debian installation down to a 9 MB image of just BusyBox and our binary. That’s a 98% reduction in size.

For one of our real services at litl, we reduced the image size from 300 MB (squashed) to 25 MB and the time to build and deploy the container from 8 minutes to 2. That time is now dominated by building the container and software, and not by cleaning and squashing the resulting image. We didn’t have to give up on using cgo and glibc, as some of its features are essential to us. If you’re using Docker to deploy services written in Go, this approach can save you a lot of time and disk space. Good luck!

In this article I will show you how to use text-based shell build into libgreattao(interface generation library). Libgreattao can works in one of three modes: normal mode(UI), shell mode(CLI), network mode(using tao protocol). In  each mode way libgreattao works is different. In this article I will focus on shell mode.

Libgreattao allows applications written in libgreattao to creating windows, adding abstracts(for this article purpose I will call it event path) to windows, setting abstract attributes__(poprawić: chodzi o skojarzenia)__. Examples of abstract attribute are: description, name, icon. In shell window class don’t do anything. It was used only to doing operations on windows. Access to window are possible by window path. Window path looks like this:

window_class_name[instance_number]

In network mode it’s also no matter about window class, but in UI(normal)  mode window class are very important. In normal mode window classes are associated with class file. Class file contains GUI elements definitions and process elements, which translates abstracts and abstract’s attributes. I attached example below:

void *window = tao_new_window("/desktop/dialogs/question_dialog");

tao_add_handler(window, "/message", NULL, NULL);
tao_add_handler(window, "/actions/exit", &exit_callback, value_for_callback);
tao_add_handler(window, "/actions/retry", &retry_callback, value_for_callbask2;

tao_set_hint(window, "/message", HINT_NAME, "Critical error");
tao_set_hint(window, "/message", HINT_DESCRIPTION "We cannot open file required by application");
tao_set_hint(window, "/action/exit", HINT_DESCRIPTION "Shutdown application");
tao_set_hint(window, "/action/retry", HINT_DESCRIPTION "Will retry");

In above example we’ve used three abstracts and one window. First abstract is related to message. Next two abstracts are related to actions user can perform. In case of /message abstract third and last parameter doesn’t have meaning. Why we attached only description to buttons(ehmmm… .actions)? The answer is name are not needed here. In shell mode we have using abstract by window path and abstract path/name(second argument) and window class file perhaps contains  rules for automatically attaching labels for buttons using part of abstract path/name. There’s no icons in shell mode.

Running application in shell mode

Libgreattao supports two switches to run libgreattao in shell mode. This it tao-shell-execute-file and tao-na-shell-interactive. In second parameter we have used string -na-, because we don’t supply any value for parameter. It is added, because maybe in future I would like add possibility to specify a value to an parameter.

Both of those switches makes libgreattao to run in shell mode. First of it will start script given as an argument. Second will start interactive mode. You can connect both of it to execute script first and run interactive mode once script ends of it execution.

You can exit from interactive mode by type command exit exit_code. If application doesn’t define exit function, application will ends with given exit code. In other case application will decide  what do with exit code.

About language

When creating this language I wish to follow lower price, but also by introducing many exciting features. By following this goal I decided to:

1. Integers are only strings – you cannot create integer variable
2. Other result of this decision are syntax of language – it looks like assember

Second point means there’s no operation marks – we have instructions, like =add, =text, etc. I attached example below:

=add variable_name value1 value2

In opposite with making my libgreattao language as simple as possible,  I added exciting features to it, like closures, dictionaries or variables expanding. If you want to have arrays support, you can include file called types in shell-utils directory of libgreattao sources directory. Arrays are kind of dictionaries, so I have used dictionaries to implement arrays.

Language build in libgreattao don’t have name still, but I created two names: “goto languiage” and “scope switcher”. You can suggest your’s name too. Command interpreter have about 6500 lines of code with function declarations, macrodefinitions and others.

Limitations

• There’s no possibility to use varnames as functions/code blocks names.
• Module names cannot start with ‘@’ or ‘.’ characters
• Name of variables cannot start with dots – only names of build in variables can start with dot
• Labels cannot start with dot – only special labels can start with dot
• Maximum command length can have 1024 characters

Typing commands and about commands

In interactive mode, commands should been typing with accepting each by enter. There’s exceptions from this rule. In this cases, command interpreter will wait for rest of command. This happens, when:

• Before newline character we put backslash
• You don’t close quatation marks
• Count of close parenthesis characters are not equal of count of open parenthesis characters

Attention: forwarding an character with backshalsh character makes forwarded character no special character. By putting enter in case shell awaits something different you can force shell to put newline character in place of pressing enter.

Parameters

Before you place parameter in code, excepting first, you should place command default separator. Inverted commas and apostrophe are normal characters, when you place before or after it different character than command default separator. For example:

echo a"

Will display

a"

For other example:

echo "a"

Will display

a

Variables substitution

For use variable from current context, you should use dollar sign. By placing variable name with forwarding dollar sign into inverted commas, apostrophe or parenthesis, or by forwarding dollar sign by backslash disallow to put variable value in place of name.
You can use many dollar signs to get value of variable visible on some level(it depends how many dollar signs you have use). For example one dollar sign means variable visible on current level. Two dollar signs means variable visible at one level bellow. You must be aware, while using multiple dollar sign, because mechanism for accessing variable in different level can be used only with special commands. One command which allow to use multiple dollar signs is scopelevel. There’s an example:

function a
=text a 1
function b
=text a 3
scopelevel -1 =add a $$a $a
endblock b
b
echo $a
endblock a
a

This code will display 4. Scope switching mechanism allow me to avoid unnecessary problems with implementing return statement and references, etc. Functions can return any number of results into any levels.

Typing blocks of code

After met keyword function or block and ascending name, shell will wait for instructions. Functions/blocks of code must be closed with keyword endblock – programmer can pass block/function name after enblock keyword to make code cleaner and to allow interpreter to detects errors.

Parenthesis

Parenthesis are normal strings, but starting with ( and ending with ). It have special purpose only, when using as command. In this case content of parenthesis are parset as normal command(without start and end characters). Parenthesis, which contains command should have default separator character after ( character. Example of usage of parenthesis are displayed below:

(\n echo
This is a string)

Code placed above will display “This is a string”. Another example to display “This is a string” below:

echo "This is a string"

(; echo;This is a string)

Empty parameter

To use empty parameter, you can do one of below:

• Place empty inverted commas or apostrophe
• Place two default separators for current command near, but only if it’s not space

For example:

echo ""

(\n echo

)

(In above example are placed two newline characters)

(; echo;;)
<h2>Selected commands</h2>
Libgreattao's shell are created to allow to control of application in shell way, so base command is command to run action associated with some abstract. It's run. As first parameter it required window and as second event path(abstract name). This is example for główny.o program, which will ran editor.o and exits from główny.o in next step. Sources of both programs are included in source of libgreattao.

run /desktop/dialogs/question_dialog[1] /actions/Button3
run /desktop/dialogs/question_dialog[1] /actions/exit

You can use variable to remember window in it:

=window main_window /desktop/dialogs/question_dialog[1]
run $main_window /actions/Button3
run $main_window /actions/exit

Using window path as first argument will make shell try to found window. This operation can fail. Errors are generated, when you give unexpected value, like below:

=add i $a "We cause error"

=Window command

=window variable_name window_path[window_number_with_this_path]

It will cause search for variable with given name in current block of code. If variable doesn't exist in current block of code, it will be created. In situation variable with given name exist with current block of code, but it has other type, it will cause error.
Commands starting with equal character are exposed to change variable value or creating new variables. This commands will write to variable with name given as second argument(first argument is command name). Commands starting with exclamation mark causes iteration over some values. For example, command below will show all abstracts for given window:

!event /desktop/dialogs/question_dialog[1] a "echo $a" 

Variables

Libgreattao's shell supports those kind of variables:

• String
• Window
• Dictionary

Two additional types are only other strings, but for commands requested some type, shell will check value can be converted into requested type. These two types are:

• Integer(signed)
• Natural number(unsigned)

For removing variable from current block of code, you should use unset.

=text text "This is sample text"
unset text
echo $text

This will cause error.

Expanding variables

Imagine that you have string contains two parameters, like window path and abstract name.We will now run the run command with those two parameters. You can achieve this by expanding this variable. Variable is expanded, when you pass .:~ after last dollar sign before first character of variable name. This is example:

=text parameters "/desktop/dialogs/question_dialog[1] /actions/Button3"
run $.:~parameters

Assigning and removing variables

When you change value of variable it looks like it was removed. I presented below list of action performed by shell, when you delete/assign variable:

• Deleting string - value are removed from memory; Creating string - value is copied
• Deleting window - decreasing reference count for some window - window will be removed once reference counter drops to 0 and program marks window to be removed; Creating window - increase reference count for window
• Deleting dictionary - value are removed from memory; Creating dictionary - all elements of dictionary are copied

Dictionaries

This article introduces how to convert string into window reference. I must also tell how to convert string into dictionary. String, which representing dictionary, should looks like:

[key1 = value1 [key2 = value2 ... [keyN = valueN]]]

For example:

window = /desktop/dialogs/question_dialog[1] action = /actions/exit

Spaces around equal sign are necessary. Also spaces between each element are necessary. You cannot use variables to create dictionary. To add value of variable to dictionary, you should create empty dictionary and use appendtodict. Example are below:

=dictionary name ""
appendtodict name key value

To obtain value from a dictionary you should use =fromdict, for example:

=fromdict a $dictionary key_name

Dollar sign before dictionary name is put specially, because this command take dictionary instead of dictionary name . In place of $dictionary you can put string that represents an dictionary and get value from it.
To remove element from dictionary, you should use removefromdict:

removefromdict dict_name key_name

You can expand dictionary in the same way as string. Order of putting values of elements into command line depends on order of modify/create elements in dictionary.

Blocks of code and functions

Libgreattao supports two kind of code blocks(but there's way to use additional code block using parenthesis and execute-all command), but these two blocks of code can realize the same tasks as blocks of code in structural paradigm. Each code block supports commands such like continue, break, return. I will describe these commands, when I will describing conditional statements. You can achieve break label or continue label by using scopelevel or scopebyname.
Which kind of code blocks are supported? Normal block of code and function. How each blocks looks like I show you before. What's a difference? Normal block of code is executed, when after an currently executed instruction interpreter meets block of code and we didn't use call, callonce, goto, throwerror, return, break or conditional instruction. Function are executed in invoke time and there can be recursion of function. Another think you need to know is that before function is executed, there's created dictionary $.unnamed_params and positional arguments are added to this dictionary. Additional difference is that only inside function you can use commands functionuse and usemodule.
Each function and code block must contains name, but name don't must be unique.

Labels and goto

Labels and goto instructions can be used to change program execution. You can use labels only inside functions or code blocks. It is dedicated need to reduce memory consumption and simplify interpreter code.
I put two examples of infinite loop below:

block infinity_loop
echo Started
=text message "Iteration "
=text i 1
: iteration
=cat how_many $message $i
echo $how_many
goto iteration
endblock infinity_loop

block infinite_loop
 
function next
scopelevel -1 goto iteration
endblock next
 
echo Started
=text message "Iteration "
=text i 1
: iteration
=cat how_many $message $i
echo $how_many
next
endblock infinite_loop

block iteration
=cat how_many $messages $i
echo $how_many
continue
endblock iteration
endblock infinity_loop

Return and break

Return and break are not the same. The meaning are very low intuitive. Break not necessary cause return from current block. There's way to call label in current block - you can use call and callonce. Difference is in break return from current invocation, but return will return from block. This is used, when interpreter calling .initialize label.

Function parameters

In tao's shell function have two kind of parameters. First is named parameters and second is positioned parameters. Positioned parameters should be placed after function name, separated by space. This is an example:

Function_name [parameter1 [parameter2 ... [parameterN]]]

You can pass named parameters using example below:

appendtodict .named_params name_of_first_parameter value1
appendtodict .named_params name_of_second_parameter value2
Function_name

It will works only, when variable .named_params already exist. If not, you may create it:

=dictionary .named_params ""

Variables with this name is exception for rule disallowing to put dot at beginning of variable name, when changing value or creating. Named parameters could been used to simulate this from object programming, because dictionary of named parameters would been used between many calls.
Programmer can deliver both named and positioned parameters.
You could use both named and positioned parameters.
How to use parameters inside function? First way is variable $.@. This variable contains what you write in command line after function name. This way isn't ideal. Substitution variable $.@ as function parameter will causes variable $.@ inside invoked function will have $.@ substring. To solve this problem, you should use function expandvariablesandcallfunction from shell-utils/functions file from libgreattao source. This is an example:

expandvariablesandcallfunction function_name $.@

You can use =shellparsestring and =shellgettokencount. This example will show how to display each element passes in command line:

function echo_all
=shellgettokencount i $.@
=text j 0
block process
ifintless $j $i
=add j $j 1
=shellparsestring wynik $j $.@
echo $wynik
continue
:  .false
endblock process
endblock echo_all

I will write about conditional instructions later.
How to access named and unnamed parameters directly? Function have access to two dictionaries. First is for named params($.named_params) and next is for unnamed params($.unnamed_params). Named params isn't in current context, but unnamed params is in current context. $.unnamed_params contains elements counted starting from 1. Function have access also to two variables: $.unnamed_param_count and $.named_param_count. You can enjoy with these variables, but I've created simpler method - paramrequires function. This function is defined in shell-utils/function in directory with libgreattao sources. In next example I will show definition of two run function. Second of it gets two parameters: abstract name and window. In next step it calls build in run function. First function simply invoke run.

function run
paramrequires positioned_window window positioned_string  path
buildin run $window $path
endblock
 
function run
errordisplaymode void
paramrequires any_string path any_window window
errordisplaymode all
buildin run $window $path
break
:  .error
errordisplaymode all
next
endblock

We had use error support and next instruction here. About errors I will write later. Instruction next call next function with the same name in current context. Thanks to these two functions, we can invoke run or run . Why? Because once we call run ..., second function is called and when we call run , paramrequires will throw error and next instruction is invoked.
I will now explain paramrequires function. From now parameters meaning all parameters without function name. For paramrequires you may pass natural number divisible by 2. Each unpaired parameter contains source, underline and type. Each paired parameter contain name of newly created variable. When these rules are not adjusted, paramrequires will throw error. Supported sources are: any(named and positioned), positioned, named. Supported types are: string, window, unsigned, signed. Paramrequires will first search in named dictionary and next in positioned dictionary for each parameters with any source. Search process in unnamed dictionary start for each parameter for next natural number, starting from 1.
You can call paramrequires from code of block inside function, like this:

scopetype readvariable scopebyname function_name paramrequires source1_type1 name1 ...

For example:

function error
=text error_code ""
block if_two_parameters
ifinteq $.unnamed_param_count 2
varscopelevel -1 error_code
varscopelevel -1 message
scopetype readvariable scopevbyname error paramrequires positioned_unsigned error_code positioned_string message
break
: .false
varscopelevel -1 message
scopetype readvariable scopevbyname błąd paramrequires positioned_string message
endblock if_two_parameters
=cat error_code $error_code " "
=cat message $error_code $message
echo $message
endblock

varscopelevel works similar to scopelevel, but makes all access to specified variable are related to specified level.

Switch block

Switch block of code looks very similar to other programming language. It will works as libgreattao block, but also like switch block:

block our_switch
ghostmode
goto $i
: 1
block 1
echo "have selected 1"
endblock
break
: 2
block 2
echo "have selected 2"
endblock
break
: 3
block 3
echo "have selected 3"
endblock
break
: .error
echo Default
endblock

In case of missing label, interpreter was jump to .error label. Ghostmode instruction makes our_switch block are invisible, so on error condition in subblock shell won't display Default.

Conditions and loop

In shell of libgreattao, conditions are normal instructions, but started with if prefix. They must behaves in special way - on false they should jump one level higher to .false label. Thats all! You can write your own conditional instructions. You can use label .true and .intialize. Instruction endconditionlist will call .initialize label(if exist) at first execution and next true label. You won't perhaps use .true label, because conditions are normal instructions - if condition are true, next instruction will be executed. This makes:

• Conditions are connected with something like and operator
• Conditions are executed until any condition are not true(this behavior are similar to Python and modern C)

I put an example of negation below:

function not
execute $.@
scopelevel -1 goto .false
: .false
endblock not

Negation are implemented in file shell-utils/logic in directory with libgreattao source. There's also implementation of OR and AND in the same file.

Assertions

W pliku shell-utils/logic of directory with libgreattao source are implementation of assertions. Assertion does nothing, but return if condition are true and jump to .error one level higher on false. I must also tell if label is missing, then shell jumps to .error label. There's also assert_verbose, which also on false print command line.

Loop examples

While:

block while
condition1
condition2
...
conditionN
instructions
continue
: .false
endblock

do-while

block do_while
instructions
condition1
condition2
...
conditionN
continue
: .false
endblock

repeat-until:

block repeat_until
: .false
instructions
end_condition
endblock

For:

block for
inicjalization
goto first_iteration
: iteration
ex.increase_counter
: first_iteration
condition1
condition2
...
conditionN
instructions
goto iteration
endblock

Lovely condition formatting

Condition list in libgreattao looks very good. There's gap with OR and AND instructions. You could use it in this way:

OR "ifstreq a b" "ifstreq a c"

But this doesn't looks good. The solution is below:

(\n OR
ifstreq a b
ifstreq a c)

You can insert new line character before OR or AND as newline character is default separator. You can do the same with close parenthesis mark. You should remember that putting newline characters near means the same as putting empty parameter.

Closures

Closures can be used to return function from other function. Returned function can use selected variables from context, which return function. Returned function can used only remembered parameters.
For example I will show function, which returns function with assembled number to be powered with parameter.

function create_power
=fromdict power $.unnamed_params 1
scopelevel -1 function power
functionuse power
=fromdict number $.unnamed_params 1
scopelevel -1 =text power 1
block wykonaj
ifintless 1 $power
=sub power $power 1
scopelevel -2 =mul power $$power $number
continue
: .false
endblock
endblock power
endblock create_power

Most interesting thing is functionuse. It remembers value of variable with given name, when parsing(when function is returned) and restore value to context of called function, when executed.
Code below throws error:

function a
echo $a
=text a s
functionuse a
endblock
a

Error will be caused on echo instruction, because variable a isn't set.

Modules

In libgreattao shell you can include files in this way:

script ścieżka_do_pliku

You can also load files as modules:

loadmodule ścieżka_do_pliku wewętrzna_nazwa_modułu

Each file, which includes libgreattao instructions libgreattao can be included by first or second method. Second method makes each symbol, which doesn't be exported used scopebylevel or similar, private. To understood this you can download source of tao-file-manager in second version from libgreattao home page and look at libgreattao module of this program. Possibility to create custom module are not introduced only to allow user create custom scripts, but it's also for programmer to allow user do not write long and many commands. For example libgreattao module of tao-file-manager supports functions, like ls, copy, rm, cd, pwd.

Usemodule instruction

Usemodule instruction are introduced especially to use inside blocks of code. This instruction makes module symbols(context) are inserted one level below. When block of code didn't use loadmodule before, module are inserted, but in other case module are replaced. This helps to create public function and private variables. To create public function. you need to use scopelevel functionuse to remember module name and usemodule.

Instruction hascurrentfiledescriptionflag

This instruction can be used to check that structure representing current file description has set specified flag. While starting interactive, debug mode or loading file, libgreattao will create special structure contains some information. This help with detects file is loaded by script command or as module.

Exceptions

Libgreattao supports four behavior on error condition: throw, continue, exit, debug. First do jump to label .error in current block, but only when current context doesn't have set error flag(this flag is reset by default). In addition, throw set this flag. If this flag is set, throw jumps to first .error label below and set error flag for each context between current context and context with first .error label below. There's one exception to this rule. When checking block does not have throw behavior, there is no propagation of error, but error behavior for this block are invoked. Continue does nothing in error condition. Exit stops execution of script. Debug run interactive debug mode.
Current way to support debug mode can be selected by code below:

errorbehaviour name

There's no way to set errorbehavior to debug, when program wasn't started with --tao-na-shell-enable-debug. In case program wasn't invoked with this switch and script tries to change debug behavior into debug, old behavior take precedence. Default behavior of support errors is throw.
Error behavior for current block is shipped in variable $.error_handling_behavior. Before block of code is invoked, there's $.error_handling_behavior variable append to new context. Value from $.error_handling_behavior_for_new_context are assigned to newly created $.error_handling_behavior, but there's one exception, when $.error_handling_behavior_for_new-context is inherit. In case the value is inherit, value from current $.error_handling_behaviour is assigned.
Error handling describes also flag about displaying errors with backtrace. To activate this flag, you should use this code:

errordisplaymode all

To deactivate this flag, you should use this code:

errordisplaymode void

One word

That's all!

In case you missed the openSUSE images for Docker got suddenly smaller.

During the last week I worked together with Marcus Schäfer (the author of KIWI) to reduce their size.

We fixed some obvious mistakes (like avoiding to install man pages and documentation), but we also removed some useless packages.

These are the results of our work:

• openSUSE 13.2 image: from 254M down to 82M
• openSUSE Tumbleweed image: from 267M down to 87M

Just to make some comparisons, the Ubuntu image is around 188M while the Fedora one is about 186M. We cannot obviously compete with images like busybox or Alpine, but the situation definitely improved!

Needless to say, the new images are already on the DockerHub.

Have fun!

After I was asked again today how to properly patch a rubygems package, here is a quick how to.

First things first we branch the package.

darix@tengu ~/osc/tmp % osc bco devel:languages:ruby:extensions/rubygem-net-ssh-session
A    home:darix:branches:devel:languages:ruby:extensions
A    home:darix:branches:devel:languages:ruby:extensions/rubygem-net-ssh-session
A    home:darix:branches:devel:languages:ruby:extensions/rubygem-net-ssh-session/net-ssh-session-0.1.6.gem
A    home:darix:branches:devel:languages:ruby:extensions/rubygem-net-ssh-session/rubygem-net-ssh-session.changes
A    home:darix:branches:devel:languages:ruby:extensions/rubygem-net-ssh-session/rubygem-net-ssh-session.spec
At revision 9edd872bc9121e1c68a71d057dfa607f.
Note: You can use "osc delete" or "osc submitpac" when done.

darix@tengu ~/osc/tmp % cd home:darix:branches:devel:languages:ruby:extensions/rubygem-net-ssh-session

So far nothing unexpected a gem file and spec+changes.

darix@tengu rubygem-net-ssh-session % l
total 28
drwxr-xr-x 3 darix users   122 Jul 23 12:21 ./
drwxr-xr-x 4 darix users    47 Jul 23 12:21 ../
-rw-r--r-- 1 darix users 13824 Jul 23 09:54 net-ssh-session-0.1.6.gem
drwxr-xr-x 2 darix users  4096 Jul 23 12:21 .osc/
-rw-r--r-- 1 darix users   143 Jul 23 10:09 rubygem-net-ssh-session.changes
-rw-r--r-- 1 darix users  1472 Jul 23 09:54 rubygem-net-ssh-session.spec

I prefer quilt for my patch handling but quilt setup doesn’t know how to handle our spec files yet. So some manual work required. This btw is also the reason why I include the series file in the source package.

x11vnc allows one to view remotely and interact with real X displays (i.e. a display corresponding to a physical monitor, keyboard, and mouse) with any VNC viewer. In this way it plays the role for Unix/X11 that WinVNC plays for Windows.

• log in to your target machine via ssh
• call "x11vnc -storepasswd" (attention: this will store the password in ~/.vnc/passwd)
• log out and from your "viewer machine" call "x11vnc_ssh username@targethost"

if vncviewer :$port $3; then break; fi

Something I wanted to share is a device I recently built. It is a complete net player for music files, served by a NAS. It’s now finished and “in production” now, so here is my report. [caption id=“attachment_616” align=“alignleft” width=“595”] The cherry ear net player, based on raspberry pi and HifiBerry Amp+.[/caption]

Hardware

The device is based on a Raspberry Model B+ with a HifiBerry Amp+. [caption id=“attachment_618” align=“alignleft” width=“446”] Cherry Ear backside with connectors and power supply[/caption]The Amp+ is a high-quality power amplifier that is mounted on the Raspberry mini computer, shortcutting the sub optimal audio system of the raspy. Only loudspeakers need to be connected, and this little combination is a very capable stereo audio system.

Software: Volumio

On the Raspberry runs a specialised linux distribution with a web based audio player for audio files and web radio. The name of this project is Volumio and it is one of my most favorite projects currently. What I like with it is that it is only a music player, and not a project that tries to manage all kind of media. Volumios user interface is accessible via web browser, and it is very user friendly, modern, pretty and clean.

It works on all size factors (from mobile phone to desktop) and is very easy to handle. Its a web interface to mpd which runs on the raspi for the actual music indexing and playing.

[caption id=“attachment_619” align=“alignleft” width=“400”] Volumio User Interface[/caption]The distribution all is running on is an optimized raspbian, with a variety of kernel drivers for audio output devices. Everything is pre configured. Once the volumio image is written to SD-Card, the Raspberry boots up and the device starts to play nicely basically after network and media source was configured through the web interface.

It is impressive how the Volumio project aims for absolute simplicity in use, but also allows to play around with a whole lot of interesting settings. A lot can be, but very little has to be configured.

Bottomline: Volumio is really a interesting project which you wanna try if you’re interested in these things.

The Housing

I built the housing out of cherry tree from the region here. I got it from friends growing cherries, and they gave a few very nice shelfs. It was sliced and planed to 10mm thickness. The dark inlay is teak which I got from 70’s furniture that was found on bulky waste years ago.

After having everything cut, the cherry wood was glued together, with some internal help construction from plywood. After the sanding and polishing, the box was done.

[caption id=“attachment_621” align=“alignright” width=“300”] Sizes [cm][/caption]The Raspberry and Hifiberry are mounted on a little construction attached to the metal back cover, together with the speaker connector. The metal cover is tightend with one screw, and the whole electronics can be taken off the wood box by unscrewing it.

At the bottom of the device there is a switching power supply that provides the energy for the player.

How to Operate?

The player is comletely operated from the web interface. The make all additional knobs and stuff superflous, the user even uses the tablet or notebook to adjust the volume. And since the device is never switched off, it does not even have a power button.

I combined it with Denon SC-M39 speakers. The music files come from a consumer NAS in the home network. The Raspberry is very well powerful enough for the software, and the Hifiberry device is surprisingly powerful and clean. The sound is very nice. Clear and fresh in the mid and heights, but still enough base, which never is annoying blurry or drones.

I am very happy about the result of this little project. Hope you like it :-)

The first part of this little blog series explained the basic operations of chunk file upload as we set it up for discussion. This part goes a bit beyond and talks about an addition to that, called announcing the upload.

With the processing described in the first part of the blog, the upload is done savely and with a clean approach, but it also has some drawbacks.

Most notably the server does not know the target filename of the uploaded file upfront. Also it does not know the final size or mimetype of the target file. That is not a problem in general, but imagine the following situation: A big file should be uploaded, which would exceed the users quota. That would only become an error for the user once all uploads happened, and the final upload directory is going to be moved on the final file name.

To avoid useless file transfers like that or to implement features like a file firewall, it would be good if the server would know these data at start of the upload and stop the upload in case it can not be accepted.

To achieve that, the client creates a file called _meta in /uploads/ before the upload of the chunks starts. The file contains information such as overall size, target file name and other meta information.

The server’s reply to the PUT of the _meta file can be a fail result code and error description to indicate that the upload will not be accepted due to certain server conditions. The client should check the result codes in order to avoid not necessary upload of data volume of which the final MOVE would fail anyway.

This is just a collection of ideas for an improved big file chunking protocol, nothing is decided yet. But now is the time to discuss. We’re looking forward to hearing your input.

The third and last part will describe how this plays into delta sync, which is especially interesting for big files, which are usually chunked.