Recently the German computer magazin C’t posted an article about file sync solutions (“Unter eigener Regie”, C’t 23, 2018) with native sync clients. The article was pretty positive about the FOSS solution of… Nextcloud! I was wondering why they had not choosen ownCloud’s client as my feeling is that ownCloud is way more busy and innovative developing the desktop client for file synchronization together with community.

[caption id=“attachment_943” align=“alignright” width=“338”] Code lines changed as of Nov. 10, 2018[/caption]

That motivated me to do some investigation what the Nextcloud client actually consists of (at due date Nov. 10, 2018). I was looking into the NC desktop client git repoository grouped the numbers of commits of people that can be associated clearly to either the ownCloud- or Nextcloud project, or to “other communities” or machine commits. Since the number of commits could be misleading (maybe some commits are huge?) I did the same exercise with numbers of changed lines of code.

When looking on the changed lines, the first top six contributors to the Nextcloud desktop client are only active in the ownCloud project. Number seven is an “other community” contributor whos project the client was based on in the beginning. Number eight to eleven go to Nextcloud, with a low percentage figure.

[caption id=“attachment_944” align=“alignnone” width=“666”] # of commits to the Nextcloud Desktop repository as of Nov. 10, 2018[/caption]

As a result, far more than 80% of the changed lines of the Nextcloud client is actually work that ownClouders did (not considering the machine commits). In the past, and also today. The number would be even higher if it considered all the commits that go into the NC repo with an NC author, but are actually ownCloud patches where the original author got lost on the way by merging them through a NC branch. It looks like the Nextcloud developers were actually adding less commits to their client than all “other community” developers so far.

No wonder, it is a fork, you might think, and that is of course true. However, to my taste these numbers are not reflecting a “constructive” fork driving things forward when we talk about sync technology.

That is all fine, and I am proud that the work we do in ownCloud is actually stimulating two projects, with different focus areas nowadays. On the other hand, I would appreciate if the users of the technology would take a closer look to understand who really innovates, drives things forward and also fixes the nasty bugs in the stack. As a matter of fairness, that should be acknowledged. That is the motivation that keeps free software contributors busy and communities proud.

Christmas is around the corner, but I couldn’t wait and I have already been trying the new Ruby release! :christmas_tree: :tada: Ruby 2.6, apart from efficiency improvements, which include the initial implementation of a just-in-time compiler, brings us many new cool features. Taking advance of the cold outside, let’s discover some of them! :snowflake:

Array#union & Array#difference

This Ruby version is particularly special to me because it includes my two new methods for the Array class, which I presented in my talks at EuRuKo and Brighton Ruby. Array#union and Array#difference are just readable alias for Array#| and Array#- respectively when only having two arrays:

[1, 3, 5, 7, 9].union([2, 3, 4, 5, 6]) #=> [1, 3, 5, 7, 9, 2, 4, 6]
[1, 1, 3, 3, 5, 7, 9].difference([3, 4, 7]) #=> [1, 1, 5, 9]

Array#union is also equivalent to combine Array#concat and Array#uniq (with the difference that concat modifies the array), but more readable. But what is really important about those new methods, are the gains in efficiency when having more than two arrays. We need some Benchmark now. :wink:

Using Array.new(num_elements) { Random.rand(20_100_000) } to create four arrays with 20,000,000, 30,000,000, 8,000,000 and 25,000,000 elements, those are the times for the different options:

• (array1 | array2 | array3 | array4) ~ 20.043 seconds
• array1.union(array2, array3, array4) ~ 13.390 seconds
• array1.concat(array2, array3, array4).uniq ~ 20.633 seconds

So please, stop using concat + uniq :pray: and let’s finally refactor some Ruby code! :tada:

Hash#merge with multiple parameters

Hash#merge and Hash#merge! were only able to merge two hashes at the same time. With Ruby 2.6, we can merge as many hashes as we want at once, which provides an performance improvement similar to Array#union and Array#difference when merging several big hashes.

{ a: 1, b: 2 }.merge({ b: 3, c: 4 }, { d: 5 }) #=> {:a=>1, :b=>3, :c=>4, :d=>5}

Enumerable#to_h

Enumerable#to_h accepts a block which allows to do things that used to require iterating manually or prepending map, being consequently more efficient. The following examples illustrate how it works:

(1..5).to_h { |x| [x, x % 3] } #=> {1=>1, 2=>2, 3=>0, 4=>1, 5=>2}
[1, 1, 2, 4].to_h { |x| [x, true] } #=> {1=>true, 2=>true, 4=>true}

Endless range

Ruby 2.6 introduces endless ranges like (1..), (74..) and (1..nil), whose size is infinite:

(1..).size => Infinity

It provides a nice alternative to [1..-1] to retrieve a slice up to the end of an Array or String:

[1, 2, 3, 4, 5][3..] => [4, 5]
'hello world'[6..] => "world"

It can also be combined with other methods to produce such elegant code:

# Selects from an array a range and from an element to the end
[0, 1, 2, 3, 4, 5, 6].values_at(1..3, 5..) #=> [1, 2, 3, 5, 6]

# Iterates over more than one array at the same time with index
[:a, :b, :c].zip([10, 37, 30], 1..) { |x1, x2, index| puts "#{index}: #{x1} #{x2}" }

# Multiples of π less than 100
(1..).lazy.map { |x| x * Math::PI }.take_while{ |x| x < 100 }.force

Last, it can be used to write infinit loops with index: (1..).each { |n| ... }, however we had already several alternatives to do this. I personally find the following ones more readable:

1.step { |n| ... }
n = 1; loop { ...; n += 1 }

Note: Be careful when trying infinite ranges, as if you iterate over a range (for example with map) and forget lazy (or to stop it for instance with break), it may consume all the memory of your computer, as it happened to me. :sweat: You can use ulimit to limit the memory available for the console where you execute irb when playing with it to avoid this (on Linux and macOS). :nerd_face:

Range#=== uses cover? instead of include?

Although Matz always aims to prioritize backwards compatibility, in this case performance has won. Range#=== uses now cover? instead of include? to check if an object is an element of the range. This is a reasonable but also breaking change, which modifies the behaviour of statements like:

(Date.today..(Date.today + 1)) === DateTime.now #=> true (false in Ruby 2.5)

Take into account that Range#=== is used in case, so the following example also change its behaviour:

case DateTime.now
when (Date.today..(Date.today + 1))
  'you are in Ruby 2.6!'
else
  'update to Ruby 2.6'
end

Proc#« & Proc#»

We can now compose procs both from left to right (>>) and from right to left (<<):

double = proc { |x| x * 2 }
increment = proc { |x| x + 1 }
(double >> increment).call(2) #=> 5
(double << increment).call(2) #=> 6

Procs with multiple arguments are also supported:

f = proc { |x, y| x + y }
g = proc { |x, y| [x * 2, -(y * 3)] }
(f << g).call(2, 1) #=> 1

Enumerator#+ and Enumerable#chain

Ruby 2.6 introduces Enumerator::Chain, a new class to represent a chain of enumerables that works as a single enumerator as well as methods to create chain of enumerables: Enumerable#chain and Enumerator#+:

chain = (1..3).chain([7, 8]) #=> #<Enumerator::Chain: [1..3, [7, 8]]>
chain.to_a #=> [1, 2, 3, 7, 8]

chain = (1..3).each + [7, 8] #=> #<Enumerator::Chain: [#<Enumerator: 1..3:each>, [7, 8]]>
chain.map { |x| x * 2 } #=> [2, 4, 6, 14, 16]

Dir#each_child & Dir#children

The Dir class had already the class methods children and each_child. Ruby 2.6 add the equivalent instance methods:

d = Dir.new("/home/ana") #=> #<Dir:/home/ana/>
d.children #=> [".config", "bin", ".gitignore", ".vimrc", "github", "cat_pictures", ".bashrc", "VirtualBox VMs"]

non-ASCII constant names

Constant names can now start with non-ASCII capital letters. I am not sure how useful this is, but you can do funny things like:

class Σ♥²; end

In few days, you can update to Ruby 2.6 and have fun too! :wink:

One more year, Google Summer of Code (GSoC), a mentoring program in which openSUSE helps university students contribute to open source project, has come to an end. So, before 2018 ends as well and we start preparing for the new edition of GSoC, it is time to speak about all the great things that happened this year. :sparkles:

GSoC 2018

Let’s start by the most important part, our students’ work. Our successful students, Ankush Malik and Liana Xu, have spent 3 months hacking on openSUSE projects, during which they have written a lot of impressive code. But GSoC is much more than code, it is about learning, having fun and becoming part of the openSUSE community. Both Ankush and Liana claim that it has been an inspiring experience and are really thankful for the support they received from their mentors.

Ankush’s improvements in the Hackweek tool are noticeable. In his last weeks of work, he focused on giving all projects the chance to be viewed and on implementing a mailer. Check the last chapter of his GSoC journey: https://medium.com/@ankushmalik631/gsoc-wrap-up-86bba25bbb6d

Liana has been working on integrating Cloud Input in ibus-libpinyin and she has learnt ton about GNOME developer libraries and functions. Read about her project from her own words in her last blog post: https://liana.hillwoodhome.net/2018/08/13/about-programming-life-during-gsoc

It is also worthwhile mentioning the great collaboration which makes me particularly proud of the openSUSE community. Thanks to the help of contributors all around the world, the blog posts about GSoC were shared, republished and translated to languages like Japanese, Spanish and Indonesian. We also had the help of the openSUSE Indonesian community to design “Thank you” mugs to send to our mentors. Look how cute they are: :cupid:

Last but not least, I would like to thank our passionate mentors and admins who took out time from their busy schedules to guide the students, our motivated students for their willingness to learn and good work, Douglas DeMaio who helped with shipping packages and organization, Google and specifically its open source team not only for the program itself but also for the well organized conference, SUSE for their support (especially economically), TSP which allows our students to attend the openSUSE conference every year, the blog post translators, Pramasta Ramadha and the rest of the designers who helped with the mugs design and everybody else who made this year GSoC amazing. Because of people like you, openSUSE is much more than just software. :green_heart:

GSoC 2019

Now let’s get ready for next year to keep helping new passionate students becoming part of openSUSE! Google has already announced Google Summer of Code 2019 and openSUSE is looking for mentors and organization admins who would like to help bringing new programmers to our community. We need at least one more organization admin and several openSUSE related projects to be able to participate. The application period for organizations is open from January 15 to February 6, so if you would like to participate as an organization admin please get in touch with Hernán, Christian or me by January 20. For mentors, the deadline to create an issue with your project(s) in the mentoring page is January 31. If you want more information about the program and what openSUSE has been doing, check out last blog posts, our mentoring page, Google’s Mentor Guide and the following video:

See you next year! :wave:

About me

My name is Ana and I am a mentor and an organization admin for openSUSE in GSoC, openSUSE Board member, Open Build Service Frontend Engineer at SUSE and open source contributor in projects inside and outside openSUSE.

Last week I attended the reproducible builds world summit in Paris.
It was very well organized by Holger, Gunner and their hidden helpers in the background. Very similar to the last 2 summits I attended in Berlin.

Because we were around 50 participants, introductions and announcements were the only things done in the big group. All actual work happened in 5-10 smaller circles.

We had participants from large companies like Google (with bazel), MicroSoft and Huawei, but also from many distributions and open source projects. Even MirageOS as non-Linux OS.

We did knowledge-sharing, refine definitions of terms, evolve concepts like “rebuilders” for verifying builds and allow users to better trust software they install, and such.

I learned about the undocumented DB dump (153 MB) and DB schema

And we had some hacking time, too, so there is now
a jenkins job that renders the list of unreproducible openSUSE Factory packages.

Also, my maintainer tool now has added support for the Alpine Linux distribution, thanks to help by one of its maintainers: Natanael Copa.
This is meant to help all cross-distro collaboration, not just for reproducible builds.

There is still work to be done to make better use of Mitre CPE to map package names across distributions.

I think, one major benefit of the summit was all the networking and talking going on, so that we have an easier time working with each other over the internet in the future.

As a follow-up to my RPM blog last week, here are instructions installing syslog-ng Open Source Edition (syslog-ng OSE) on the Debian / Ubuntu version. If you read my previous blog, skip to the installation part at the end, otherwise: read on.

The syslog-ng application is part of all major Linux distributions, and you can usually install syslog-ng from the official repositories. If you use just the core functionality of syslog-ng, use the package in your distribution repository (apt-get install syslog-ng), and you can stop reading here. However, if you want to use the features of newer syslog-ng versions (for example, send log messages to Elasticsearch or Apache Kafka), you have to either compile the syslog-ng from source, or install it from unofficial repositories. This post explains you how to do that.

For information on all platforms that could be relevant to you, check out all my blog posts about installing syslog-ng on major Linux distributions, collected in one place.

In addition, syslog-ng is also available as a Docker image. To learn more, read our tutorial about logging in Docker using syslog-ng.

Why is syslog-ng in my distribution so old?

Most Linux distributions have a number of limitations. Of course these are not limitations in the traditional sense, rather ways of quality control.

• Distribution releases are done on a schedule: after a release candidate is out, software in the distribution cannot be upgraded. This ensures that a known state of the distribution can be tested and polished, and external applications are installed on a stable base. But it also means that distributions include an older version of syslog-ng, which lags behind a few minor or major versions.
• The use of bundled libraries is often prohibited. Some functionality of syslog-ng is only available in bundled libraries, either because it requires a modified version, or requires a version that is not yet available in distributions.
• Distributions may lack certain depencencies (tools, sources) that are required to enable certain features in syslog-ng. This makes compiling Java-based destinations nearly impossible, as most tools and dependencies are missing, or have a different version than required by syslog-ng.

All of this means that syslog-ng in distributions is locked to a given version with a limited feature set from half a year to up to half a decade, depending on the release cycle. Thus, the included syslog-ng version can be five years old.

If you need a feature or fix not available for some reason in the distribution package, you can either compile syslog-ng for yourself or use one of the unofficial syslog-ng DEB repositories. Using the repositories is usually easier 

Where to find new DEB packages of syslog-ng?

We, the developers of syslog-ng maintain several unofficial repositories for different distributions. The natural question is: why are these called “unofficial”? The short answer is: these packages are not officially supported by Balabit or a Linux distribution. If you need tested binaries, commercial support with guaranteed response times and other goodies, you either need a commercial Linux distribution, which includes syslog-ng (see possible problems above), or the commercial syslog-ng Premium Edition developed by Balabit. We support the unofficial repositories on a best effort level, which is sometimes quicker than commercial support, but most often is not

Looking for RPM packages? Check my previous blog covering RPM packages.

Which package to install?

You can use many sources and destinations in syslog-ng. The majority of these require additional dependencies to be installed. If all of the features would be packaged into a single package, installing syslog-ng would also install dozens of other smaller and larger dependencies, including such behemoths as Java. This is why the syslog-ng-core package includes only the core functionality, whereas features requiring additional dependencies are available as sub-packages. The most popular sub-package is syslog-ng-mod-elastic, which installs the Java-based Elasticsearch destination driver, but there are many others as well. The command “apt-cache search syslog-ng” will list you all the possibilities.

Install syslog-ng on Ubuntu or Debian

At the time of writing, the latest syslog-ng version is 3.19 and Debian and Ubuntu packages are available on the Open Build Service at https://build.opensuse.org/project/show/home:laszlo_budai:syslog-ng . The installation steps below are for Ubuntu 17.04, but you can use them with minimal modifications in any other supported distributions, just change the URLs. You can see the list of supported distributions at the upper right side of the page, when you open the above URL.

1. Download and install the release key:

wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_17.04/Release.key | sudo apt-key add -

2. Add the repository containing the latest unofficial build of syslog-ng to the APT sources. Under the /etc/apt/sources.list.d/ directory create a new file, for example syslog-ng-obs.list. Add the following line to this file:

deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_17.04 ./

Run the following command:

apt-get update

3. Install syslog-ng and any of its subpackages:

apt-get install syslog-ng-core