Skip to main content

the avatar of Federico Mena-Quintero

Moving to a new blog engine

In 2003 I wrote an Emacs script to write my blog and produce an RSS feed. Back then, I seemed to write multiple short blog entries in a day rather than longer articles (doing Mastodon before it was cool?). But my blogging patterns have changed. I've been wanting to add some more features to the script: moving to a page-per-post model, support for draft articles, tags, and syntax highlighting for code excerpts...

This is a wheel that I do not find worth reinventing these days. After asking on Mastodon about static site generators (thanks to everyone who replied!), I've decided to give Pelican a try. I've reached the age where "obvious, beautiful documentation" is high on my list of things to look for when shopping for tools, and Pelican's docs are nice from the start.

The old blog is still available in the old location.

If you find broken links, or stuff that doesn't work correctly here, please mail me!

the avatar of Efstathios Iosifidis

Install Nextcloud client for openSUSE, Arch Linux, Fedora, Ubuntu based, Debian, Android, iOS

You have your Nextcloud instance installed. But how can you sync files from your computer to your server? You need a client on your desktop-laptop. We will see how your can install desktop client for openSUSE, Arch Linux, Fedora, Ubuntu based distros and of course you can see the links to mobile clients.

ANDROID/iOS
Let's start with the easy ones, the mobile clients.
Download for Android or for iOS (iOS costs $0.99).

GNOME
You don't need to install anything if you have GNOME version 3.24. You can go to SYSTEM SETTINGS>ONLINE ACCOUNTS and you'll see Nextcloud option.


You add your server and username/password.


And you can see a bookmark on Nautilus (Files). This option doesn't save anything on your local disk. It uploads the files to your instance. So it might be a little slow (depends on your bandwith).

If you have older version of GNOME, you can use ownCloud option. It works fine with your Nextcloud instance.

openSUSE
openSUSE has nextcloud client in the repositories.


If you cannot find the files, you can search for the packages here and use 1 click install.

Here is how you can install the client

# If you have GNOME
zypper in nautilus-extension-nextcloud nextcloud-client

# If you have MATE
zypper in caja-extension-nextcloud nextcloud-client

# If you have Cinnamon
zypper in nemo-extension-nextcloud nextcloud-client

# If you have KDE
zypper in nextcloud-client-dolphin nextcloud-client

Arch Linux
The client package is in AUR repository. You can read Arch Wiki for more information.

yaourt -S nextcloud-client

Fedora
The client is available for all versions and architectures. You can find more about the package here:

https://apps.fedoraproject.org/packages/nextcloud-client/

You can install it using the command

dnf install nextcloud-client

or download the rpm and install it.

Ubuntu based distros
The client is available in Launchpad.

To install it, open your terminal and use the following commands:

sudo -s
add-apt-repository ppa:nextcloud-devs/client
apt update
apt install nextcloud-client

Debian
You need to add to sources.list (nano /etc/apt/sources.list) one of the source lines below corresponding to your Debian version:

deb http://download.opensuse.org/repositories/home:/ivaradi/Debian_9.0_update/ /
deb http://download.opensuse.org/repositories/home:/ivaradi/Debian_9.0/ /
deb http://download.opensuse.org/repositories/home:/ivaradi/Debian_8.0/ /
deb http://download.opensuse.org/repositories/home:/ivaradi/Debian_7.0/ /

Before installing, you also need to add the respository's key to the list of trusted APT keys with a command line:

wget -q -O - /Release.key | apt-key add -y

For example (as root):

echo 'deb http://download.opensuse.org/repositories/home:/ivaradi/Debian_9.0/ /' > /etc/apt/sources.list.d/nextcloud-client.list
wget -q -O - http://download.opensuse.org/repositories/home:/ivaradi/Debian_9.0/Release.key | apt-key add -
apt-get update
apt-get install nextcloud-client

SETUP THE CLIENT
Next step is to configure the client. It's very easy.

First of all enter the URL for your instance.

Then enter username and password.

Then configure what to be synced. Press connect.

And when everything is OK press Finish.

If you want to check the client properties, click on the cloud icon. There, you can add a second account, maybe from another Nextcloud instance.

the avatar of Bernhard M. Wiedemann

The issues with contributing to projects only once

I work to improve the openSUSE Tumbleweed (GNU/)Linux distribution. Specifically I make sure that all packages can be built twice on different hosts and still produce identical results, which has multiple benefits. This generates a lot of patches in a single week.

OBS
Sometimes it is enough to adjust the .spec file – that is a small text file usually specific to us. Then it is straight-forward

  1. osc bco
  2. cd $PROJECT/$PACKAGE
  3. optional: spec_add_patch $MY.patch $SOME.spec
  4. edit *.spec
  5. osc build
  6. osc vc
  7. osc ci
  8. osc sr

And OBS will even auto-clean the branch when the submit-request is accepted. And it has a ‘tasks’ page to see and track SRs in various stages. For the spec_add_patch to work, you need to do once
ln -s /usr/lib/build/spec_add_patch /usr/local/bin/

When you want to contribute patches upstream, so that other distributions benefit from your improvements as well, then you first need to find out, where they collaborate. A good starting point is the URL field in the .spec file, but a google search for ‘contribute $PROJECT’ often is better.

github
Then there are those many projects hosted on github, where it is also pretty low effort, because I already have the account and it even remains signed in. But some repos on github are only read-only mirrors.

  1. check pull requests, if some have been merged recently
  2. fork the project
  3. git clone git@github.com:…
  4. cd $REPO
  5. edit $FILES
  6. git commit -a
  7. git push
  8. open pull request
  9. maybe have to sign a CLA for the project
  10. When change is accepted, delete fork to not clutter up repository list too much (on github under settings)

sourceforge
The older brother of github. They integrate various ways of contributing. The easiest one is to open a Ticket (Patch or Bug) and attach the .patch you want them to merge with a good description. While many developers do not have the time and energy to debug every bug you file, applying patches is much easier, so gets your issue fixed with a higher chance.

devel Mailinglist
Some projects collaborate mainly through their development MLs, then I need to

  1. subscribe
  2. confirm the subscription
  3. git format-patch origin/master
  4. git send-email –to $FOO-devel@… –from $MYSUBSCRIBEDEMAIL 000*.patch
  5. wait for replies
  6. if it is a high-volume ML, also add an IMAP folder and an entry to .procmailrc
  7. unsubscribe
  8. confirm

project bugtracker
Like https://bugzilla.gnome.org/ https://bugs.python.org/ https://bugs.ruby-lang.org/ https://bz.apache.org/bugzilla/

  1. create unique email addr
  2. sign up for account
  3. add info to my account list
  4. optional: search for existing bug (90% of the time there is none)
  5. file bug
  6. attach patch

So as you can see there is a wide range of ways. And most of them have some initial effort that you would only have once… But then I only contribute once per project, so I always pay that.

Thus, please make it easy for people to contribute one simple fix.

the avatar of Frank Karlitschek

Is doing nothing evil?

Last weekend I attended the openSUSE conference in Nürnberg. This is a really nice conference. Awesome location, great people, and an overall very relaxed atmosphere. I gave a talk about Nextcloud Security and what we plan to do in the future to make hosting a Nextcloud instance even easier and more secure.

I attended the Saturday keynote which triggered a reaction on my side that I wanted to share. This is only my personal opinion and I’m sure a lot of people think differently. But I can’t help my self.

The talk was about management of infrastructure and automation. It was a really good talk from a technical perspective. Very informative and detailed. But in the middle of the talk, the presenter mentioned that he was involved in building autonomous military submarines. This is of course controversial. I personally wouldn’t get involved in actual weapon development, building things which sole purpose is to kill people. But I understand that people have different opinions here and I can live with such a disagreement.

However, a bit later the presenter mentioned that he also worked for the US intelligence community to build surveillance systems to spy on people on a mass scale. Global, mass scale surveillance, which obviously involves all the people in the room. Which he pointed out as a some kind of joke, noting he might have helped spy on the people in the room.

I’m sorry but I don’t think this is funny at all. The global surveillance systems are undemocratic, in a lot of cases illegal and an attack on the basic human rights of people.

I understand that playing and working with cool technology is fun. And there is a lot of opportunity to do this for secret services and for the military to earn money. But I think we as software developers have a responsibility here. We are building the technology of the future. So we as developers are partly in control of how the world looks like in a few years.

We can’t just say: I close my eyes because this is only a job. Or I don’t want to know how this technology is used. I didn’t ask and no one told me so I’m innocent and not involved. Let me quote a well known saying here: “All that is necessary for the triumph of evil is that good men do nothing.”

I really have a hard time accepting that some people think that building mass surveillance systems is somehow funny or cool. And it is even more troubling to tell this the people your helped put under surveillance into their face and think that this is fun.

Sorry for the rant. But technology matter. Developers matter. Software matters and can be used in good ways and bad ways. We as developers and free software community have a responsibility and should not close our eyes.

a silhouette of a person's head and shoulders, used as a default avatar

LMBench versus GCC Optimisations

A quality benchmark is authoritative and trustworthy, and when you’re using one it’s a bit like playing the card game Snap: the rules are easy, and when the game is over it’s obvious who won.

But a poor benchmark makes performance work more like trying to solve a twisted version of the Knights and Knaves riddle where you’re not sure if the answers you’re getting are truths or lies, no one ever wins, and you only stop playing because you’re exhausted.

LMBench definitely has that riddle vibe.

I just don’t trust the test results that it spits out because I’ve run into too many dead ends when investigating performance issues that turned out to be false positives. And if there’s one thing that you need to be sure of when measuring performance, it’s the accuracy of your results.

So I was less than convinced when I recently saw that the int64-mul subtest of the LMBench ops microbenchmark was taking between 10% and 20% longer to run with a new enterprise kernel.

With my suspicions suitably heightened, I started reading the source code to understand exactly what the test was doing.

The int64-mul subtest tests the CPU speed of 64-bit integer multiplication. Here’s an edited version:

Seeing the register keyword always sets alarm bells ringing for me. Not because it has no purpose – you can use it to disallow using the unary address-of operator on a variable, which lets the compiler optimise accesses to that variable – but because it usually indicates that the benchmark has been written with a specific compiler implementation, or version, in mind. LMBench was released in 1996 which would have made GCC 2.7 the current version.

Using the register keyword may have helped old compilers optimise access to variables by allocating registers for them, but modern compilers ignore register when making register allocation decisions.

Before doing anything else, I wanted to verify that the compiler was emitting those 64-bit multiplication operations on lines 17-21 above.

00000000004004cb <do_int64_mul>:
  4004cb:       89 f2                   mov    %esi,%edx
  4004cd:       8d 46 06                lea    0x6(%rsi),%eax
  4004d0:       48 c1 e0 20             shl    $0x20,%rax
  4004d4:       48 8d 84 02 2c 92 00    lea    0x922c(%rdx,%rax,1),%rax
  4004db:       00 
  4004dc:       83 ef 01                sub    $0x1,%edi
  4004df:       83 ff ff                cmp    $0xffffffff,%edi
  4004e2:       75 f8                   jne    4004dc <do_int64_mul+0x11>
  4004e4:       89 c7                   mov    %eax,%edi
  4004e6:       e8 cb ff ff ff          callq  4004b6 <use_int>
  4004eb:       f3 c3                   repz retq 

Nope. There’s a complete lack of 64-bit multiplication anywhere in there. As far as the compiler is concerned, the following C code is equivalent to LMBench’s do_int64_mul():

Which makes the test useless because GCC optimised it away.

Why did GCC optimise out the test?

GCC could tell exactly how many times it needed to add all of those 64-bit constants together and used techniques like Constant folding and propagation to calculate the end value at compile time instead of runtime.

While investigating this issue I discovered that GCC didn’t throw away the useless loop on lines 8-9 because LMBench uses the -O switch which doesn’t include the necessary optimisation flag. Here’s the full list of optimisations and which level they are enabled for.

This is the problem with microbenchmarks that assume a specific toolchain version or implementation – upgrading the toolchain can break them without you realising. Instead of writing the inner loops in C (the authors wanted it to be portable), inline assembly would have prevented the compiler from eliminating them.

Tests like int64-mul are so low-level that I’ve heard them referred to as nanobenchmarks; they are notoriously easy to misuse and misunderstand. Here’s Aleksy Shipilëv, infamous JVM performance expert, showing how to use them with JMH, a benchmark harness:

Is it time to retire LMBench?

As much as I distrust LMBench, I actually plan to keep using it. Why? Because it has some other subtests that are useful, like the fork() microbenchmark test, which detected the overhead of the vm_ops->map_pages() API when it was introduced.

But the CPU ops subtest? No, that nanobenchmark definitely needs to go in the trash.

a silhouette of a person's head and shoulders, used as a default avatar

Manual encryption partition setup for stronger full disk encryption

When installing openSUSE or SUSE Linux Enterprise, YaST is able to configure encrypted LVM using LUKS for full disk encryption. The default configuration is aes-xts-plain64 using a 256 bit master key. However, due to how the XTC mode splits the key into two halves, this reduces the effective key size used for AES to 128 Bits.

In order to use a 512 bit key for 256 effective AES, one needs to perform manual formatting prior to installation:
cryptsetup LuksFormat --key-size 512 /dev/sda1
However the installer suffers from boo#1030299 which prevents it from writing an entry to /etc/crypttab in this instance. This results in a system that is unable to boot after installation.

The work-around is as follows: Boot into the rescue system, open the crypto device and enter the installed system as a chroot:

cryptsetup luksOpen /dev/sda1 crypto
mount /dev/mapper/system-root /mnt
for X in proc dev sys; do mount -bind /$ /mnt/$X; done
chroot /mnt

(This example assumes /dev/sda1 to be the crypto device, and an LVM VG named system with a LV named root, and no separate /boot.)

Then in the chroot, edit /etc/crypttab to have the following line:

crypto /dev/sda1 none none

See man crypttab for additional settings and options. To finalize, regenerate the initrd and reboot

mkinitrd
exit
reboot

A future rewrite of the YaST storage abstraction layer is planned which should address this issue.

the avatar of Klaas Freitag

SMB on openSUSE Conference

The annual openSUSE Conference 2017 is upcoming! osc17finalNext weekend it will be again in the Z-Bau in Nuremberg, Germany.

The conference program is impressive and if you can make it, you should consider stopping by.

Stefan Schäfer from the Invis server project and me will organize a workshop about openSUSE for Small and Medium Business (SMB).

SMB is a long running concern of the heart of the two of us: Both Stefan, who even does it for living, and me have both used openSUSE in the area of SMB for long and we know how well it serves there. Stefan has even initiated the Invis Server Project, which is completely free software and builds on top of the openSUSE distributions. The Invis Server adds a whole bunch of extra functionality to openSUSE that is extremely useful in the special SMB usecase. It came a long way starting as Stefans own project long years ago, evolving as proper maintained openSUSE Spin in OBS with a small, but active community.

The interesting question is how openSUSE, Invis Server and other smaller projects like for example Kraft can unite and offer a reliable maintained and comprehensive solution for this huge group of potential users, that is now locked in to proprietary technologies mainly while FOSS can really make a difference here.

In the workshop we first will introduce the existing projects briefly, maybe discuss some technical questions like integration of new packages in the openSUSE distributions and such, and also touch organizational question like how we want to setup and market openSUSE SMB.

Participants in the workshop should not expect too much presentation. We rather hope for a lively discussion with many people bringing in their projects that might fit, their experiences and ideas. Don’t be shy :-)

a silhouette of a person's head and shoulders, used as a default avatar

ICC Examin 1.0 on Android

ICC Examin allows since version 1.0 ICC Color Profile viewing on the Android mobile platform. ICC Examin shows ICC color profile elements graphically. This way it is much easier to understand the content. Color primaries, white point, curves, tables and color lists are displayed both numerically and as graphics. Matrices, international texts, Metadata are much easier to read.

Features:
* most profile elements from ICC specification version 2 and version 4
* additionally some widely used non standard tag are understood

ICC color profiles are used in photography, print and various operating systems for improving the visual appearance. A ICC profile describes the color response of a color device. Read more about ISO 15076-1:2010 Standard / Specification ICC.1:2010-12 (Profile version 4.3.0.0), color profiles and ICC color management under www.color.org .

The ICC Examin App is completely rewritten in Qt/QML. QML is a declarative language, making it easy to define GUI elements and write layouts with fewer code. In recent years the Qt project extended support from desktop platforms to mobiles like Nokias Meego, Sailfish OS, iOS, Android, embedded devices and more. ICC Examin is available as a paid app in the Google Play Store. Sources are currently closed in order to financially support further development. This ICC Examin version continues to use Oyranos CMS. New is the dependency to RefIccMAX for parsing ICC Profile binaries. In the process both the RefIccMAX library and the Oyranos Color Management System obtained changes and fixes in git for cross compilation with Android libraries. Those changes will be in the next respective releases.

The FLTK Toolkit, as used in previous versions, was not ported to the Android or other mobile platforms. Thus a complete rewrite was unavoidable. The old FLTK based version is still maintained by the same author.

a silhouette of a person's head and shoulders, used as a default avatar

CEPHALOCON 2017 - CANCELLED

I already submitted to talks for the Cephalocon 2017 in Boston end of August this year but unfortunately - for the community - the conference was cancelled on Friday this week.

Here a quote from the cancellation statement:
The Ceph community has always been based on the principles of openness, inclusion, and a level playing field for all participants. We have worked hard to ensure that no matter what time zone you’re in, who you are, or how technically knowledgeable you are there is always a way for you to participate and contribute to the Ceph community.

It is our sincere hope that the people of the United States will course-correct their leadership to conform to the principles that the country was founded on, and has served so well throughout history. In the meantime, we wont be hosting any events that require members of our community to travel to the U.S. and potentially suffer harassment, digital privacy violations, or rejection at the border.
For the full statement please check this page

Though these are `SAD` news for the community, I fully support the decision and the statement. I'm looking forward for a new date for a Cephalocon, propably in Canada, Mexico or around the next OpenStack Summit in Sydney end of this year. It would be really good if the full Ceph community - including the developers - could get together at least once in a while.

a silhouette of a person's head and shoulders, used as a default avatar

Tracing IDispatch::Invoke calls in COM applications

At Collabora Productivity we recently encountered the need to investigate calls in a third-party application to COM services offered by one or more other applications. In particular, calls through the IDispatch mechanism.

In practice, it is use of the services that Microsoft Office offers to third-party applications that we want to trace and dump symbolically.

We looked around for existing tools but did not find anything immediately suitable, especially not anything available under an Open Source license. So we decided to hack a bit on one of the closest matches we found, which is Deviare-InProc. It is on GitHub, https://github.com/nektra/Deviare-InProc.

Deviare-InProc already includes code for much of the hardest things needed, like injecting a DLL into a target process, and hooking function calls. What we needed to do was to hook COM object creation calls and have the hook functions notice when objects that implement IDispatch are created, and then hook their Invoke implementations.

The DLL injection functionality is actually "just" part of the sample code included with Deviare-InProc. The COM tracing functionality that we wrote is based on the sample DLL to be injected.

One problem we encountered was that in some cases, we would need to trace IDispatch::Invoke calls that are made in a process that has already been started (through some unclear mechanism out of our control). The InjectDLL functionality in Deviare-InProc does have the functionality to inject the DLL into an existing process. But in that case, the process might already have performed its creation of IDispatch implementing COM objects, so it is too late to get anything useful from hooking CoGetClassObject().

We solved that with a hack that works nicely in many cases, by having the injected DLL itself create an object known to implement IDispatch, and hoping its Invoke implementation is the same as that used by the interesting things we want to trace.

Here is a snippet of a sample VBScript file:

     Set objExcel = CreateObject("Excel.application")
     set objExcelBook = objExcel.Workbooks.Open(FullName)

     objExcel.application.visible=false
     objExcel.application.displayalerts=false
         
     objExcelBook.SaveAs replace(FileName, actualFileName, prefix & actualFileName) & "csv", 23

     objExcel.Application.Quit
     objExcel.Quit

And here is the corresponding output from tracing cscript executing that file. (In an actual use case, no VBScript source would obviously be available to inspect directly.)

Process #10104 successfully launched with dll injected!
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.

# CoGetClassObject({00024500-0000-0000-C000-000000000046}) (Excel.Application.15)
#   riid={00000001-0000-0000-C000-000000000046}
#   CoCreateInstance({0000032A-0000-0000-C000-000000000046}) (unknown)
#     riid={00000149-0000-0000-C000-000000000046}
#     result:95c668
#   CoCreateInstance({00000339-0000-0000-C000-000000000046}) (unknown)
#     riid={00000003-0000-0000-C000-000000000046}
#     result:98aad8
#   result:95dd8c
# Hooked Invoke 0 of 95de1c (old: 487001d) (orig: 76bafec0)
95de1c:Workbooks() -> IDispatch:98ed74
98ed74:Open({"c:\temp\b1.xls"}) : ({"c:\temp\b1.xls"}) -> IDispatch:98ea14
95de1c:Application() -> IDispatch:95de1c
95de1c:putVisible(FALSE)
95de1c:Application() -> IDispatch:95de1c
95de1c:putDisplayAlerts(FALSE)
98ea14:SaveAs(23,"c:\temp\converted_b1.csv")
95de1c:Application() -> IDispatch:95de1c
95de1c:Quit()
95de1c:Quit()

Our work on top of Deviare-InProc is available at https://github.com/CollaboraOnline/Deviare-InProc.

Binaries are available at https://people.collabora.com/~tml/injectdll/injectdll.zip (for 32-bit applications) and https://people.collabora.com/~tml/injectdll/injectdll64.zip (64-bit). The zip archive contains an executable, injectdll.exe (injectdll64.exe in the 64-bit case) and a DLL.

Unpack the zip archive somewhere. Then go there in Command Prompt, and in case the program you want to trace the IDispatch::Invoke use of is something you know how to start from the command line, you can enter this command:

injectdll.exe x:\path\to\program.exe “program arg1 arg2 …”

where program.exe is the executable you want to run, and arg1 arg2 … are command-line parameters it takes, if any.

If program.exe is a 64-bit program, instead download injectdll64.zip, and run injectdll64.exe, otherwise similar.

if you can’t start the program you want to investigate from the command line, but you need to inspect it after it has already started, just pass only the process id of the program to injectdll.exe instead. (Or injectdll64.exe) This is somewhat less likely to succeed, depending on how the program uses IDispatch.

In any case, the output (symbolic trace) will go to the standard output of the program being traced, which typically is nowhere at all, and not useful. It will not go to the standard output of the injectdll.exe program.

In order to redirect the output to a file, set an environment variable DEVIARE_LOGFILE that contains the full pathname to the log file to produce. This environment variable must be visible in the program that is being traced; it is not enough to set it in the Command Prompt window where you run injectdll.exe.

Obviously all this is a work in progress, and as needed will be hacked on further. For instance, the name "injectdll" is just the name of the original sample program in upstream Deviare-InProc; we should really rename it to something specific for this use case.