Some weeks ago, someone asked on the opensuse-wiki mailinglist if it's acceptable to move documentation (in this case about Icecream) from the openSUSE wiki to the upstream repo on github. One of the arguments was:

I think I should store anything I do for the openSUSE project somewhere in the openSUSE.org domain, not in a RedHat.org or Canonical.org domain or a SourceForge.net or GitHub.com domain.

While this sounds like a valid argument and for sure shows good intentions, I wrote a longish reply:

You are overlooking an important point here - collaboration.

It doesn't make sense to think of "we" vs. "them" when it comes to other distributions or upstream projects. It's quite the opposite - everybody can save time by working together with other distributions, upstream projects etc. We have more important things to do than re-inventing the wheel just because we need a green one.

As an example: You might know that I maintain AppArmor in openSUSE and also contribute upstream (OMG, the upstream mailinglist is @lists.ubuntu.com, not at a "neutral" domain!)

Some not-so-known details:

• I implemented support for new AppArmor rule types (dbus, signal etc.) in aa-logprof, but those are not yet supported in the upstream kernel (and also not in openSUSE) - so currently only Ubuntu users benefit from that
• I always send patches upstream so that everybody can benefit (no, saying "use openSUSE, it's fixed there" is not a good idea ;-)
• In 2015, I visited DebConf (I'd guess I was the only one there who had never used Debian before) and even gave a talk.
• I closely follow AppArmor-related bugreports in Debian and Ubuntu, and help them to get things fixed - even if it's distro-specific

So, tell me - am I working for the enemy? ;-)

 

BTW: This isn't a one way road. Quite some AppArmor contributions done by Ubuntu (some other upstream developers work for Canonical) and Debian contributors end up in openSUSE ;-)

Needless to say that AppArmor is just an example. What I said is basically valid for every package, project, whatever. Either you collaborate (and everybody wins), or you "cook your own soup" and never find out that someone else has a receipe for a much more tasty soup ;-)

Since I talked a lot about AppArmor in the above text, let's see what's new there.

You might have noticed that there were some AppArmor releases recently:

• AppArmor 2.9.4 (with several bugfixes and profile updates) was already released as an update for openSUSE 13.2 shortly before it went out of support.
• AppArmor 2.10.2 (also with several bugfixes and profile updates) is already available in Tumbleweed, and updates for Leap 42.1 and 42.2 are under review
• AppArmor 2.11 (with lots of improvements and new features - for example, I rewrote the handling of file rules in aa-logprof etc.) is on its way to Tumbleweed (SR 453537), but it seems splitting off libapparmor into its own spec file (to fix a build loop) triggered a bug in the factory-auto review bot. Given my talent to find bugs, I'm not even surprised ;-)

The rewrite of the file rule handling resulted in a nice series of 42 patches which replace 1600 lines of code using a deeply nested array with 1200 lines with the more readable and easier maintainable FileRule and FileRuleset classes (a total of 530 lines) and functions using these classes. Even with 400 lines less code, I added some small features (for example, rules with leading permissions like "r /etc/fstab," are now supported) and fixed some bugs along the way.

The old code to handle file rules had very few unittests, which made this rewrite (and especially avoiding breakage and regressions) quite challenging. On the positive side, my patch series added full test coverage for the FileRule and FileRuleset classes, and also added unittests for most of the functions using FileRule and FileRuleset. (Unfortunately full test coverage isn't always easy, especially for the interactive parts of aa-logprof.) Those unittests add about 1400 lines of code, but as long as such additions happen in the tests directory, I'm more than happy about them ;-)

Oh, and the final challenge hit the other AppArmor developers. AppArmor has the policy that all patches have to be reviewed, and reviewing the whole patch series (which summed up to +2600 -1628 lines) took some time ;-)

That all said, let's not forget to answer where the documentation should live:

To come back to the origin of this discussion: I don't care too much where the Icecream developers host their documentation as long as

• it is complete and up to date (having it at the developers' favorite place makes this more likely)
• it can be easily found (also not a problem, it's linked from the wiki, and your favorite search engine will also find it)

I see the main purpose of the openSUSE wiki to provide openSUSE-specific information.

Information about upstream projects (even if a project is done by openSUSE) is "nice to have", but it's also ok if it lives upstream. It's better have one good upstream documentation than pages at 5 distro wikis that are all incomplete and out of date ;-)

BTW: The question "Am I working for the enemy?" was mostly meant as a rhetoric question - but if you want to answer nevertheless, please add a comment ;-)

Когда-то, в ноябре 2014 года, разработчики «Дубльгиса» выпустили новенькую бета-версию своего справочника для Linux. Там был чистый интерфейс на Qt5 и QML, плавная работа, возможность скачать карту любого города России и некоторых других стран… Красота! Однако же, дальше беты дело не пошло, и вскоре сайт, посвящённый новой версии 2GIS, закрылся, а разработка перспективного справочника в формате ПК-версии была прекращена. Но репозиторий со сборками дубльгиса для Ubuntu продолжал жить, и он работает до сих пор. В openSUSE имеется несколько частных репозиториев с rpm-пакетами 2GIS. В подобном пакете, на самом деле, содержится не сама программа, а скрипт, который вытягивает deb-пакет из сети и перепаковывает его чем-то вроде alien. В openSUSE 13.2 и 42.1 такой трюк работал без сучка и задоринки, но уже в 42.2 программа перестала запускаться:

Очевидно, что приложение, собранное под старую версию Ubuntu 14.04, уже не может запускаться в более новых версиях Linux. Забавно, что неосиляторы с Гиктаймса в подобной ситуации сдались ещё раньше: на этапе установки пакета. Однако я не первый раз запускаю проприетарный софт в Linux и знаю, что большинство подобных программ (XnRetro, Dropbox, Skype и т.п.) поставляются с собственным набором некоторых системных библиотек. Всегда можно попробовать удалить одну или несколько таких библиотек и посмотреть как программа попытается использовать общесистемные. Короче говоря, если избавиться от файла /usr/lib/2GIS/v4/lib/libpthread.so.0, то Дубльгис прекрасно запускается и работает:

The Lego Mindstorms EV3 robots can be controlled with an Android app (Lego Mindstorms Commander) communicating with the brick via Bluetooth. The command protocol is documented by Lego in the EV3 Communication Developer Kit and the commands themselves in the EV3 Firmware Developer Kit (get them from Mindstorms Downloads).

I wondered what exactly goes on and I decided to capture the communication and decode it, to learn both about Bluetooth and about the details of the EV3 protocol.

I succeeded and made a couple of useful tools along the way:

• extract the RFCOMM data from a btsnoop HCI log
• decode the EV3 command protocol

See also the previous post about sending data (EV3 commands) over USB.

Outline

1. Enable Android Bluetooth logging
2. Run the Commander app and exercise the robot a bit
3. Transfer the log to a PC
4. Extract the serial data (RFCOMM) from the Bluetooth dump
5. Decode the EV3 protocol
6. Disassemble the EV3 instructions

1. Enable Android Bluetooth Logging

• Open Settings
• In the System section, choose Developers (this needs to be enabled first by tapping Build number 7 times)
• Enable Bluetooth HCI Log

2. Run the Commander app and exercise the robot a bit

3. Transfer the log to a PC

On the phone/tablet:

• Open Settings
• System > Developers
• Disable Bluetooth HCI Log

Connect to the PC with a USB cable.

My older Android phone offered to mount its storage as a USB disk drive, but the newer one no longer has that option, offering MTP instead. I transfered the log file with a KDE tool:

$ kioclient cp 'mtp:/Xperia Z3/Interní úložiště/btsnoop_hci.log' .

4. Extract the serial data (RFCOMM) from the Bluetooth dump

The tool I made for this is btsnoop-decode.rb.

I learned the bare minimum needed about Bluetooth so it is very likely the tool only works for this specific use case.

Originally I opened the btsnoop log with Wireshark and guessed my way through the BT protocol layers. In the end the RFCOMM length field was harder than usual to guess and half of my packets were wrong. So I resorted to finding the appropriate part of the Linux kernel source to find out the format.

5+6. Decode the EV3 protocol and dissassemble the EV3 instructions

The people of the ev3dev project have already produced a disassembler which we will use in the next step. But that one assumes you start with a program file (RBF).

Here we have a log containing not only the usual RBF instructions but also System Commands.

I made an ugly hack of the lmsdisasm tool and arrived at a version that disassembles the log produced by the RFCOMM extractor.

Play time

The above experiments enabled me to put together a little script that can control the robot from a Linux terminal, having it ride around and even speak a custom sound file: lethargic-ministers/lms.py.

As DVB-T(1) is phased out in Germany soon, I got me a new DVB-T2 stick. The MyGica T230 is supported under Linux, and has a quite low price (~20€).

Instead of the expected T230, I received a T230C which has silently replaced the T230. The T230C is – although quite similar to the T230 – currently (Linux 4.10rc2) not supported by the mainline kernel.

Compared to the T230, the T230C uses the same Cypress FX2 CY7C68013A USB bridge chip, a Silabs Si2168-D60 demodulator (new revision) and a new tuner chip, Silabs Si2141-A10 (T230 uses a Si2157).

Beyond the bridge/RF chips, there is an I2C eeprom (FX2 firmware?), two LowPowerSemi adjustable, synchronous DC/DC buck converters (marking LPS A36j1, LP3202?), a 74ACT1G00 NAND gate (marking A00), two 24MHz oscillators and a bunch of passives.

It is pretty easy to install openSUSE Linux on a MacBook as operating system. However there are some pitfalls, which can cause trouble. The article gives some hints about a dual boot setup with OS X 10.10 and at time of writing current openSUSE Tumbleweed 20170104 (oS TW) on a MacBookPro from early 2015. A recent Linux kernel, like in TW, is advisable as it provides better hardware support.

The LiveCD can be downloaded from www.opensuse.org and written with ImageWriter GUI to a USB stick ~1GB. I’ve choose the Live KDE one and it run well on a first test. During boot after the first sound and display light switches on hold Option/alt key and wait for the disk selection icon. Put the USB key with Linux in a USB port and wait until the removable media icon appears and select it for boot. For me all went fine. The internal display, sound, touchpad and keyboard where detected and worked well. After that test. It was a good time to backup all data from the internal flash drive. I wrote a compressed disk image to a stick using the unix dd command. With that image and the live media I was able to recover, in case anything went wrong. It is not easy to satisfy OS X for it’s journaled HFS and the introduced logical volume layout, which comes with a separate repair partition directly after the main OS partition. That combination is pretty fragile, but should not be touched. The rescue partition can be booted with the command key + r pressed. External tools failed for me. So I booted into rescue mode and took the OS X diskutil or it’s Disk Utility GUI counter part. The tool allows to split the disk into several partitions. The EFI and the rescue ones are hidden in the GUI. The newly created additional partitions can be formatted to exfat and later be modified for the Linux installation. One additional HFS partition was created for sharing data between OS X and Linux with the comfortable Unix attributes. The well know exfat used by many bigger USB sticks, is a possible option as well, but needs the exfat-kmp kernel module installed, which is not by default installed due to Microsofts patent license policy for the file system. In order to write to HFS from Linux, any HFS partition must have switched off the journal feature. This can be done inside the OS X Disk Utility GUI, by selecting the data partition and holding the alt key and searching in the menu for the disable journaling entry. After rebooting into the Live media, I clicked on the Install icon on the desktop background and started openSUSE’s Yast tool. Depending on the available space, it might be a good idea to disable the btrfs filesystem snapshot feature, as it can eat up lots of disk space during each update. An other pitfall is the boot stage. Select there secure GrubEFI mode, as Grub needs special handling for the required EFI boot process. That’s it. Finish install and you should be able to reboot into Linux with the alt key.

My MacBook has unfortunedly a defect. It’s Boot Manager is very slow. Erasing and reinstalling OS X did not fix that issue. To circumvent it, I need to reset NVRAM by pressing alt+cmd+r+p at boot start for around 14 second, until the display gets dark, hold alt on the soon comming next boot sound, select the EFI TW disk in Apple Boot Manager and can then fluently go through the boot process. Without that extra step, the keyboard and mouse might not respond in Linux at all, except the power button. Hot reboot from Linux works fine. OS X does a cold reboot and needs the extra sequence.

KDE’s Plasma needs some configuration to run properly on a high resolution display. Otherwise additional monitors can be connected and easily configured with the kscreen SystemSettings module. Hibernate works fine. Currently the notebooks SD slot is ignored and the facetime camera has no ready oS packages. Battery run time can be extended by spartan power consumption (less brightness, less USB devices and pulseaudio -k, check with powertop), but is not too far from OS X anyway.

В номере №205 журнала Linux Format я писал о новом графическом формате FLIF (Free Lossless Image Format), который превосходит PNG и WebP по сжатию данных без потерь. В комплекте с исходным кодом FLIF (т.н. reference implementation) имеются библиотеки кодировщика и декодера, сам кодировщик, а также простейшее средство просмотра — консольная программа viewflif. С таким минимальным набором вполне можно работать, однако формат, по сути, в этом случае выполняет роль архиватора: ни листать, ни редактировать FLIF-файлы без предварительной конвертации нельзя. Правда, есть ещё набор консольных утилит ImageMagick, который с некоторых пор поддерживает формат FLIF. Приличных графических программ просмотра, которые были бы основаны на ImageMagick, в природе нет, к тому же, большинство готовых сборок ImageMagick собраны без поддержки FLIF и не могут его читать. Хотите поддержку — собирайте из исходного кода сами.

На этом фоне Qt FLIF Plugin оказался глотком свежего воздуха. Данная разработка сделана датским программистом Себастьяном Валем (Sebastian Wahl), который ведёт свой блог и увлекается алгоритмами сжатия изображений. Суть Qt FLIF Plugin проста: в вашем распоряжении появляется разделяемая библиотека libflif.so (не путать с одноимённой библиотекой из состава самого кодировщика FLIF!), которая может быть использована любыми Qt-приложениями, поддерживающими QImageIOPlugins, например Gwenview, Kolourpaint и многими другими. Автор плагина также написал свой собственный минималистичный просмотрщик графических файлов, вполне неплохой!

Итак, для сборки нам потребуется слегка подредактировать файл project.pro, добавив туда строку CONFIG += c++14:

Если этого не сделать, то проект просто не соберётся современными версиями GCC. Затем нужно убедиться в том, что заголовочные файлы FLIF лежат в нужном месте — между прочим, их нужно заранее вручную положить в /usr/include/FLIF:

Далее командуем make и ждём несколько секунд. Получившуюся библиотеку нужно проверить на успешную линковку с кодировщиком FLIF (дело в том, что плагин почему-то иногда не линкуется):

Дальше, нужно установить саму библиотеку, скопировать .desktop-файлы и зарегистрировать соответствующий тип файла для share MIME database. Следующие команды я выполнял из директории ~/qt_flif_plugin/configuraton:
sudo cp ../libflif.so /usr/lib64/qt5/plugins/imageformats/
sudo cp qimageioplugins/x-flif.desktop /usr/share/kservices5/qimageioplugins/
sudo cp imagethumbnail-flif.desktop /usr/share/kservices5/
sudo cp x-flif.xml /usr/share/mime/packages/
sudo /usr/bin/update-mime-database /usr/share/mime

Результат будет заметен сразу же. Во-первых, заработает генератор миниатюр в файловом менеджере Dolphin:

Во-вторых, можно будет смотреть FLIF-файлы в Gwenview, стандартной программе просмотра из набора KDE:

На данный момент можно использовать лишь версию Gwenview 16.08 или более старую, так как из-за этого коммита программа, начиная с версии 16.12, содержит другой механизм поддержки сторонних форматов. Но это уже вопрос к автор FLIF-плагина — надеюсь, он обновит свой код когда-нибудь. Меня же очень радует сжатие, которое обеспечивает FLIF. Только посмотрите:

This post it to document how we setup cloud.suse.de which is one of our many internal SUSE OpenStack Cloud deployments for use by R&D.

In 2016-06 we started the deployment with SOC6 on 4 nodes. 1 controller and 3 compute nodes that also served for ceph (distributed storage) with their 2nd HDD. Since the nodes are from 2012 they only have 1gbit network and spinning disks. Thus ceph only delivers ~50 MB/s which is sufficient for many use cases.

We did not deploy that cloud with HA, even though our product supports it. The two main reasons for that are

• that it will use up two or three nodes instead of one for controller services, which is significant if you start out with only 4 (and grow to 6)
• that it increases the complexity of setup, operations and debugging and thus might lead to decreased availability of the cloud service

Then we have a limited supply of vlans even though technically they are just numbers between 1 and 4095, in SUSE we do allocations to be able to switch together networks from further away. So we could not use vlan mode in neutron if we want to allow software defined network (=SDN) (we did not in old.cloud.suse.de and I did not hear complaints, but now I see a lot of people using SDN)
So we went with ovs+vxlan +dvr (open vSwitch + Virtual eXtensible LAN + Distributed Virtual Router) because that allows VMs to remain reachable even when the controller node reboots.
But then I found that they cannot use DNS during that time, because distributed virtual DNS was not yet implemented. And ovs has some annoying bugs are hard to debug and fix. So I built ugly workarounds that mostly hide^Wsolve the problems from our users’ point of view.
For the next cloud deployment, I will try to use linuxbridge+vlan or linuxbridge+vxlan mode.
And the uptime is pretty good. But it could be better with proper monitoring.

Because we needed to redeploy multiple times before we got all the details right and to document the setup, we scripted most of the deployment with qa_crowbarsetup (which is part of our CI) and extra files in https://github.com/SUSE-Cloud/automation/tree/production/scripts/productioncloud. The only part not in there are the passwords.

We use proper SSL certs from our internal SUSE CA.
For that we needed to install that root CA on all involved nodes.

We use kvm, because it is the most advanced and stable of the supported hypervisors. Xen might be a possible 2nd choice. We use two custom kvm patches to fix nested virt on our G3 Opteron CPUs.

Overall we use 3 vlans. One each for admin, public/floating, sdn/storage networks.
We increased the default /24 IP ranges because we needed more IPs in the fixed and public/floating networks

For authentication, we use our internal R&D LDAP server, but since it does not have information about user’s groups, I wrote a perl script to pull that information from the Novell/innerweb LDAP server and export it as json for use by the hybrid_json assignment backend I wrote.

In addition I wrote a cloud-stats.sh to email weekly reports about utilization of the cloud and another script to tell users about which instances they still have, but might have forgotten.

On the cloud user side, we and other people use one or more of

• Salt-cloud
• Nova boot
• salt-ssh
• terraform
• heat

to script instance setup and administration.

Overall we are now hosting 70 instance VMs on 5 compute nodes that together have cost us below 20000€

Wow, what a jargon-filled post title. Basically, we do a lot of our deep learning currently on the AWS EC2 cloud – but to use the GPU there with all the goodies (up to CuDNN that supports modern Theano’s batch normalization) is a surprisingly arduous process which you basically need to do manually, with a lot of trial and error and googling and hacking. This is awful, mind-boggling and I hate that everyone has to go through this. So, to fix this bad situation, I just released a community AMI that:

• …is based on Ubuntu 16.04 LTS (as opposed to 14.04)
• …comes with CUDA + CuDNN drivers and toolkit already set up to work on g2.2xlarge instances
• …has Theano and Keras preinstalled and preconfigured so that you can run the Keras ResNet model on a GPU right away (or anything else you desire)

To get started, just spin up a GPU (g2.2xlarge) instance from community AMI ami-f0bde196 (1604-cuda80-cudnn5110-theano-keras), ssh in as the ubuntu@ user and get going! No hassles. But of course, EC2 charges apply.

Edit (errata): Actually, there’s a bug – sorry about that! Out of the box, the nvidia kernel driver is not loaded properly on boot. I might update the AMI later, for now to fix it manually:

1. Edit /etc/modprobe.d/blacklist.conf (using for example sudo nano) and append the line blacklist nouveau to the end of that file
2. Run sudo update-initramfs -u
3. Reboot. Now, everything should finally work.

This AMI was created like this:

• The stock Ubuntu 16.04 LTS AMI
• NVIDIA driver 367.57 (older drivers do not support CUDA 8.0, while this is the last driver version to support the K520 GRID GPU used in AWS)
• To make the driver setup go through, the trick to install apt-get install linux-image-extra-`uname -r` per
• CUDA 8.0 and CuDNN 8.0 set up from the official though unannounced NVIDIA Debian packages by replaying the nvidia-docker recipes
• bashrc modified to include cuda in the path
• Theano and Keras from latest Git as of writing this blogpost (feel free to git pull and reinstall), and some auxiliary python-related etc. packages
• Theano configured to use GPU and Keras configured to use Theano (and the “th” image dim ordering rather than “tf” – this is currently non-default in Keras!)
• Example Keras deep learning models, even an elephant.jpg! Just run python resnet50.py
• Exercise: Install TensorFlow on the system as well, release your own AMI and post its id in the comments!
• Tip: Use nvidia-docker based containers to package your deep learning software; combine it with docker-machine to easily provision GPU instances in AWS and execute your models as needed. Using this for development is a hassle, though.

Enjoy!

I’m proud to announce the release of GNU Screen v.4.5. This time it’s mostly a bugfix release. We added just one new feature: now it’s possible to specify logfile name by using parameter -L (default name stays screenlog.0). Myself also spent some time to make source code a bit cleaner.

As you probably noticed we were going to release 4.5 until Christmas. Unfortunately, we could not do it because of some internal GNU problems. I apologise for that.

As usual, we merged some community patches from our bug tracing system (small patches also were presented in IRC) and I would like to thank everyone who contribute to Screen and helps us to test development git-version!

For openSUSE users: I updated our devel-package already. It’s soon in factory and, as usual, after openQA routine new package will be available in Tumbleweed.