Say we have a robot with a USB connection and command documentation. The only thing missing is knowing how to send a command over USB. Let's learn the basic concepts needed for that.

Installing the Library

We'll use the pyusb Python library. On openSUSE we install it from the main RPM repository:

sudo zypper install python-usb

On other systems we can use the pip tool:

pip install --user pyusb

Navigating USB Concepts

To send a command, we need an Endpoint. To get to the endpoint we need to descend down the hierarchy of

1. Device
2. Configuration
3. Interface
4. Alternate setting
5. Endpoint

First we import the library.

#!/usr/bin/env python2

import usb.core

The device is identified with a vendor:product pair included in lsusb output.

Bus 002 Device 043: ID 0694:0005 Lego Group

VENDOR_LEGO = 0x0694
PRODUCT_EV3 = 5
device = usb.core.find(idVendor=VENDOR_LEGO, idProduct=PRODUCT_EV3)

A Device may have multiple Configurations, and only one can be active at a time. Most devices have only one. Supporting multiple Configurations is reportedly useful for offering more/less features when more/less power is available. EV3 has only one configuration.

configuration = device.get_active_configuration()

A physical Device may have multiple Interfaces active at a time. A typical example is a scanner-printer combo. An Interface may have multiple Alternate Settings. They are kind of like Configurations, but easier to switch. I don't quite understand this, but they say that if you need Isochronous Endpoints (read: audio or video), you must go to a non-primary Alternate Setting. Anyway, EV3 has only one Interface with one Setting.

INTERFACE_EV3 = 0
SETTING_EV3 = 0
interface = configuration[(INTERFACE_EV3, SETTING_EV3)]

An Interface will typically have multiple Endpoints. The Endpoint 0 is reserved for control functions by the USB standard so we need to use Endpoint 1 here.

The standard distinguishes between input and output endpoints, as well as four transfer types, differing in latency and reliability. The nice thing is that the Python library nicely allows to abstract all that away (unlike cough Ruby cough) and we simply say to write to a non-control Endpoint.

ENDPOINT_EV3 = 1
endpoint = interface[ENDPOINT_EV3]

# make the robot beep
command = '\x0F\x00\x01\x00\x80\x00\x00\x94\x01\x81\x02\x82\xE8\x03\x82\xE8\x03'
endpoint.write(command)

Other than Robots?

Robots are great fun but unfortunately they do not come bundled with every computer. Do you know of a device that we could use for demonstration purposes? Everyone has a USB keyboard and mouse but I guess the OS will claim them for input and not let you play.

What Next

• PyUSB
• PyUSB tutorial
• USB in a nutshell goes deeper, and is aimed more at firmware developers for the devices, but still is much shorter than the 650 page USB 2.0 specification
• EV3 documentation at Mindstorms Downloads

The Full Script

250 Ok: queued as F117E148DE4

poll myacc via shellservername.tld with proto imap:
    plugin "ssh -C %h bin/imapd" auth ssh;
    user seife there is seife here options keep stripcr
    folders Mail/inbox Mail/s3e-spam Mail/thirdfolder
    mda "/usr/bin/procmail -f %F -d %T"

fetchmail && mailsync myacc

:0fw: spamassassin.lock
* < 1048576
| $HOME/perl/bin/spamassassin

kiwi has long been the one standard way of building images in openSUSE, but even though there exist extensive writings on how to use it, for many it is still an arcane thing better left to the Great Magicians.

Thus, I started to use a simpler alternative image building method, named altimagebuild when I built our first working Raspberry Pi images in 2013 and now I re-used that to build x86_64 VM images at
https://build.opensuse.org/package/show/home:bmwiedemann/altimagebuild
after I found out that it even works in OBS, including publishing the result to our mirror infrastructure.
It is still in rpm format because of how it is produced, so you have to use unrpm to get to the image file.

This method uses 3 parts.

• a .spec file that lists packages to be pulled into the image
• a mkrootfs.sh that converts the build system into the future root filesystem you want
• a mkimage.sh that converts the rootfs into a filesystem image

The good thing about it is that you do not need specialized external tools, because everything is hard-coded in the scripts.
And the bad thing about it is that everything is hard-coded in the scripts, so it is hard to share general improvements over a wider range of images.

In the current version, it builds cloud-enabled partitionless images (which is nice for VMs because you can just use resize2fs to get a larger filesystem and if you later want to access your VM’s data from outside, you simply use mount -o loop)
But it can build anything you want.

To make your own build, do osc checkout home:bmwiedemann/altimagebuild && cd $_ && osc build openSUSE_Leap_42.2

So what images would you want to build?

adb shell \ 

  dd if=/dev/zero \

  of=/dev/block/platform/msm_sdcc.1/by-name/misc

It is Christmas time and I have got cookie cutters by openSUSE and ownCloud. What can you create as a happy Working Student at ownCloud and an openSUSE Contributor?

Normally you deploy ownCloud on openSUSE. But do you know the idiom „to be in seventh heaven“ (auf Wolke 7 schweben)?

I want to  show you openSUSE Leap 42.2 on ownCloud 9.

9.1 is the latest release, and 7 not up to date and insecure for the openSUSE chameleon. The second reason is that the chameleon has got a perfect place on the cloud.

You can watch the success in both projects!

I wish you all a merry christmas and a lot of fun with your cookie cutters!

In the past three weeks, the openSUSE community KDE team has been pretty busy to package the latest release of Applications from KDE, 16.12. It was a pretty large task, due to the number of programs involved, and the fact that several monolithic projects were split (in particular KDE PIM). This post goes through what we did, and how we improved our packaging workflow.

Some prerequisites

In openSUSE speak, packages are developed in “projects”, which are separate repositories maintained on the OBS. Projects whose packages will end up in the distro, that is where they are being developed to land in the distribution, are called devel projects. The KDE team uses a number of these to package and test:

• KDE:Qt for currently-released Qt versions
• KDE:Frameworks5 for Frameworks and Plasma packages
• KDE:Applications for the KDE Applications packages
• KDE:Extra for additional software not part of the above categories

The last three have also an Unstable equivalent (KDE:Unstable:XXX) where packages built straight off git master are made, and used in the Argon and Krypton live images.

A new development approach

With the release of Leap 42.2, we also needed a way to keep only Long Term Support packages in a place were could test and adjust fixes for Leap (which, having a frozen base, will not easily accept version upgrades), so we created an additional repository with the LTS suffix to track Plasma 5.8 versions (and KF 5.26, which is what Leap ships).

As you can see, the number of repositories was starting to get large, and we’re still a very low number team, with everyone contributing their spare time to this task. Therefore, a new approach was proposed, prototyped by Hrvoje “shumski” Senjan and spearheaded by Fabian Vogt during the Leap development cycle.

The idea was to use only one repository as an authoritative source of spec files (for those not in the know, spec files are files needed to build RPM packages, and describe the structure of the package, it sources, whether it should be split in sub-packages, and so on), only do changes there and then sync back the changes to the other repositories.

In this case, KDE:Frameworks5 was used as source. All changes were then synced to both the LTS repository and to the Unstable variant with some simple scripting and the use of the osc command, which allows interacting with the OBS from the CLI. This significantly reduced divergences between packages and eased maintenance.

Enter Applications 16.12

When we started packaging Applications, we faced a number of problems that involved the existence of kdelibs 4.x applications that were now obsoleted by KF5 based versions (see Okular, but not only that). Additionally, there were a major number of splits, meaning that we had to track and adjust packaging to keep in mind that what used to be there wasn’t around anymore.

We had already a source that kept track of these changes: the KDE:Unstable:Applications repository, which followed git master. A major problem was that its development had gone in a different direction than the original KDE:Applications, meaning that there was a significant divergence in the two.

Initially we set up a test project and tried to figure out how to lay out a migration path for the existing packages. It didn’t work too well by hand: too many changes, too many packages to keep track of. That is when Raymond Wooninck had the idea to automate the whole process and change the development workflow of Applications packaging.

The new workflow worked as such:

1. The authoritative source of changes is the Unstable repository, because that’s where the changes end up first, before release
2. On beta release time, packages would be copied from Unstable to the stable repository, dropping any patches present that were upstreamed
3. openSUSE specific patches (integration, build system, etc.) would stay in both repositories
4. upstream patches (patches already committed but not part of a release) would only stay in the stable repository

In order to ensure that this would be done automatically, Raymond created a repository and wrote scripts to do both the Unstable->stable transition and to automate packaging of new minor releases. Once we switched to this workflow, adjustments were much easier, and we were able to finish the job, at least: yesterday (as of writing) the new Applications release was checked in Tumbleweed.

This new workflow requires some discipline (to avoid “ad hoc” solutions) but dramatically reduces the maintenance required, and allows us to track changes in the packages “incrementally” as they happen during a release cycle. At the same time, this guarantees that all the openSUSE packaging policies are followed also in the Unstable project (which was more lax, due to the fact that it would never end up in the distro).

Final icing on the cake

The last bit was to ensure timely updates of all the Unstable project hierarchy avoiding to watch git commits like hawks. I took up the challenges and wrote a script, which, coupled with a repository mapping file would cache the latest “seen” (past 24h) git revision of the KDE repositories, and trigger updates only if something changed (using git ls-remote to avoid hitting the KDE servers too hard).

I put this on a cron job which runs every day at 20 UTC+1, meaning that even the updates are now fully automated. Of course I have to check every now and then for added dependencies and build faiures, but the workload is definitely less than before.

Final wrap up

A handful of tools and some quick thinking can make a massive collection of software manageable by a small bunch of people. At the same time, there’s always need for more helping hands! Should you want to help in openSUSE packaging, drop by in the #opensuse-kde IRC channel on Freenode.

I know Ruby but have little experience with web apps. If you're like me then this article could be useful.

I needed a way to browse API documentation of multiple related code repositories. (Yes, it's YaST).

I made a tool for that in the form of a web application. This was really easy with the Sinatra framework.

First I ran it locally on my machine for myself. Then I ran it on a machine in the company network for team mates to use. It was a VM that I repurposed from a previous experiment. Then Pepa said it would be nice to have it publicly accessible. How hard could that be?

I had heard that Heroku makes that sort of thing easy, and it turned to be true!

1. It's free. A low profile app, that only needs to run occasionally, fits into their Free service plan. It sleeps after 30 minutes and takes 10 seconds to wake up.

2. Easy to sign up. Enter your e-mail, pick a password. No other details required.

3. Easy app creation: pick the region (US or EU). Optionally pick a name (I got salty-waters-71436 for my demo app).

4. Easy to set up the tooling. Well, they install the curl | bash way. Over https. And then the downloaded code downloads some more.

   If you want to start small, the setup by hand is easy too, now download required:

   touch ~/.netrc
   chmod 600 ~/.netrc
   echo "machine git.heroku.com login YOUR_EMAIL password ffffffff-ffff-ffff-ffff-ffffffffffff" >> ~/.netrc

   Where the hex string is your API Key (Top-right Person icon > Account Setings > scroll down)

Now let's write a trivial web app.

1. Make a git repo.

2. Make a two-line Sinatra app.

   require "sinatra"
   get "/" { "Hello, world!" }

3. Add a two-line Gemfile declaration; add also Gemfile.lock to Git.

   source "https://rubygems.org"
   gem "sinatra", "~> 1.4.0"

4. Add a oneliner Procfile.

   web: bundle exec ./timeserver
   

   (This was new to me. It's not needed locally but needed for Heroku, and anyway useful once you outgrow oneliners. Use foreman start to use it)

5. Use your app name as the remote repo name. Push to deploy (or set up automatic deployment):

   git remote add heroku https://git.heroku.com/salty-waters-71436.git
   git push heroku

That's it! See the app in action: https://salty-waters-71436.herokuapp.com/?q=1_500_000_000.

To see my actual app, instead of the trivial demo built for this blog post, go to http://apitalism.herokuapp.com/.

Here is something that might be a little outdated already, but I hope it still adds some interesting thoughts. The rainy Sunday afternoon today finally gives the opportunity to write this little blog.

Recently an ownCloud fork was coming up with a little shiny box with one harddisk, that can be complemented with a Rapsberry Pi and their software, promoting that as your private cloud.

While I like the idea of building a private cloud for everybody (I started to work on ownCloud because of that idea back in the days), I do not think that this example of gear is a good solution for private cloud.

In fact I believe that throwing this kind of implementations on the table is especially unfortunate because if we come up with too many not optimal proposals, we waste the  willingness of users to try it. This idea should not target geeks who might be willing to try ideas on and on. The idea of the private cloud needs to target at every computer user who wants to store data safely, but does not want to care about longer than ever necessary. And with them I fear we only have very little chances, if one at all, to introduce them to a private cloud solution before they go back to something that simply works.

Here are some points why I think solutions like the proposed one are not good enough:

Hardware

That is nothing new: The hardware of the Raspberry Pi was not designed for this kind of usecases. It is simply too weak to drive ownCloud, which is an PHP app plus database server that has some requirements on the servers power. Even with PHP7, which is faster, and the latest revisions of the mini computer, it might look ok in the beginning, but after all the neccessary bells and whistles were added to the installation and data run in, it will turn out that the CPU power is simply not enough. Similar weaknesses are also true for the networking capabilities for example.

A user that finds that out after a couple of weeks after she worked with the system will remain angry and probably go (back) to solutions that we do not fancy.

One Disk Setup

The solution comes as one disk setup: How secure can data be that is on one single hardisk? A seriously engineered solution should at least recommend a way to store the data more securely and/or backup, like on an at homes NAS for example. That can be done, but requires manual work and might require more network capabilities and CPU power.

Advanced Networking

Last, but for me the most important point: Having such a box in the private network requires to drill a whole in the firewall, to allow port forwarding. I know, that is nothing unusual for experienced people, and in theory little problem.

But for people who are not so interested, that means they need to click in the interface of their router on a button that they do not understand what it does, and maybe even insert data by following an documentation that they have to believe. (That is not very much different from downloading a script from somewhere letting it do the changes which I would not recommend as well). Doing mistakes here could potentially have a huge impact for the network behind the router, without that the person who did it even has an understanding for.

Also DynDNS is needed: That is also not a big problem in theory and for geeks, but in practice it is nothing easily done.

With a good solution for private cloud, it should not be necessary to ask for that kind of setups.

Where to go from here?

There should be better ways to solve this problems with ownCloud, and I am sure ownCloud is the right tool to solve that problem. I will share some thought experiments that we were doing some time back to foster discussion on how we can use the Raspberry Pi with ownCloud (because it is a very attractive piece of hardware) and solve the problems.

This will be subject of an upcoming blog here, please stay tuned.