AJ requested DKMS which is the good news.

What is not so good are some comments that oppose need for DKMS.


Free and Open Source Software (FOSS) ideals are nice, and very special word in that name is Free, like freedom. Creating obstacles that will limit users ability to use what works for them will not help us, nor FOSS. We can recommend opensource software and install it by default, but we can't make sure that it works for everyone, in which case giving that as the only choice is forcing users to give up on Linux, or go elsewhere. Does that help to build user base?

With alternatives that are not very strict in enforcement of ideals (and ironically, bring few times more new users to the free and open source software), those that are not happy with openSUSE have no trouble to find what they need.

DKMS is solution for only one of obstacles, and there is many more. If we can't offer some stuff for legal reasons, there is no need to create other problems for people that want to run openSUSE.

Adding files to a SVN server is usually a task done in seconds. However, having several independent SVN repositories and wanting to “combine” them, this is not trivial—especially if you want to preserve the history.

The doc team had had three different, independent repositories on BerliOS (opensuse-ha-doc, opensuse-docmaker, and opensuse-lfl) all holding separate information. This was a bit silly, so my task was to consolidate them into opensuse-doc by keeping all history.

A SVN history is nice as you can see what you or others have done with the files. Loosing the history is the same as starting from scratch — which can be sometimes a good idea. In that case, the doc team wanted to preserve the history as we use it very often. So I had to think of a solution how to merge different SVN repositories into one. I came up with a solution containing the following steps:

1. Create a test repository
2. Create the dump files
3. Filter the dump files and extract trunk only
4. Rename trunk content
5. Edit the dump files
6. Load dump files into test repository

Several tools are needed: svnadmin and svndumpfilter are already available from the subversion package. Additionally, Darix recommended the svndumptool from devel:tools:scm:svn (thanks Darix!). A very convenient tool when dealing with SVN dumps.

Apart from this, all three SVN repositories consist of the usual trunk, branches, and a tags directories.

Step 1: Creating Test Repository

Before I got my hands dirty, I needed a test repository. Luckily, the subversion package contains everything what I need.  In my case, I’ve used the hotcopy command to create this test repository. I’ve logged in to BerliOS and made a hotcopy of my target SVN repository:

# ssh svn.berlios.de
# svnadmin hotcopy /svnroot/repos/opensuse-doc opensuse-doc

I did the same for my other repositories as well, just for safety reasons:

# svnadmin hotcopy /svnroot/repos/opensuse-lfl opensuse-lfl
# svnadmin hotcopy /svnroot/repos/opensuse-docmaker opensuse-docmaker
# svnadmin hotcopy /svnroot/repos/opensuse-ha-doc opensuse-ha-doc

Later, I will use the opensuse-doc hotcopy to test my modifications before I apply them to the production SVN repository. With this method, I wanted to make sure everything is correct. Of course, you have to be sure that nobody commits to your repositories, otherwise any later commits won’t be incorporated.

Step 2: Creating Dump Files

Dump files contain everything what’s inside a repository including SVN properties. However, they don’t contain the configuration or repository hooks from the repository. If you want to keep them, you’ll have to save them manually.

It’s very easy to create a dump file. In BerliOS, everything lives under /svnroot/repos/PROJECT:

# svnadmin dump /svnroot/repos/opensuse-lfl > opensuse-lfl.dump
# svnadmin dump /svnroot/repos/opensuse-docmaker > opensuse-docmaker.dump
# svnadmin dump /svnroot/repos/opensuse-ha-doc > opensuse-ha-doc.dump

The above lines create a dump file for each of my SVN repositories. As the BerliOS server doesn’t have the svndumptool command, I have to copy them to my own machine:

toms@earth:~ > scp shell.berlios.de:~/opensuse-*.dump .

Step 3: Extracting trunk

In most SVN repositories, tags and branches contain information which were at some point in time copied (for tags) or copied and modified later (for branches). I haven’t found a satisfying solution to use svndumptool and take into account these two directories. Anyway, I’ve decided to just concentrate on trunk and manually adjust tags and branches later. This makes it easier when dealing with dump files.

To extract only parts of a dump file, the svndumpfilter command is the right tool. Combined with the needed options and subcommands, I’ve used:

toms@earth:~ > svndumpfilter --renumber-revs --drop-empty-revs \
    include trunk < opensuse-docmaker.dump > docmaker-trunk.dump
toms@earth:~ > cat dumps/opensuse-ha-doc.dump | \
    svndumpfilter --renumber-revs --drop-empty-revs  include "trunk" | \
    svndumpfilter exclude trunk/package > ha-doc-trunk.dump
toms@earth:~ > svndumpfilter --renumber-revs --drop-empty-revs \
    include trunk < opensuse-lfl.dump > lfl-trunk.dump

The commands are not exactly the same, as the SVN repositories are slightly different. For example, the opensuse-ha-doc repo contained a trunk/package directory which I don’t want. Therefor you see the exclude command in svndumpfilter. The resulting dump files contain only trunk, nothing else.

Step 4: Renaming trunk Directory

The last step created dump files which contain only the trunk directory. The dump files can already be loaded into the target SVN repository. However, this is only partly successful. After loading you have to rename and move subdirectories around which is unconvenient. To avoid this cumbersome task, I’ve used the svndumptool command which does the job directly on the dump file!

For example, the former docmaker repository has to appear under trunk/tools/docmaker, not directly under trunk. This can be done with the merge subcommand of svndumptool:

toms@earth:~ > svndumptool merge -o docmaker-trunk-rename.dump \
   -i docmaker-trunk.dump \
   -s '^trunk' 'trunk/tools/docmaker' \
   -d trunk/tools/docmaker

The -d option creates the trunk/tools/docmaker directory in case it doesn’t exist in your target. The -i and -o options are the input and output stream of the dump files, -s contains a regular expression how to rename the source (^trunk) into the target (trunk/tools/docmaker). The other repositories are similar:

toms@earth:~ > svndumptool merge -o ha-doc-trunk-rename.dump  \
  -i ha-doc-trunk.dump \
  -d trunk/documents/ha/ \
  -s '^trunk/books/en' 'trunk/documents/ha/en' \
  -s '^trunk/books' 'trunk/documents'
toms@earth:~ > svndumptool merge -o lfl-trunk-rename.dump  \
  -i lfl-trunk.dump \
  -d trunk/documents/lfl/ \
  -s '^trunk/books/en' 'trunk/documents/lfl/en' \
  -s '^trunk/books' 'trunk/documents'

After this step I have three files, named *-trunk-rename.dump which contains the renamed directories.

Step 5: Editing Dump File

This step is a bit tricky and I haven’t found a better solution yet. Probably this can also be done with cat and sed magic, but I preferred vi. The task is to list the content of the dump files and decide which directory entries need to be deleted. This step is absolutely necessary to avoid any SVN error (something like “directory/file already exists”) when trying to load the dump file into the repository.

1. Get an overview of the contents:

   toms@earth:~ > svndumptool ls ha-doc-trunk-rename.dump
   /trunk
   ...
   toms@earth:~ > svndumptool ls docmaker-trunk-rename.dump
   /trunk
   /trunk/documents

   Usually this is /trunk and probably some other directories (depending on the structure).

2. Modify the dump file(s) and remove any directories which are available in the target SVN repository.
   The renamed directories from the last step contains a single trunk “node” which has to be removed. The same applies for trunk/documents. Both exist already on the BerliOS server. Manually edit the dump files and remove the following lines:

   Node-path: trunk
   Node-action: add
   Node-kind: dir
   Prop-content-length: 10
   Content-length: 10
   PROPS-END
   ...
   Node-path: trunk/documents
   Node-action: add
   Node-kind: dir
   Prop-content-length: 10
   Content-length: 10
     [[ Some other content could be available here ]]
   PROPS-END

3. Save the dump file

Be careful! It is very important not to destroy any structure in the dump file!

I’ve checked the result with “svndumptool ls DUMPFILE”. No lonely trunk directory should be shown anymore. To be on the safe side, I’ve checked the two files with svndumptool check -A DUMPFILE, too.

Step 6: Loading Dump Files into Test Repository

The last step created the dump files with the correct structure. As this was done on my own machine, I have to copy the files back to BerliOS (remember to use the shell.berlios.de server):

toms@earth:~ > scp *-rename.dump  shell.berlios.de:

It’s almost done! Logged into BerliOS and loaded the dump files in my hotcopy test repository (see step 1):

# svnadmin load opensuse-doc  < docmaker-trunk-rename.dump
# svnadmin load opensuse-doc  < ha-doc-trunk-rename.dump
# svnadmin load opensuse-doc  < lfl-trunk-rename.dump

After playing around a bit, everything was successful. So I repeated it, but replaced the opensuse-doc hotcopy with the correct path /svnroot/repos/opensuse-doc.

The remaining pieces are the tags and branches directory. As revision numbers have been changed, the revision number in the original repository and the revision number in opensuse-doc are obviously not the same anymore. Creating tags can only be done with a correct date, because dates are preserved. For this reason, it is convenient to still have the original repositories around to look for the exact date and log message:

$ svn copy -r "{2010-10-20T10:00:00}" -m "..." \
   $BERLIOSURL/trunk/tools/docmaker \
   $BERLIOSURL/tags/tools/docmaker/obs

The branches are a bit different. Maybe the history can be preserved with the same method as with trunk, but I haven’t tried it. Our three repositories hadn’t had meaningful information in branch, or it wasn’t worth the effort.

Conclusion

The steps above showed how to merge different SVN repository into one while keeping the history.  Unfortunately, this task could be easier. Although svndumpfilter is very easy, it is also very limited. Especially when you have to modify the dump file itself, svndumpfilter won’t rescue you. It would be very good, if svndumpfilter could be extended to work with dump files in the same way as svndumptool. That would be fun.

One of the features we want to include in the next MonoDevelop release is integrated support of Git. I committed a first version of a git add-in some months ago which works by invoking the git command and parsing the results. This solution satisfied some basic needs (Mono was moving to git at that time), but it was done as a short term solution, since parsing git output is not very reliable, and has portability issues.

We have a new place where we collect static documentation like manuals, user guides, quick start pages, developer documentation … and so on.
This new place is: rtfm.opensuse.org doc.opensuse.org.
Our “fabulous manuals” are also accessible at: doc.opensuse.org (in case you don’t like the word “fabulous”).

A few days ago Thomas Schraitle already wrote about this site before it was in a state that we wanted to announce – so sorry for any confusion this might have created.

This site is not meant to be a competitor to the documentation in the wiki. It was rather born from the need to have a central place to publish generated static documentation. Content on rtfm is not meant to be edited. If you want to contribute documentation we very welcome this in the wiki and encourage  you to link back to rtfm where applicable. The pages on rtfm are static but however they will be updated automatically upon changes (new releases of a project or product).

Currently we have published some great user guides and quick start pages for openSUSE (KDE, Gnome, Security, …) and SLES (AppArmor, Admin Guide, kvm, Xen, Security, …) as well as a user and vendor guide for WebYaST. There is also developer documentation available for YaST and Zypp (libzypp, satsolver) development.
In the next days or weeks we will add more documentation as soon as it is ready to be published.

Update:
We changed the domain name of that site. To read the fabulous manuals please use and link only to doc.opensuse.org. The domain docs.opensuse.org will be alias for it – please only link to doc and do not use rtfm anymore. Thanks for your appreciation.

Been doing some work and playing around with opensuse's great build service and as a result, my packages has been accepted by the opensuse team. So, now pensuse has python-flask and a few related packages in there.

Use the devel:languages:python and enjoy.

Alternatively use the search http://software.opensuse.org/113/en to find what you need.

openSUSE conference is over. It was an amazing and exhausting time. And the most important thing I learned was: The openSUSE project is a commuity. Of cause I meet many community members employed by Novell. But also many contributors NOT payed by that sponsor. The project is moving more and more into the direction of independence (e.g. with it's community based new strategy).
I was suspisious myself when I choose my distro some time ago. Shall I really try openSUSE, with all the deals the main sponsor is doing? From now on I am sure that these prejudices were wrong.


I had many inspiring conversations with many intresting people. One I want to mention here. There were three impressive guys from Austria. They created Linux for schools (desktop4education) based on openSUSE. The desktop is installed with 4 question, the server only needs 2 questions to be answered. More information at: http://d4e.at/ (DE-only)
I meet these heros end of November again and I hope we are going to improve the cooperation between their project, the KDE edu team and openSUSE-li-f-e (edu-project from openSUSE).
Greetings to Austria. You rock!

With established ARM support in OBS the as well as emulated MIPS and PowerPC is getting more mature, the last big embedded architecture not working in OBS with QEMU user mode was SH4. QEMU developers community had done a lot of work in improving QEMU user mode during the last months, so I can proudly present with currently only a few patches to QEMU git master OBS builds working with the SH4 port of Debian Sid. The new QEMU 0.13 released recently is a big milestone for this.

Another news is that I had fixed the bugs in Virtual Machine builds (build script) when using them with some architectures like PowerPC 32bit and SH4. So now also the combination of using for example KVM (XEN should also work) in a worker together with ARM, MIPS, PowerPC and SH4 is working. The appropriate fixes are in one of the next build script releases (if not even released already now with OBS 2.1, I have to check that). You can select architecture “sh4” with OBS 2.1 and also start a scheduler with “sh4”.

With the use of the QEMU User Mode, you can build also accelerated native cross toolchains for your host architecture so time critical parts like the compiler can run without the emulator. This works with .deb as well as with .rpm based backages. The MeeGo Project as well as the openSUSE Port to ARM uses this technique to provide an optimum between compatibility and performance. It means you can mix natively build packages and use cross toolchains on it. The “CBinstall:” feature helps you to use native or cross builds automatically depending on if your build host is a native machine or a x86 machine with cross build. In summary, we have the current classics of linux embedded archs together now in OBS: ARM, x86, MIPS 32, PowerPC 32 and SH4.

I have uploaded the fixed QEMU package to the OBS project openSUSE:Tools:Unstable inside the package “qemu-devel” after some more testing. I have of course also a OBS meta prjconf file working with Debian Sid. The SH4 port of Debian Sid you can find at Debian Ports Site.

And last but not least I would like to thank Riku Voipio of the Debian Project, QEMU project and MeeGo project and other major contributors during the QEMU 0.13 development cycle for the restless work on QEMU user mode improvements. In case of KVM, QEMU is used even twice, with QEMU-KVM as well as QEMU User Mode. I am sure I had forgotten other important people, so thanks to them also.

Before and during the openSUSE conference, some nice people (Jens-Daniel, Jürgen, Darix) created the following site for you:

http://rtfm.opensuse.org http://doc.opensuse.org

Thank you guys! I like the thrilling name.

It’s a static page (at the moment?) and collects the current documentation from several products and projects. Probably you will see more to come in the next weeks.

Have fun!

Update (AJ since Thomas is ill) 2010-10-27: Based on the feedback received, we’re going to  change now rtfm.opensuse.org to docs.opensuse.org. So, you can reach the fine side under http://docs.opensuse.org and http://doc.opensuse.org.

I couldn't resist snapping as many KDE folk at the openSUSE conference as I could, and editing them together into: a short video.

If anyone can tell me how to embed in kdedevelopers.org or enable the download of the OGG version from blip.tv, let me know!

Yet another foto stream from the openSUSE conference. You see the desktop leads from KDE and Gnome (Cornelius Schumacher and Vincent Untz) giving a talk about the past and future of the free desktop, Stephan Kulow about the future of the distribution, Bernhard Wiedemann about QA testing and so on.

Most important may be the presentation of the openSUSE board (mainly by Pascal Bleser) how they plan to found an independent foundation for openSUSE as non-profit organization. An important rule of that foundation is that it is independent of any company (no majority of Novell here) but can handle sponsoring, partnering and trademark questions.

We had also very filled rooms during the OBS talks, but I was unable to take pictures at that point of time unfortunately