During a last Hackweek I decided to work on some tool which will helps users with submitting a new bug. My idea is to create something like reportbug from Debian, but targeted to SUSE. So the first step was find the way how to communicate with a bugzilla (especially to bugzilla.novell.com – bnc). Fortunately I found several libraries, which should be used for it:

• perl-SUSE-BugzillaClient
• pybugz
• python-bugzilla

perl-SUSE-BugzillaClient is written by Thomas Schmidt especially for bnc, so it is able to login using iChain. It also provides few functions for bugzilla communication, but unfortunately only for querying. Pybugz is written by Alastair Tse for Gentoo development in Python. But it is not able to login through iChain and I did not extended it for bnc, because it uses a plain GET/POST communication with parsing of HTML using regexes.

The reason why I have used the python-bugzilla written by Will Woods from RedHat is simple. It uses XMLRPC API, which bugzilla provides. And it’s also written with a flexibility in mind, so writing of another type of Bugzilla was easy. Just some subclassing and reimplementing of few methods and voila, the NovellBugzilla type is here. Because it uses iChain, it was necessary to reimplement login/logout methods – I spend a lot of time to understand how it works. Osc uses a HTTP authentication, which should be supported too, but even if it works with api.opensuse.org, it did not work with bnc. So I used a Firefox and LiveHTTP headers extension to track a communication and implemented the login according it.

The useful advantage of Novell bugzilla is that I extended a method readconfig, which reads a config file. Novell Bugzilla can read the username/password from ~/.oscrc, so you don’t need to have a same password in another config file, if you use osc.

So on the end those few lines of Python code

import bugzilla

bnc = bugzilla.NovellBugzilla(url="https://bugzilla.novell.com", user="mvyskocil", password="XXXXX")
kwargs = {
    'product': 'openSUSE 11.2',
    'severity': 'Normal',
    'cc': [],
    'rep_platform': 'x86-64',
    'component': 'Java',
    'summary': '[Java:packages/ant]: Cannot find a Java virtual machine',
    'version': 'unspecified',
    'assigned_to': 'mvyskocil@novell.com',
    'op_sys': 'Linux',
    'description': 'This is a testing bug report\n'
    }
bug = bnc.createbug(**kwargs)
print bug.bug_id

Have created this bug#525549. Even if Python is readable for regular programmers, the **kwargs should be expressed. It maps the dictionary to named function arguments (called keyword arguments – kwargs – in a Python world), so both calls in following snippet are equivalent.

d = {'name' : 'value')
call(**d)
call(name='value')

The python-bugzilla comes also with a simple command line tool called bugzilla, so this tool should be used also from shell scripts. It is sometimes little bit raw. For example listing of all products returns an output as a Python dictionary

bugzilla --bztype NovellBugzilla --user mvyskocil --password xxxxxx info -p
...
{'description': 'openSUSE 11.1', 'internals': {'disallownew': 0, 'classification_id': 7340, 'name': 'openSUSE 11.1', 
'votestoconfirm': 0, 'milestone_required': 0, 'id': 651, 'votesperuser': 20, 'maxvotesperbug': 5, 'defaultmilestone': '---',
'milestoneurl': '', 'description': 'openSUSE 11.1'}, 'name': 'openSUSE 11.1', 'id': 651} 
{'description': 'openSUSE 11.2', 'internals': {'disallownew': 0, 'classification_id': 7340, 'name': 'openSUSE 11.2',
'votestoconfirm': 0, 'milestone_required': 0, 'id': 755, 'votesperuser': 20, 'maxvotesperbug': 5, 'defaultmilestone': '---',
'milestoneurl': '', 'description': 'openSUSE 11.2'}, 'name': 'openSUSE 11.2', 'id': 755}
...

which might be hard to parse in other language. But it provides a lot of functionality, so for other informations about usage, please use man bugzilla.

Minutes after I posted The new shiny Smolt client I've received lots of feedback via email, IM and Novell Bugzilla. The most annoying bugs were:

• Bug 525324 - [home:cgoncalves:playground] the Privacy Policy window has not scrollbar

• Bug 525322 - [home:cgoncalves:playground] smolt does not display special chars

All known bugs were fixed and Smolt 1.3.2 tagged.

Q: So, what's really new?!
A:

1. Smolt KDE4 popup made by Thomas Goettlicher:
   
   

The notification will be shown automatically upon KDE startup (X-KDE-autostart-phase=2)


2. Qt.ToolButtonTextBesideIcon (the text appears beside the icon) is now default per Nuno Pinheiro's suggestion:
   
   


3. Window size auto-adjusting:
   
   
   As you can also see BNC #525322 is indeed fixed.
   

I'm preparing a Smolt RPM update to submit to openSUSE Factory to get included in time for openSUSE 11.2, though you can try it already by using my home:cgoncalves:playground repository:

openSUSE 11.1:
# zypper ar http://download.opensuse.org/repositories/home:/cgoncalves:/playground/openSUSE_11.1/ home:cgoncalves:playground
openSUSE Factory:
# zypper ar http://download.opensuse.org/repositories/home:/cgoncalves:/playground/openSUSE_Factory/ home:cgoncalves:playground

# zypper ref home:cgoncalves:playground
# zypper in smolt-snapshot smolt-popup

As always, please give us feedback!

Available today as part of the SUSE Appliance Program is SUSE Studio 1.0 based on the  image creator technology called kiwi. When creating an appliance with SUSE Studio you also have the possibility to export the appliance description to your local computer and use the kiwi backend directly to understand more about image creation and deployment

A professional linux distribution should be able to work as an appliance which is an ll-in-one solution including the application and the operating system. A basic appliance to start with is the JeOS – Just Enough Operating System. kiwi provides these as examples in the kiwi-templates package. To create your first SUSE 11.1 appliance just type:

kiwi --build suse-11.1-JeOS -d /destination/path

The primary image type of a JeOS template is a virtual disk which you can run in a virtual machine like QEmu, KVM, Vmware, VirtualBox, etc… To do this with qemu just call:

qemu /destination/path/LimeJeOS-openSUSE-11.1.i686-1.11.1.vmdk

and here you go with your first appliance. You want to know more about kiwi, just take a look at the wiki here:

KIWI Cookbook

or read the full system documentation as PDF here:

KIWI System Documentation

Remember to have fun

During the last week I had the possibility to work on anything I wanted, Novell’s hackweek is so cool :)

I decided to dedicate myself to an idea that has been obsessing me since a long time. Last December my brand new hard disk suddenly died, making impossible to recover anything. Fortunately I had just synchronized the most important documents between my workstation and my laptop, so I didn’t lose anything important. This incident make me realize that I should perform backups regularly and I immediately started looking for a good solution.

Personally I think that doing backups is pretty boring so I wanted something damned easy to setup and use. Something that once configured runs in the background and does the dirty job without bothering you. Let’s face the truth: I wanted Apple’s Time Machine for KDE.

After some searches I realized that nothing was fitting my requirements and I decided to create something new: kaveau.

What is kaveau?

Kaveau is a backup program that makes backup easy for the average user.

As you will see while coding/planning kaveau I made some assumptions and so only few things are configurable. I really think that sometimes “less is more”.

What does kaveau?

Current features:

• it performs backups to an external storage device: I don’t think it will ever store backup data on a remote host. If you want to do that just use some good project like Backup pc.
• it backs up the complete home directory of the user: storage is cheap and average users (like me) keep everything inside their home directory ( but it’s possible to exclude some directories from the backup).
• it performs incremental backups.
• the backup data are neither compressed nor stored in fancy formats: in this way you can plug your external disk into another machine and access your data without additional work.
• backups are performed automatically every hour (of course only if your external disk is plugged).
• notification messages are shown if your backup is older that a week. More enhancements are coming…

What technologies does it use?

Backups are performed using rdiff-backup because it’s damned easy to use, well tested (it’s used also in production environments) and packaged by all distributions.

The awesome solid library is used for interacting with the external disks is super easy.

Status of kaveau

I have been working on kaveau just for five days, so there’s still a lot of work to do.

A screenshot tour will give you the right idea of its status.

First run - external storage device attached

####Backup wizard - page 1

####Backup wizard - page 2

####Backup wizard - final page

####Backup operation in progress

####Backup completed

Right now the code is available on this git repository but I don’t recommend you to try it (unless you want to find and fix bugs ;) ).

I would really appreciate:

• feedback about the user interface (right now it looks too much like Time Machine).
• icons: it would be great to have a desktop icon and some system tray icons (contact me for more details).
• new code, bug fixes, code reviews, hints,…

Wrote a small utility to print live graphs of I/O per-process. Find more information here. As a bonus added graphs for CPU as well.

RPM can be found here: http://download.opensuse.org/repositories/home://nikanth/openSUSE_11.1/noarch/

It has been a long since I last blogged (last dating from January)... I haven't blogged since then mostly because I'm not that good when it comes to write more than 5 lines of text and also because I've been busy with my studies/exams which have finished 2 weeks ago. So... what I've been up to in these last 2 weeks?! Well, first week was dedicated to random useless stuffs like catching up on new movies, going out with friends (lots of drinks involved, obviously!) and, most of all, sleeping (yesterday when I woke up and looked at the clock it was 4pm!) - it fells so good sleeping with nothing to worry you about, doesn't it?!

Since end of last week I restarted working on my Smolt Qt Client, which was born during first semester exams (nothing to do in that boring and meaningless period, right?). But before I continue, let me quote what the Smolt project is all about:

Smolt is, very simply, a hardware reporting tool for Linux based systems. It was originally written for Fedora and now also supports SuSE, Debian and Ubuntu. It gathers information from users, and stores them in a queryable database with a web frontend that produces statistics.

More information about Smolt here. The original Smolt client was written in GTK but the Smolt maintainer decided to replace it by mine - "With great power comes great responsibility".
The client is quite a straightforward port of the GTK one as you can see:




Noticed the new "Distribution" tab? In a near future it will provides distribution specific information such as installed packages. It has been developed by Sebastian Pipping as a Google Summer of Code 2009 project to support Gentoo, though support for other distributions are currently under development (openSUSE/SUSE Linux, Fedora and Debian):


I will be fixing last bugs found to then push a new release version out late this week, so before openSUSE 11.2 feature/version freeze. In the meantime if you want to try it and even report bugs, checkout my home:cgoncalves:playground repository for the smolt-snapshot package:

openSUSE 11.1:
# zypper ar http://download.opensuse.org/repositories/home:/cgoncalves:/playground/openSUSE_11.1/ home:cgoncalves:playground
openSUSE Factory:
# zypper ar http://download.opensuse.org/repositories/home:/cgoncalves:/playground/openSUSE_Factory/ home:cgoncalves:playground

# zypper ref home:cgoncalves:playground
# zypper in smolt-snapshot

Please, give us feedback! We want to know your opinion about Smolt!

In addition to well-developed code, testing is a major part of ensuring a rock-solid openSUSE Linux distro. To make sure testing of openSUSE 11.2 (and beyond) are done in a well-organized way, we’re improving the way the openSUSE Core Test Team works together to ensure top quality for openSUSE.

In the next couple of days Milestone 4 of openSUSE 11.2 will be available. This is a very good moment to have a closer look into the next openSUSE release: some new features are already implemented and there is enough time to fix reported problems.

Thus we are looking for 15 members of the openSUSE Community that are willing to contribute to the openSUSE project by joining the openSUSE Core Test Team and operate in the following areas:

• Check if new features are implemented and working as requested
• Have a deeper look into the install and update system of openSUSE and ensure a broad hardware coverage
• Creating, improving and executing test cases for various areas of the distribution

Repository for test cases and tracking system for all test results will be Testopia, the test case management extension of Novell’s Bugzilla. Members of the Core Test Team will get access to the openSUSE test plans and will so be able to contribute. Beside storing test results they are also able to create or modify test cases.

If you are really interested in joining the openSUSE Core Test Team and willing to spend a reasonable amount of time to move the openSUSE project forward please get in contact with Holger Sickenberg <holgi at suse.de> providing following information:

• Your Linux experience
• Previous testing experience, if any
• Areas you are interested in testing

The number of members is limited to 15 at the moment to ensure we are able to adequate support everyone of them. We will add more once we’ve figured out in the smaller group whether everything works.

Of course everybody is still able to contribute to the openSUSE project by testing parts of the distribution. Enabling the openSUSE Core Test Team will not have any impact on that. Further information on testing is available at www.opensuse.org/Testing.

We are looking forward to your application. Deadline for applications is August, 15th 2009.

I've added support for PowerPlayInfo_V4 (the one I reverse engineered lately) in radeonhd today - and a whole bunch of heuristic validation routines. Which means that the driver finally has some idea about the possible chip frequencies and voltages .
Determination of some of the values is still scary, and I don't know yet whether we will stumble over them - especially the minimum frequencies are sort-of guessed from the (known) minimum PLL output frequencies. But we need them, because some cards (e.g. one RV770 I have here) only have one known good memory clock configuration, while it can save tons of power if it is lowered (tested + measured, GDDR5 memory really uses a lot of power). Therefor, the values are checked for reasonable bounds, and rejected if they exceed. So if a future chip needs less than 500mV for operation, we will have to update these tests...

The current code has all these bits in place, but doesn't configure anything different than previous versions. First, a set of reasonable settings has to be determined, which will need additional heuristics (this is done in rhdPmSelectSettings(), which needs some love still). This selection also depends on how difficult it will be to change the engine and memory clock, and the VDDC voltage. So far we only set the engine clock, but code is already in place for the other two.
Before setting a clock, it must be made sure that the engines using this clock are idle - that's why we (ahem!) only set the engine clock once at the beginning so far:

• For setting the engine clock, the engine must be idle (surprise).
• For setting the memory clock, memory accesses must be prohibited. Which means that the screen will be blank during this phase. It remains to be seen whether the vertical blank is long enough to do this on the fly. Otherwise this means that we typically shouldn't change the memory clock during runtime. Things get messy when multiple screens are attached...
• For setting the VDDC voltage, all engines must be idle. Also, certain combinations of engine and memory clock require certain VDDC voltages.

The amount of known good configurations is limited, and sometimes they are contradicting . Remains to be seen whether we can sort-of interpolate between good settings, because typically there are more voltage values available than used...

Later we will need an oracle for selecting the right power state according to the current usage pattern. It might be easier to do that in kernel space, but this remains to be seen.

That's it for today, I actually hoped to get more accomplished during our HackWeek. But reading out dynamic AtomBIOS tables can be... intriguingly complex.

This years Linux Bierwanderung will return to its origins, namely Upper Franconia with its breweries and nice country side.

There is no entrance fee, you just have to pay your own hotel+food+travel ;)

It is not too late to join, in fact you can even just drop by, say hi and join in.

I will probably go, even though I have not booked yet. ;)

During the ongoing Novell Hack Week I have been working on a port of Valgrind to Windows. I had done some initial hacking on this earlier this year in my spare time, but then left it to rest.

At first I had to resolve lots of SVN merge conflicts after doing an SVN update for the first time in months. The code for Darwin had been merged into Valgrind's trunk and this of course had touched lots of the same places that also my changes touch. After getting it to build again I was relieved to find that it still worked more or less like last time I had hacked on it, and I could continue with real coding.

Despite warnings from upstream developers that a Windows port of Valgrind would basically mean a total rewrite, it hasn't turned out to be that bad. In most of the source files with conditional compilation for the different platforms, the Windows code is basically just one more elif branch. Many parts of Valgrind, like the VEX virtual CPU or the Stabs and Dwarf debug information handling, practically work as such also on Windows.

So, does Valgrind work on Windows? Yes but actually no;)

For trivial console programs, sure, it kinda works, Memcheck detects the kinds of errors it is supposed to detect. (Memcheck is the only Valgrind tool that builds so far.)

But for "real" programs with a GUI it basically doesn't work. A lot of work remains to be done, and some mysteries remain to be solved. A healthy dose of pessimism is useful: I am not that sure it will ever work well enough to be useful.

What kinds of applications could Valgrind eventually be useful for on Windows? Personally, I would look forward to the possibility to use it on Open Source applications based on Open Source libraries like the GTK+ and (partial) GNOME stacks. GIMP or Evolution come to mind. Why not Qt-based applications, too. Perhaps even OpenOffice.org. I am not personally interested in making Valgrind work for applications based on weird proprietary technologies like COM.

I have been hacking it on 32-bit XP. Some minor additions are needed to handle also WOW64, i.e. 32-bit code on 64-bit Windows, and more major additions for 64-bit Windows.

So, what can you do to help? If you are an application developer with no in-depth Windows knowledge, don't bother. If you are an experienced Windows hacker with knowledge of kernel and Win32 subsystem system calls (or are prepared to learn about such things, like I was), you can help.

For anybody interested in helping, prerequisites would be as follows:

• A basic understanding of how Valgrind works on Linux
  
• A MinGW and MSYS installation, and some experience of gcc in general and MinGW in particular
• An SVN client, to keep up with upstream development
  
• Autotools, to be able to run the autogen.sh. If requested, I might be convinced to run a make dist now and then to create a tarball that includes my diffs and a pre-generated configure script.
• Debugging tools for Windows, to get the latest windbg.dll, and to get WinDbg which is essential for finding out how undocumented things work. (Of course, if you have more advanced tools like SoftICE available, the better.)
• Literature (or a good knowledge of these things by heart...). I recommend the following books:
• Windows NT/2000 Native API Reference by GaryNebbett. Documents the kernel "native" API. Needed for writing wrappers for kernel system calls. Might be hard to find. Much of the same information is also available on the web.
  
• Windows Internals, Fifth Edition by Mark E. Russinovich, David A. Solomon and Alex Ionescu. A classic, the fifth edition of which has just been published.
• Advanced Windows Debugging by Mario Hewardt and Daniel Pravat.
• Windows Graphics Programming: Win32 GDI and DirectDraw by Feng Yuan might also be useful.
  
• Wine and ReactOS sources can be useful at times

What remains to be done? Lots of things. Off the top of my head:

• Decide whether all this makes any sense at all, whether it's best just to declare it a failure before wasting any more time.
  
• Write wrappers for more kernel system calls.
• Write wrappers for such Win32 (GDI, USER) system calls that modify data (to tell the tool what memory has been written to, to avoid invalid warnings about use of uninitialized data). Mostly undocumented territory here.
  
• Fix the mysterious issues in handling callbacks from system calls (this happens a lot in GUI programs; in fact most user code in such executes in callbacks). Callbacks are a bit like signal handlers in POSIX, except that they aren't, as callbacks and system calls can be stacked arbitrarily high.
• Fix issues related to communication with csrss.exe. Again, this is mostly undocumented foo...
  
• Does SEH need any handling?
• Read MSVC debugging information (.pdb files). (Either using Microsoft's APIs, or the existing code in Valgrind.)
  
• Test on Vista and Windows 7
• Make it cross-compilable from Linux
  
• Make it buildable with Microsoft's tools. (Either just a makefile for nmake, or a Visual Studio project.)
  
• Once it works well enough, some kind of integration with Visual Studio

It is a bit sad that when doing web searches for undocumented Windows information useful in a project like this, a lot of the times you end up at information originating in the malware-oriented community.

Making Valgrind buildable with Microsoft tools would not be impossible. If I get offers of help from otherwise qualified people who would want to help but can't/won't use MinGW, I might try. A couple of short inline assembly parts would need MSVC syntax versions. A couple of assembler source files would need MASM syntax versions. Does MSVC support vararg macros in its preprocessor?

I have diffs at http://www.iki.fi/tml/valgrind/ . Apply to valgrind trunk from SVN as of the time indicated. Making binaries available is not really useful at this stage (if ever).