Skip to main content

a silhouette of a person's head and shoulders, used as a default avatar

About memory ownership in SWIG and Python

Today, a long-standing bug in libyui-bindings was finally fixed.

About libyui and libyui-bindings

libyui-bindings provides SWIG based bindings for Ruby, Python and Perl to access libyui functionality. libyui is the YaST user interface library, allowing to write dialogs in a generic way so they can be displayed in text (ncurses) or graphical (Qt) environments.

Reference counting

The core of the problem is libyui relying on YaST data structures. These data structures implement reference counting, so one doesn't have to release allocated memory explicitly. That all fine, as long as you use YaST's YCP programming language to code dialogs. But coding YCP is not necessarily something a lot of developers are looking forward to.
Thats where libyui-bindings enter stage and remove the YCP restriction. I've blogged about it ealier and Jan-Simon was especially active in providing lot of Python examples.

Competing on memory ownership

Now Python is a nice scripting language but has the bad habit of doing reference counting itself. This makes coding Python extensions in C quite awkward and clashes with libyui. Speaking of bad habits, libyui also has one - claiming ownership of memory passed as pointers to some functions.
Fun starts when freeing memory. libyui and Python enter a nasty fight which Linux ends with a SIGSEGV. Jan-Simon reported this as a bug and provided a nice test case.

Swig typemaps

After some googling and reading the SWIG documentation a couple of times, SWIGs DISOWN typemap was the solution. Documentation for this is sparse and well hidden within the Chicken language chapter. Grrr.
SWIG typemaps apply to function parameters and match the parameter type and name. One cannot specify per-function typemaps, only per-parameter. Fixing the bug required libyui to explicitly name parameters claiming ownership of memory. All such parameters within the libyui API are now ending with _disown.

Solving the case

Applying the typemap is done using the %apply directive:
%apply SWIGTYPE *DISOWN { YItem *item_disown };
%apply SWIGTYPE *DISOWN { YEvent *event_disown };
%apply SWIGTYPE *DISOWN { YTableCell *cell_disown };
%apply SWIGTYPE *DISOWN { YWidgetID *newId_disown };
%apply SWIGTYPE *DISOWN { YTableHeader *header_disown };
%apply SWIGTYPE *DISOWN { YTableHeader *header_disown };
%apply SWIGTYPE *DISOWN { YWidget *parent_disown };
Now Python gives up ownership for structures passed through a _disown parameter to libyui and all libyui Python examples run without coredumps. Case closed.
a silhouette of a person's head and shoulders, used as a default avatar

the avatar of Andrew Wafaa

Community Content Required

At the launch of 11.0 there were several little projects started by the community these projects were actually really helpful and we would like to think contributed to the uptake and success of the 11.0 release. Now I will admit that I can’t substantiate this with any hard (or soft) facts, but none the less that’s our gut feeling 🙂

So what are the projects in question? Well they are: Helping Hands, openSUSE-Tutiorials and openSUSE TV. Funny I can hear some people moaning and groaning that these are GNOME Team projects, WRONG!! These projects have indeed been started by members of the GNOME Team, but they are for the whole distribution and project. I know suseROCKs has tried in the past to try and get some contributions from other teams, so don’t blame the GNOME Team for the content blame yourselves.

Actually on second thought, drop the whole blame game thing. A better solution is to start a fresh and keep up the momentum. I am basically asking members of the openSUSE Community (that means ALL of you!) to step up and take the wild Geeko by the reigns and help teach all those non-believers that it really is easy to use, has some brilliant features (easily found and hidden), and generally a great distro and project to use and be involved in.

So if we take each item in turn:

1) Helping Hands: HH (as we affectionately call it) is aimed at giving an insight into certain aspects of the distro that a user may come across or wish to use. In the past we have had sessions on Inkscape, Evolution, Banshee, general GNOME usage and other applications. We are hoping to have one on packaging RPMs and using the Build Service in the not so distant future. We would love to see members of the KDE community enlighten us (yes some of the GNOME users are scared of things that begin with K) about things that are going on in there, I know there are loads of great things I’m just really knowledgeable in them or even know 25% of them. We would also like members of XFCE and any other desktop environment that’s out there to do the same. You can focus on a specific application, a suite of applications or anything that will be helpful to users. If possible get someone from upstream join in, it makes things much more interesting and actually opens eyes on both sides of the fence as to what is going on. If you are interested (why wouldn’t you be?) and are willing to help out (you know you want to) then please let wither myself (FunkyPenguin) or suseROCKs know on IRC, we hang out in most of the openSUSE channels.

2) openSUSE-Tutiorials: oS-T is aimed at being a repository of insights, tips & tricks etc on applications and other things with the distro. Why not just use the Wiki? Well in a nutshell there are times when the Wiki just isn’t suited, and this way topics can be easily grouped so are ultimately easily searchable. Not only by the big search engines, but also by any user who visits the site. Content is moderated prior to being published, and before you start screaming of a conspiracy to silence the truth it is actually to try and ensure that those items published are actually of good quality and will genuinely be of use. For more info on publishing something for oS-T then please ask either decriptor or suseROCKs on IRC (again they loiter in most channels).

3) openSUSE TV: oS TV is aimed at providing a medium to show all the wonderous videos that we have on and about openSUSE. These don’t have to be any of the official videos that are done at conferences but can also include screencasts, interviews and almost any other form of video that we have about openSUSE (both distro and project). The channel is part of the blip.tv service and as such is aimed at all platforms, not just Linux. We want to try and grow our user base and community which means looking at the competing Operating Systems. I am hoping to do a series of interviews at the upcoming FOSDEM show similar to those that I did in Nurenberg for Hack Week III; I have pencilled in the title of “Face to Face @ FOSDEM’09”. So if you’re going to be at the event (try and make it if you can because it is really a great event), and you have a specific topic you’d like to speak about then let me know. I would love to get more content on there, screencast would be great as would videos from LUG meetings where openSUSE is used/showcased etc. Remember oS TV is about the community, for the community, by the community. So your input is needed!

There is no reason why you can’t do an item on all three of the above and have them linked. For an example you could do a HH item on the KDE desktop (general first look etc), you could also do a screencast of that item and have the screencast posted to oS TV and to finish it up you could have an oS-T article on tuning your desktop to get the best out of it. If you feel that any of these are wrong/bad/need improving/$COMMENT then please keep those comments to yourself!! In all seriousness, they can only improve with your help so please let us know all and any feedback that you may have. Most importantly, please join in 🙂

the avatar of Rupert Horstkötter

openSUSE forums has reached 20K members

The openSUSE forums has reached another key milestone. After hitting the 10K members in September 2008, we now did it again – Member 20.000 registered yesterday at 20:37 UTC.

Congratulations to the openSUSE forums team!

If you haven’t signed up for the forums yet, please do so. Sharing your knowledge with other openSUSE users at the openSUSE forums is a great way to contribute to the openSUSE community in a non-developing capacity.

a silhouette of a person's head and shoulders, used as a default avatar

One of the most unknown, yet most useful feature of MonoDevelop is the Go to File dialog.

One of the most unknown, yet most useful feature of MonoDevelop is the Go to File dialog. This dialog can be quickly opened by pressing Alt + Shift + O, and it can be used to quickly find and open a file of the project.

Some weeks ago I did several performance improvements, and introduced support for acronym matching. The filtering algorithm tries to find the best match, not only by searching substrings, but also by splitting the search string in several parts and trying to match them through all the words that compose the string being checked. This is an example:



When filtering files, the algorithm tries to match the file name, and if it fails, then it tries to match the whole file path. This allows filtering by directory. Following the example, if I type 'xml/ccomdata', it will fill find results under the MonoDevelop.XmlEditor.Completion directory. When there are several files with the same name, the list shows the parent directory name next to the file (inside brackets), so you can decide which one is the good one. Besides filtering, the dialog automatically highlights what it considers the best match.

After several weeks of use, I'm very happy about how the matching algorithm is working. Most of the time it manages to highlight the file I'm looking for by just typing 3 or 4 letters.

BTW, the same dialog supports searching by type name, using the same algorithm. In this case the shortcut is Control+Shift+T.
the avatar of Holger Macht

GreenIT: Power Consumption and -Optimization of PCs

"GreenIT: Power Consumption and -Optimization of PCs"

In the middle of last year, I participated in a seminar called "GreenIT" at university. The result was a 17 paged whitepaper called "GreenIT: Power Consumption and -Optimization of PCs". It took quite some time to finish this up, so it would be a waste of time if I would not publish it. A shortened table of contents looks like this:

  1. Introduction
  2. ACPI
  3. System as a Whole
  4. System Components
  5. Software Policies
  6. Conclusion

Please note: If you are only interested in how things are implemented in Linux, this might not be that interesting for you. It does not include any specific operating system details. It's rather general, although of a technical nature, and thus can be applied to any operating system out there.

If you are lazy of reading through the whole document, you might consider having a look at the corresponding presentation slides. However, you need to be able to speak, or at least to read, the German language for the slides to be useful for you.

a silhouette of a person's head and shoulders, used as a default avatar

Cute Harmony: Qt goes LGPL

KDE Project:

Thanks Nokia! It will be really interesting how this will impact the Qt and KDE communities, desktop, embedded, mobile, cross-desktop collaboration. Looking forward to a friendly competition on technical merits only. :-)

a silhouette of a person's head and shoulders, used as a default avatar

Report from Fourth Annual Management Developers Conference

The fourth Management Developers Conference (mandevcon) took place November 17 - 20 2008, in Santa Clara, California.

This is the annual get together of people involved and interested in systems management technologies and standards as defined by the Distributed Management Task Force (DMTF).

The DMTF is an industry organization defining systems management standards such as the Common Information Model (CIM) or Web Services for Management (WS-Management).

For my employer, these standards play an important role as they're part of the Novell/Microsoft technical collaboration agreement about interoperable systems management.

Compared to last year, attendance to the conference was very low. I guess only about a third of last years attendance numbers were reached. This decline was mostly attributed to slashed travel budgets due to the tough economy. However, hallway conversations also hinted towards lack of scope of the conference, trying to be everything for everyone. A split into research, development and deployment topics is probably needed.

Presentations during the three days of the conference are divided into three parallel tracks. These were
  • the DMTF industry initiatives (http://www.dmtf.org/initiatives)
    like
    • SMI (storage management)
    • CDM (diagnostic)
    • CMDB (configuration database)
    • Virtualization
    • SMASH (server hardware)
    • DASH (desktop hardware)
  • CIM core technology
  • Web Services for Management
  • Developers
The full conference schedule can be obtained from here.
Sadly, the presentation slides (mostly PDF) are only available to attendees (authorization needed).

Being busy with my own slides, I wasn't able to attend as many presentations as I wanted. But I still made it to the following presentations of the first day:

Python CIM providers

Writing CIM instrumentation has a long history of being complicated and needing C or C++. Bart Whiteley of Novell presented pywbem, showing how to write CIM providers in a modern scripting language. Pywbem also comes with a provider generator, spitting out a Python skeleton provider from the definition of its CIM classes.

Multi-Tenant CIMOM, Fact or Fiction

There are a couple of CIM object managers (CIMOMs) available, mostly as open-source. However, CIM providers are mostly CIMOM-specific and can't be easily ported. With the Common Manageablity Programming Interface (CMPI), a standard ABI for CIM providers is established. Still configuration information required to install a provider is different from CIMOM to CIMOM.

This presentation showed typical obstacles in writing portable providers and gave rules and recommendations to make a provider cimom-agnostic.

cmpi-bindings - Compiler Free Provider Development

This was the first of my own presentations. Cmpi-bindings serves a similar purpose like pywbem, freeing CIM provider developers from the need for C or C++.

cmpi-bindings uses SWIG to generate bindings between the Common Manageablity Programming Interface (CMPI, the ABI between a CIMOM and a CIM provider) and a scripting language. Currently supported are Python, Ruby and Perl. Any other SWIG supported language can be added fairly easily. cmpi-bindings use SWIG in an innovative way, allowing to call into the scripting language from the CIMOM. This effectively reverses the typical flow of control for SWIG, which is calling a C or C++ library from a scripting language.

Testing with PyWBEM

Automated testing of CIM providers is often neglegted and developers use plain CIM browsers to explore their code.

Tim Potter showed various ways for testing providers with the help of Python and pywbem. Most interesting was learning about Twisted, a Python networking framework for asynchronous programming. This stuff makes your brain hurt and the O'Reilly book title illustration is worth a thousand words ;-)



On day 2, I had the opportunity to attend Microsofts 'Strategic Architect Forum' in San Francisco. So I skipped the mandevcon presentations for that day. I'll blog later about the Microsoft conference.


The last day had a complete track on WS-Management.
It started with a generic introduction by Josh Cohen from Microsoft. Josh is leading the WS-Management work group within the DMTF and the driving force behind Microsofts adaption of an open standard for systems management. Too bad his presentation isn't available online.

Openwsman swig bindings

Anas Nashif, of AutoYaST fame, showed the Openwsman swig bindings Openwsman is an open source implementation of the WS-Management protocol, used e.g. by Microsoft Windows for remote systems management.

These bindings allow for easy ws-management client programming using one's favorite programming language. Fully supported currently are Python, Ruby and Java (yuck!).

Scripting openwsman server plugins

WS-Managment is a generic protocol for resource management. Each resource is identified by a universal resource identifier (URI), which onsists of a namespace and a resource part. The openwsman server maps namespaces to plugins and lets the plugin handle the specific resource implementation.

Openwsman includes default plugins for the generic Identify operation and for CIM operations (WS-CIM). Writing plugins in C is easy, using a scripting language makes it even easier.

My presentation shows how the problem was approached and describes the implementation. It is in an alpha state currently and needs some more work to become fully functional. This will also require some changes in the openwsman plugin API to fully support plugins.
The code is part of the openwsman svn repository and available here.

ActiveWSMAN

One thing that bothers me when coding against the openwsman client API is the amount of protocol knowledge needed. Doing a simple enumeration call requires one to know about the enumerate, pull and release protocol functions. Not to mention keeping track of the context.

Having enjoyed the ActiveRecord pattern as implemented in Ruby On Rails, I collected some thoughts around using this for WS-Management client programming.

The presentation shows a concept on how to make a developers life much easier. It focuses on resources and their semantics, completely hiding the internals of the ws-management protocol.

A proof-of-concept implementation based on ActiveRecord exists and I itend to complete it in the next couple of weeks.

the avatar of Carlos Gonçalves

openSUSE 11.1 KDE3 - torrents and final thanks

Torrents are now available:

Thanks to...
  • ... Benjamin Weber, Masim Sugianto, Andre Duffeck and Magnus Boman for have mirrored openSUSE 11.1 KDE3 Live CDs and USBs.
  • ... Marcus Rueckert for have created torrents, and Andreas Jaeger (you know why ;-) )
  • ... Stephan Kulow for the useful tips given on how to create the ISOs.
  • ... Stephan Binner for patches, suggestions, and feedback.
  • ... Joe Brockmeier for the release announcement.
  • ... you (!) for the time spent reading this crap of blog :-P

Finalizing, even though I strongly encourage users to upgrade to KDE 4 not only for how awesome it is (KDE 4.1.3 on openSUSE 11.1 is solid as rock!), but also because KDE 3 will be officially unsupported and thus not included on openSUSE 11.2 and later versions, I would like to hear what's in users mind about a possibility of having KDE 3 and openSUSE 11.1+ KDE3 Live CDs maintained and released by the community - any volunteers?

(Yes, I use KDE 4 and hell not stepping back to KDE 3! heh)
a silhouette of a person's head and shoulders, used as a default avatar

The G(C)ritter

After quite some time I went on a trip to Ireland again. The first two days I stayed in Longford and went on a sight-seeing tour, i.e., a friend drove me around. Finally I could see the river Shannon and Mount St. Patrick. From the statue which is some meters up the hill we had a lovely view over the little islands below. There was Achill as well where I once ambitiously wanted to take part in a half-marathon. Needless to say I had some excuses not to go there. I think we also went to Westport. Nice tourist town with colourful houses. On our way back and before going to Lord of the Dance in Roscommon we drove through uninhabited valleys and came to the only fjord in Ireland. Somehow the whole journey reminded me a bit of the volcanoe tour on Lanzarote. It's a bizarre and lovely landscape. After Lord of the Dance there was dense fog and we could only crawl back to Longford. There were no houses on miles on end, no cars either. Only thick fog. But then it approached. The Gritter (not the Critter as I first thought). I could see huge yellow lights subdued by the fog on the other lane. My friend said: I think that's The Gritter. He seemed somehow agitated and excited. He repeated several times: I think this is The Gritter. I didn't dare asking. My friend slowed down even more and I thought he's going to stop for an encounter with The Gritter. I still didn't dare asking. Little stones were making noises when hitting the car. And then it was gone. The Gritter. The Gritter is the creature that grits salt on the street. Somehow I was relieved when it was gone. I felt like in this film 'The Fog' and the gritter being a lost soul coming out from the fog to have its revenge.
I was really lucky in Dublin then on my walk to the lighthouse. There was sun and almost no wind at all.

The River Shannon


The Fjord





Dublin Bay





Bay with Lighthouse



The Lighthouse



Lighthouse Walk



A Seagull on the Salty Beach