Skip to main content

the avatar of Cameron Seader

openSUSE 13.x / Factory processor P-States and Performance

Since the introduction of P-States in the Intel SandyBridge and newer processors and the introduction of the P-States driver in the kernel since 3.9 there have been some changes to the power management on systems in regards to userspace tools. It has moved from cpufreq to cpupower and you may have written a script in times past to help set the right power management governor for your system. On a system with P-States you find that using cpupower has no effect on the performance whatsoever when you change the governor with cpupower. In order to get high performance out of your system with P-States you will need to look at some parameters into sysfs and change them using the userspace tool cpupower. Lets have a look at what there is for P-States.
Change your directory to /sys/devices/system/cpu/intel_pstate

system:/sys/devices/system/cpu/intel_pstate # l
total 0
drwxr-xr-x  2 root root    0 Oct 21 18:45 ./
drwxr-xr-x 14 root root    0 Oct 21 18:45 ../
-rw-r--r--  1 root root 4096 Oct 21 18:45 max_perf_pct
-rw-r--r--  1 root root 4096 Oct 21 18:45 min_perf_pct
-rw-r--r--  1 root root 4096 Oct 21 18:45 no_turbo
We have the max_perf_pct and the min_perf_pct and if we cat these files we can see their values.
# cat max_perf_pct
100
# cat min_perf_pct
32
This is the default for a powersave governor which you can gather from running the following command.
# cpupower frequency-info
  analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 0.97 ms.
  hardware limits: 1.20 GHz - 3.70 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 1.20 GHz and 3.70 GHz.
                  The governor "powersave" may decide which speed to use
                  within this range.
  current CPU frequency is 3.53 GHz (asserted by call to hardware).
  boost state support:
    Supported: yes
    Active: yes
    3500 MHz max turbo 4 active cores
    3500 MHz max turbo 3 active cores
    3600 MHz max turbo 2 active cores
    3700 MHz max turbo 1 active cores
Notice the driver is intel_pstate and the current policy is set to powersave

We want the performance governor. So we will need to change our governor to performance. Execute the following.
# cpupower frequency-set -g performance

# cpupower frequency-info
analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 0.97 ms.
  hardware limits: 1.20 GHz - 3.70 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 1.20 GHz and 3.70 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 2.83 GHz (asserted by call to hardware).
  boost state support:
    Supported: yes
    Active: yes
    3500 MHz max turbo 4 active cores
    3500 MHz max turbo 3 active cores
    3600 MHz max turbo 2 active cores
    3700 MHz max turbo 1 active cores
Also if we cat /sys/devices/system/cpu/intel_pstate/min_perf_pct you will notice that it has changed to 100

Thats good its all at 100% but wait we still are not done. There is another setting for P-States. Its called Performance Bias. From the man page cpupower-set you can read the following about it.

----snip----
Options
--perf-bias, -b
Sets a register on supported Intel processore which allows software to convey its policy for the relative importance of performance versus energy savings to the processor.

The range of valid numbers is 0-15, where 0 is maximum performance and 15 ismaximum energy efficiency.

The processor uses this information in model-specific ways when it must select trade-offs between performance and energy efficiency.

This policy hint does not supersede Processor Performance states (P-states) or CPUIdle power states (C-states), but allows software to have influence where it would otherwise be unable to express a preference.

For example, this setting may tell the hardware how aggressively or conservatively to control frequency in the "turbo range" above the explicitly OS-controlled P-state frequency range.It may also tell the hardware how aggressively it should enter the OS requested C-states.

This option can be applied to individual cores only via the --cpu option, cpupower(1).

Setting the performance bias value on one CPU can modify the setting on related CPUs as well (for example all CPUs on one socket), because of hardware restrictions. Use cpupower -c all info -b to verify.

This options needs the msr kernel driver (CONFIG_X86_MSR) loaded.
----snip----

So lets set our bias to 0 so we can get absolute maximum performance. The default is 8 on openSUSE. Execute the following.
# cpupower set -b 0
and to check it.
# cpupower info
analyzing CPU 0:
perf-bias: 0
even though it only shows CPU 0 it applies for all and you can see that by adding the -c all switch before info.
Now you have a system running at full performance with P-States.
Note: This will run the CPU's hot and the fans will kick in full speed all the time. So when your away from your system or don't need full performance you will want to put it back in powersave. I'm not responsible for overheating of your CPU. :-)

the avatar of James Willcox

MP4 improvements in Firefox for Android

One of the things that has always been a bit of a struggle in Firefox for Android is getting reliable video decoding for H264. For a couple of years, we’ve been shipping an implementation that went through great heroics in order to use libstagefright directly. While it does work fine in many cases, we consistently get reports of videos not playing, not displayed correctly, or just crashing.

In Android 4.1, Google added the MediaCodec class to the SDK. This provides a blessed interface to the underlying libstagefright API, so presumably it will be far more reliable. This summer, my intern Martin McDonough worked on adding a decoding backend in Firefox for Android that uses this class. I expected him to be able to get something that sort of worked by the end of the internship, but he totally shocked me by having video on the screen inside of two weeks. This included some time spent modifying our JNI bindings generator to work against the Android SDK. You can view Martin’s intern presentation on Air Mozilla.

While the API for MediaCodec seems relatively straightforward, there are several details you need to get right or the whole thing falls apart. Martin constantly ran into problems where it would throw IllegalStateException for seemingly no valid reason. There was no error message or other explanation in the exception. This made development pretty frustrating, but he fought through it. It looks like Google has improved both the documentation and the error handling in the API as of Lollipop, so that’s good to see.

As Martin wrapped up his internship he was working on handling the video frames as output by the decoder. Ideally you would get some kind of sane YUV variation, but this often is not the case. Qualcomm devices frequently output in their own proprietary format, OMX_QCOM_COLOR_FormatYUV420PackedSemiPlanar64x32Tile2m8ka. You’ll notice this doesn’t even appear in the list of possibilities according to MediaCodecInfo.CodecCapabilities. It does, however, appear in the OMX headers, along with a handful of other proprietary formats. Great, so Android has this mostly-nice class to decode video, but you can’t do anything with the output? Yeah. Kinda. It turns out we actually have code to handle this format for B2G, because we run on QC hardware there, so this specific case had a possible solution. But maybe there is a better way?

I know from my work on supporting Flash on Android that we use a SurfaceTexture there to render video layers from the plugin. It worked really well most of the time. We can use that with MediaCodec too. With this output path we don’t ever see the raw data; it goes straight into the Surface attached to the SurfaceTexture. You can then composite it with OpenGL and the crazy format conversions are done by the GPU. Pretty nice! I think handling all the different YUV conversions would’ve been a huge source of pain, so I was happy to eliminate that entire class of bugs. I imagine the GPU conversions are probably faster, too.

There is one problem with this. Sometimes we need to do something with the video other than composite it onto the screen with OpenGL. One common usage is to draw the video into a canvas (either 2D or WebGL). Now we have a problem, because the only way to get stuff out of the SurfaceTexture (and the attached Surface) is to draw it with OpenGL. Initially, my plan to handle this was to ask the compositor to draw this single SurfaceTexture separately into a temporary FBO, read it back, and give me those bits. It worked, but boy was it ugly. There has to be a better way, right? There is, but it’s still not great. SurfaceTexture, as of Jelly Bean, allows you to attach and detach a GL context. Once attached, the updateTexImage() call updates whatever texture you attached. Detaching frees that texture, and makes the SurfaceTexture able to be attached to another texture (or GL context). My idea was to only attach the compositor to the SurfaceTexture while it was drawing it, and detach after. This would leave the SurfaceTexture able to be consumed by another GL context/texture. For doing the readback, we just attach to a context created specifically for this purpose on the main thread, blit the texture to a FBO, read the pixels, detach. Performance is not great, as glReadPixels() always seems to be slow on mobile GPUs, but it works. And it doesn’t involve IPC to the compositor. I had to resort to a little hack to make some of this work well, though. Right now there is no way to create a SurfaceTexture in an initially detached state. You must always pass a texture in the constructor, so I pass 0 and then immediately call detachFromGLContext(). Pretty crappy, but it should be relatively safe. I filed an Android bug to request a no-arg constructor for SurfaceTexture more than two years ago, but nothing has happened. I’m not sure why Google even allows people to file stuff, honestly.

tl;dr: Video decoding should be much better in Firefox for Android as of today’s Nightly if you are on Jelly Bean or higher. Please give it a try, especially if you’ve had problems in the past. Also, file bugs if you have issues!

the avatar of Andrew Wafaa

Moto-ring into Wearables

I must admit, I’m actually really happy with the Moto360. After reading a load of reviews and speaking to people that already had the device, I was fully prepared for a subpar experience. Maybe that’s the beauty of not being a bleeding edge early adopter :-) The woes that were extolled were many, and the only good thing people had to say were about the stunning good looks; battery life was woeful, not even lasting a day; performance was hit and miss; connectivity to phone was spotty; the list goes on, one almost wonders why on earth I would still want one.

a silhouette of a person's head and shoulders, used as a default avatar

A Truly Asian Summit

What a conference!!! openSUSE Asia Summit was unforgettable. The whole event had an amazing feel to it, and I had a rocking time in Beijing. openSUSE really has one of the best and most helpful communities, and the people are amazing. I had the pleasure of interacting with the active organization community. You guys absolutely rock!!!

Here are some of my experiences, and things I learned at the conference:

A little history:
The beginnings of the summit go back a year to Thessaloniki, where the idea of the Asia Summit was first mooted. Due to time constraints and clashes with openSUSE Summit in the US called for the event to be shifted to 2014. I had interacted with Sunny and Max in Thessaloniki and loved the idea of having an openSUSE event in Asia. At oSC14 in Dubrovnik, it was more or less confirmed that the summit would take place.

The Organization Team:
I joined the organization team a little late. The others had already done a lot of the hard work. I helped around a little with the invite letter and the promotion of the logo contest, and trying to find people to help around with the artwork. The opening session, where we welcomed the attendees in our native languages was cool. Also, Sunny’s speech in the beginning, which took us through the past one year was memorable, and I still remember each and every word of it.

I would take this opportunity to thank the openSUSE.Asia Summit
organization team. Today, now the openSUSE.Asia summit has started,
I’m reminded of the journey we took to get here.
I can not forget our weekly meetings, which often lasted to midnight.
I can’t forget 137 cards in trello for the preparation tracking.
And I can’t forget hundreds of emails about the Summit in our mail
boxes.

When we were on the way to reach this summit, we encouraged and
supported each other. Even though we were tired, we never gave up,
because we did believe we would finally be here. It is my honor being
a member of such a great team!

There are 17 people in the organization team, I won’t list everyone’s
name because we are a team, and we couldn’t have make any success
without each of us.

-Sunny

The organization committee did a fantastic job with the event and everything was planned to perfection. I would love to work with you all to host openSUSE Asia Summit next time as well (hopefully in India 😉 )

New things:
I absolutely loved the concept of ‘Chops’ where the workshop speakers would put a lovely ‘Geeko’ stamp on the brochure for the participants for the performance in the workshop. More than judging the performance, it gave us a good chance to interact with the attendees and have a lot of fun in the process. The gifts for the speakers and for the chops were great and well thought out. Personally, working with the organization team was very fruitful and I learned how an event of this scale is organized from the ground up. Additionally, being a room coordinator was a novel experience as well.

The Event:
To put the event in a single word: memorable. It was a very well conducted event and the speakers did a great job. The workshops and talks were conducted both the from the point of view of newbies as well as seasoned contributors. I particularly liked Richard’s opening session where the direction that openSUSE (with respect to Factory and Tumbleweed) is taking became clear. There were workshops on Bugzilla and OBS which were really helpful for getting new contributors involved.

Talk is Cheap. Show me the Code:
This was the single biggest lesson I learned during conducting my sessions. While taking the Qt Workshop, I was talking about basic object oriented concepts like Inheritance. The attendees (mostly students from the university) gave me a blank look. I was not sure whether they understood me or not. I ultimately decided to show them some code. They understood that. At that point I realized that there is one universal language that we could communicate with: CODE. That made the job a whole lot easier, and the rest of the sessions went well. I also made some ‘brilliant’ errors during the workshop, which demonstrated some or the other concept with respect to Qt. Overall, had a fun time conducting the workshop.

Food:
Being a vegetarian, I was not sure how I would survive in China. I absolutely indulged myself and tried out plenty of stuff. I have never eaten so much in a conference than I had in Beijing. Thanks to the awesome community guys, specially David who helped me a lot in finding out things to eat. The food was amazing. I can safely conclude that the Chinese take their food seriously. Plus, I learned how to use chopsticks properly. Thanks ftake for that 😉

China and Sightseeing:
This was one trip where I did not do much sightseeing. I had talks for both the days and could not spare the time. To my dismay, I found that visiting the Great Wall requires a full day, and I had just about 6 hours to spare. In the end, I just visited the Forbidden City and the Bird’s Nest (Olympic Stadium). I should have made my travel plans a little better and stayed an extra day.
I found Beijing to be an excellent city. I managed to get around pretty comfortably on the subway despite the language issues. I found the subway system quite effective and very cheap (2 Yuan is dirt cheap). The only problem I faced was the air pollution, which was a little unexpected. Other than that, the people were amazing and really helpful.

TSP:

Thanks to the openSUSE Travel Support program, that I, and many others got to attend the event. It is really an amazing program, and I hope that contributors use it very effectively.

the avatar of Jeffrey Stedfast

The Wait Is Over: MimeKit and MailKit Reach 1.0

After about a year in the making for MimeKit and nearly 8 months for MailKit, they've finally reached 1.0 status.

I started really working on MimeKit about a year ago wanting to give the .NET community a top-notch MIME parser that could handle anything the real world could throw at it. I wanted it to run on any platform that can run .NET (including mobile) and do it with remarkable speed and grace. I wanted to make it such that re-serializing the message would be a byte-for-byte copy of the original so that no data would ever be lost. This was also very important for my last goal, which was to support S/MIME and PGP out of the box.

All of these goals for MimeKit have been reached (partly thanks to the BouncyCastle project for the crypto support).

At the start of December last year, I began working on MailKit to aid in the adoption of MimeKit. It became clear that without a way to inter-operate with the various types of mail servers, .NET developers would be unlikely to adopt it.

I started off implementing an SmtpClient with support for SASL authentication, STARTTLS, and PIPELINING support.

Soon after, I began working on a Pop3Client that was designed such that I could use MimeKit to parse messages on the fly, directly from the socket, without needing to read the message data line-by-line looking for a ".\r\n" sequence, concatenating the lines into a massive memory buffer before I could start to parse the message. This fact, combined with the fact that MimeKit's message parser is orders of magnitude faster than any other .NET parser I could find, makes MailKit the fastest POP3 library the world has ever seen.

After a month or so of avoiding the inevitable, I finally began working on an ImapClient which took me roughly two weeks to produce the initial prototype (compared to a single weekend for each of the other protocols). After many months of implementing dozens of the more widely used IMAP4 extensions (including the GMail extensions) and tweaking the APIs (along with bug fixing) thanks to feedback from some of the early adopters, I believe that it is finally complete enough to call 1.0.

In July, at the request of someone involved with a number of the IETF email-related specifications, I also implemented support for the new Internationalized Email standards, making MimeKit and MailKit the first - and only - .NET email libraries to support these standards.

If you want to do anything at all related to email in .NET, take a look at MimeKit and MailKit. I guarantee that you will not be disappointed.

the avatar of Cameron Seader

SUSE Cloud 4 OpenStack Admin Appliance; Updated

I hope all who have been using the appliance are enjoying it and finding it useful. I felt it time to update the appliance to include patches to the latest known threats and critical updates to OpenStack.

Latest version 4.0.3

Changes from Github Project
  • Refreshed the Update Repositories to contain latest patches
  • Applied latest Updates to the Appliance
Direct Download links
Please visit the the landing page for the appliance to get more information and documentation here.
Some future things that are coming to look forward to. 
  • Incorporating an Installation media which includes the latest packages from the update repositories for SLES 11 SP3, High Availability Extension, and SUSE Cloud 4 OpenStack. This Installation media will allow me to exclude the full update repositories on the image and therefore reduce the size of the image to just under 2GB. 
  • Moving the build of the image over to openSUSE OBS (Open Build Service) to allow more rapid deployment and testing.
These things will allow for greater portability of the OpenStack software and inherent with it you can install anywhere. Install on VMware. Install on Virtual Box. Install on KVM. Install on Bare Metal. You can truly use this image to deploy and test it out on VMware or KVM, and from the same image you can use it to deploy a full production OpenStack on Bare Metal.  I have even used it to install and test OpenStack out on AWS. So go forth and enjoy installing OpenStack with ease. I challenge you to start using this appliance and see how easy it can be to setup and run OpenStack software. 

a silhouette of a person's head and shoulders, used as a default avatar

KDE Applications and Platform 4.14.2 available for openSUSE

Following up on KDE’s announcement of the latest stable release, we have now packages available for 12.3 and 13.1 (a 13.2 repository will be made available after it is out). You will find them in the KDE:Current repository. Current users of this repository will get the new release automatically once they update.

Why you should upgrade? You can take a look at the list of changes to get an idea. These fixes touch many important KDE applications, including KMail, Okular and Dolphin.

Packages are also on their way to openSUSE Factory.

As usual, bugs with the packaging should be reported to openSUSE and upstream bugs should be reported to KDE.

Also, if you like what KDE is doing and you feel you can not contribute directly, you may want to support this end of year fundraiser.

a silhouette of a person's head and shoulders, used as a default avatar

Geckos at Beijing

oSA14

Geckos are taking over China, at the openSUSE Asia Summit to be held this coming weekend in Beijing. It is the first time that an openSUSE event of this scale is being held so far east of the Geeko Meridian. The organizing committee has worked their socks off for the event, and things are shaping up well. The schedule is ready and their are some great talks lined up. Overall, it promises to be a great event. As for me, I am going to speak about the Google Summer of Code, openSUSE Activities in India and a workshop on the Qt Framework.So, see you in Beijing 😉

http://summit.opensuse.org

3654543066_2c8823cb03_o-e1363960517132

a silhouette of a person's head and shoulders, used as a default avatar

Libgreattao: custom UI

For this task we need newest version of libgreattao and tao-manager application. You can download tao-manager from my home page. You can download newest version of libgreattao from svn of SourceForge,.


Our configuration window

Firstly, we take care about configuration window. In this example we looks on ready class of configuration dialog belonged to “modern” design.

This window contains buttons on the left side, labels and edits. In standard design are placed tabs instead of buttons. In modern design clicking on the button will switch view.

The code are below:

<root>
<variable name="displayed_path" action="change" value="" />
<hbox>
<vbox>
<template path="/category/*">
  <variable name="blank_path" action="change" value="" />
  <if type="equal" variable1="blank_path" variable2="displayed_path">
     <variable name="displayed_path" action="change" function="path" />
     <variable name="active" action="change" function="path" />
     <variable name="active_directory" action="change" value="/" />
      <variable name="active" action="append" variable="active_directory" />
  </if>
  <button dir-for="label">
    <handler>
       <hide variable="active"  />
       <variable name="active" action="change" function="path" />
       <variable name="active_directory" action="change" value="/" />
       <variable name="active" action="append" variable="active_directory" />
       <show variable="active" />
    </handler>
  </button>
</template>
</vbox>
<vbox>
<template path="/category/*/*/text">
<variable name="current_path" action="change" function="path" />
<hide />
<if type="is_children" variable1="displayed_path" variable2="current_path">
  <show />
</if>
<hbox>
<label dir-for="label" />
<edit dir-for="event,input,output" />
<label dir-for="description" />
</hbox>
</template>
</vbox>
</hbox>
<hbox>
<button label="Ok" dir-for="label,event" path="/actions/ok" />
<button label="Cancel" dir-for="label,event" path="/actions/cancel" />
<button label="Save" dir-for="label,event" path="/actions/save" />
</hbox>
</root>
We can see one handler, few variables, conditions and show/hide elements. Show/hide elements are changing visibility:
  1. For parent element,, if there’s no “name” and “variable” attribute
  2. For elements with given path, if there’s “variable” attribute
  3. For elements with given name, if there’s “name” attribute

What we can do to show/hide elements once button is clicked? After all we must select method(using path or name). We select path and controling elements using children path “/category/*/, because buttons are present in “/category/*” and elemetns to control are in path “/catrgory/*/*”. In first step we hide all elements using “hide” element without attributes for template:

<template path="/category/*/*/text">

This way will hide all elements without category buttons. We must also show firstly created category. This code will complete this task:

<if type="is_children" variable1="displayed_path" variable2="current_path">
   <show />
 </if>
Condition typw “is_children” checks that path are child of other path. In case we need it touches only children of matched element(without parent), we must end value with slash. Jn this case we check if “current_path” are child of “displayed_path”. “current_path” is set above and the value of it is cuirrent path. We set also “active variable”, which points to children fof active element(we add slash at the end, using”append”). Before setting value of variable “displayed_path” we checks if that variable are not s,lash(this variable will contains slash near after view is loaded).
In next step we take care about showing/hidding elements on button’s click event. Our handler will complete this task. In first step, we hide active element using “active” variable. We must remember that firstly created element will be active. In next step we set “active” variable using clicked button’s action path. We also append slash to it. In  next step we show elements belonged to clicked button. “active” variable will contain path of clicked element, so after clicked another button displayed elements(without buttons) will be hide.
This is the end of describing configuration window. There’s only one matter. Once application delete active elements, there’s no elements visible(buttons are only visible). Other situation is with tabs. There’s framework will display another tab automatically.
Wygląd naszego okna konfiguracji

Custom file management window for tao-manager

Wygląd menadżera plików bez lewego panelu
Next example are about adding file information dialog for my file manager. We will change view of file manager windolw, but only for this one application. Firstly, we need to create directory “$HOME/.tao”. In this directory we need to create directory “$HOME/.tao/tao-manager” and “$HOME/.tao/tao-manager/mWidgets” and “$HOME/.tao/tao-manager/mWidgets/app”. In next step we need to create file with UI paths definitions(“$HOME/.tao/app”). In this directory we will create UI defintion directory for my file manager:
tao-manger = tao-manger
We must now copy configuration file for mWidgets design:
cp /usr/share/tao-design/mWidgets/config ~/.tao/tao-manager/mWidgets
We must now create file called “file manager” in “$HOME/.tao/tao-manager/mWidgets/app” directory. This file should looks like below:
<root>
<menu>
<template path="/actions/*">
<submenu dir-for="label">
<attr-connect name="label" function="last-dir" />
<template path="*">
<item dir-for="label,event">
<icon dir-for="icon" width="8" height="8" />
<attr-connect name="label" function="last-dir" />
</item>
</template>
</submenu>
</template>
</menu>
<hbox>
<vbox>
<template path="/fileviews/*/list/items/*">
<variable name="name" action="change" function="path" />
<variable name="name" action="append" value="/name" />
<attr-connect name="widget-name" variable="name" />
<template path="data/*">
<hbox>
  <label>
   <attr-connect name="label" function="last-dir" />'
  </label>
  <label dir-for="label" />
</hbox>
</template>
<hide />
</template>
</vbox>
<template path="/fileviews/*">
<vbox>
<template path="path_edit">
<edit dir-for="input,output,event" />
</template>
<list>
<template path="list/columns/*">
  <column>
    <attr-connect name="name" function="last-dir" />
    <attr-connect name="label" function="last-dir" />
  </column>
</template>
<template path="list/items/*">
  <selection>
    <template path="selected">
       <handler>
          <hide name="path" />
          <variable name="path" action="change" function="remove_last_dir" />
          <variable name="path" action="append" value="/name" />
          <variable name="call-path" action="change" function="path" />
          <show name="path" />
          <call var="call-path" />
       </handler>
    </template>
    <template path="data/*">
       <data dir-for="label">
         <attr-connect name="column-name" function="last-dir" />
         <attr-connect name="label" function="last-dir" />
       </data>
    </template>
  </selection>
</template>
</list>
</vbox>
</template>
</hbox>
</root>
Our information panel are definied in template with path pattern “/fileviews/*/list/items/*. In this step I must mentoin about widgets names. Many widgets can have the same name and one widget can have many names. Name are used to control visibility of widget, but in future it will be used in other tasks. We can define widget’s name usaing “attr-connect” tag with “name” attribute equal to “widget-name” and giving variable name. Our name is actions’s path with appented string “/name”. Why we don’t use action’s paths? We are using the same path for elements in lists, so using name could causing list’s elements to show/hide. I must mentoin, that displaying/hidding elements of lists are not implemented, but maybe it will be implemented in future.
Described tempalte contains template with relative path pattern “data/*”. In this pattern exist label with text equal to last directory of path to this action. There’s also label connected to value of current attribute of active file. All attributes of file are hidden by default.
Take also look at only one handler in this file. It exist in template with relative path “selected”, “selected” is event invoked, when a file is selected. “path” variable point to active file. It is empty by default, so nothink will be hidden.  This handler first will hide element(vbox for file attribute). In next step it will get path without last element(“/selected”) and next append value “/name” for it. This is way of generation path of element to show. “call-path” variable contains path pointed to “selected” action. This variable is used to call event connected with this action,.
Wynik naszych prac
There’s only one problem. Names of file attributes in left panel are not set by application.
 We use names obtained from path of template(last directory). It can be solved in very simplem way. You must only replace first code bellow with second code below:
1)
<hbox>
<label>
<attr-connect name=”label” function=”last-dir” />
</label>
<label dir-for=”label” />
</hbox>
2)
<hbox>
<variable name=”column-path” action=”change” value=”/fileviews/” />
<variable name=”column-path” action=”append” function=”nth-dir” param=”2″ />
<variable name=”column-path” action=”append” value=”/list/columns/” />
<variable name=”column-path” action=”append” function=”last-dir” />
<label>
<attr-connect name=”label” function=”last-dir” />
<attr-connect name=”label” function=”label” param=”column-path” />’
</label>
<label dir-for=”label” />
</hbox>

the avatar of Sankar P

Technology Catchup

Coincidentally three different people asked me in the last month, to write about new technologies that they should be knowing, to make them more eligible to get a job in a startup. All these people have been C/C++ programmers, in big established companies, for about a decade now. Some of them have had only glimpses of any modern technologies.

I have tried a little bit (with moderate success) to work in all layers of programming with most of the popular modern technologies, by writing little-more-than-trivial programs (long before I heard of the fancy title "full stack developer"). So here I am writing a "technology catchup" post, hoping that it may be useful for some people, who want to know what has happened in the technologies in the last decade or so.

Disclaimer 1: The opinions expressed are totally biased as per my opinion. You should work with the individual technologies to know their true merits.

Disclaimer 2: Instead of learning everything, I personally recommend people to pick whatever they feel they are connected to. I, for example, could not feel connected to node-js even after toying with it for a while, but fell in love with Go. Tastes differ and nothing is inferior. So give everything a good try and pick your choice. Also remember what Donald Knuth said, "There is difference between knowing the name of something and knowing something". So learn deeply.

Disclaimer 3: From whatever I have observed, getting hired in a startup is more about being in the right circles of connection, than being a technology expert. A surprisingly large number of startups start with familiar technology than with the right technology, and then change their technology, once the company is established.

Disclaimer 4: This is actually not a complete list of things one should know. These are just things that I have come across and experimented a little bit at least. There are a lot more interesting things that I would have have missed. If you need something must have been in the list, please comment :-)

With those disclaimers away, let us cut to the chase.


Version Control Systems

The most prominent change in the open source arena, in the last decade or so, is the invention of Git. It is a version controlled system initially designed for keeping the kernel sources and has since then become the de-facto VCS for most modern companies and projects.

Github is a website that allows people to host their open source projects. Often startups recruit people based on their github profile. Even big companies like microsoft, google, facebook, twitter, dropbox etc. have their own github accounts. I personally have received more job queries through my github projects than via my linkedin profile in the last year.

bitbucket is another site that allows people to host code and give even private repos. A lot of the startups that I know of use this, along with the jira project management software. This is your equivalent of MS Project in some sense.

I have observed that most of the startups founded by people who come from Banking or Finance companies to be using Subversion. Git is the choice for people from tech companies though. Mercurial is another open source, distributed VCS which has lost a lot of limelight in the recent times, due to Git. Fossil is another VCS, from the author of sqlite, Dr. Richard Hipp. If you can learn only one VCS for now, start with Git.

Programming Languages & Frameworks

Javascript has evolved to be a leading programming language of the last decade. It is even referred to as the X86 of the web. From its humble beginnings as a client-side scripting language to validate if the user has typed a number or text, it has grown into a behemoth and entered even the server-side programming through the node-js framework. For incorporating ModelViewController pattern, javascript has gained the AngularJS framework. JS is a dynamically typed language and to bring in some statically typed langauges' goodness, we have a coffeescript language too.

Python is another dynamically typed, interpreted programming language. Personally, I felt that it is a lot more tasteful than Javascript. It feels good on eyes too. It helps in rapid application development and is available by default in almost all the Linux distros and Mac machines by default. Django is a web framework that is built on python to make it easy to develop web applications. In addition to being used in a lot of startups, it is used in even big companies like Google and Dropbox. There are variants of Python runtime such that you can run it in the JVM using Jython or in the .NET CLR using the IronPython. I have personally found this language to be lacking in performance though, which is elaborated more in a subsequent section.

Ruby is an old programming language that shot into fame in the recent years through the popular web application framework Ruby on Rails, often called just Rails. I have learnt a lot of engineering philosophies such as DRY, COO etc. while learning RoR.

All these above languages and frameworks use a package manager such as npmBower, pip, gems etc. to install libraries easily.

Go is my personal favorite in the new languages to learn. I see Go becoming as vital and prominent a programming language as C, C++ or Java in the next decade. It is developed in Google for creating large scale systems. It is a statically-typed, automatic-memory-managed language that generates native-machine-code and helps writing concurrent-code easily.

Go is the default language that I use for any programming task in the last year or so. It is amazingly fast even though (just because?) it is still in the 1.X series. In my dayjob we did a prototype in both go and python, and for a highly concurrent workflow in the same hardware, Go puffed Python in performance (20 seconds vs 5 minutes). I won't be surprised if a lot of the python and ruby code gets converted to golang in their next edition of rewrites. Personally, I have found the quality of go libraries to be much higher compared to Ruby or nodejs as well, probably because not everyone has adapted to this language yet. However, this could be just my personal biased opinion.

If you like to get fancy with functional programming, then you can learn Scala (on top of JVM), F# (on top of .NET), Haskell, Erlang, etc. The last two are very old btw but in use even today. Most recently, Whatsapp was known to use Erlang. D is also seen in the news, mostly thanks to Facebook. Dart is another language that is from Google but still to receive any wide deployment afaik, even with Google's massive marketing machinery behind it. It has been compared to VBscript and is criticized, and as of now chrome-only. Dart has received criticism from Mozilla, Webkit (rendering engine that powers Safari (and chrome earlier)), Microsoft IE as well. Dart is done by Lars Bak et al. (the people who gave us V8, chrome's Javascript engine)

Rust is another programming language that is aimed for high-performance concurrent systems. But I have not played around with it, as they don't maintain a stable API and they are not 1.0 yet. Julia is another programming language aimed at doing distributed systems, about which I have heard a lot of praise, but it still remains a exotic language afaik. R is another language which I have seen in a lot of corporate demos where the presenters wanted to show statistics, charts. Learning this may be useful even if you are not a programmer and works with numbers (like a project manager).


There is a Swift programming language from Apple to write iOS apps. I have not tried Swift yet, but from my experience of using Objective C, it cannot be worse.

Bootstrap is a nice web framework from twitter, which provides various GUI elements that you can incorporate into your application, to rapidly prototype beautiful applications, that are fluidic even when viewed in mobile.

jquery is a popular javascript library that is ubiquitous. Cascading Style Sheets (shortly CSS) is a markup language that helps configure the style of the web page UI elements. CSS is becoming mature to the extent of showing animations too. You should ideally spend a few weeks to learn about HTML5 and CSS.

Text Editors

Sublimetext is what the cool kids use these days as the editor. I have found the tutorial on tutsplus to be extra-ordinarily good at explaining sublime. It is a free (as in beer) software and not open source.

Atom is a text-editor from github built using nodejs and chromium. I did not find a linux binary and so did not bother to investigate it. But I have heard it to be good for Javascript programmers than any others, as the editor could be extended by javascript itself.

Brackets is another editor that I have heard good things about. Lime is an editor that is developed in Go, aimed to be an open-source replacement for the sublimetext.

Personally, after trying various text editors, I have always comeback to using vim. There are a few good plugins for vim in the recent times. Vundle, Pathogen are nice plugin managers for vim to ease up installation of plugins. YouCompleteMe is a nice plugin for auto-completion. vim-spf13 is a nice distro of vim, where various plugins and colorschemes are pre-packaged.

Distributed Computing

In the modern day of computing, most programs have been driven by a Service Oriented Architecture (shortly SOA). Webservices are the preferred way of communication among servers as well. While we are talking about services, please read this nice piece by Steve Yegge.

memcached is a distributed (across multiple machines), caching system which can be used in front of your database. This was initially developed by Brad Fritzpatrick, while he was the head of the LiveJournal and who is now (2014) a member of the Go team at Google. While at Google, he has started GroupCache which as the project page says is a replacement for memcache in many cases.

GoogleFileSystem (GFS) is a seminal paper on how Google created a filesystem to suit their large needs of data processing. There is a database built on top of this filesystem named BigTable which powered Google's infrastructure. Apache Hadoop is an open source implementation of these concepts, which was originally started in Yahoo and now a top-level apache project. HDFS  is the equivalent of GFS for the Hadoop. Hive and Pig are technologies to query and analyze data from the Hadoop.

As with the evolution of any software, GFS has evolved into a Colossus filesystem and BigTable has evolved into a Spanner distributed database. I recommend you to read these papers even if you are not going to do any distributed computing development.

Cassandra is another distributed database which was started in Facebook initially, but is used in many companies such as Netflix and Twitter. I have used Cassandra more than any other distributed project and actually like it a lot. It uses a SQL like query language called CQL - Cassandra Query Language. It is modelled after the DynamoDB paper from Amazon. I am too tempted to write an alternative to this in Go, just to have the idea of writing a large scale distributed system, instead of just using it as a client, but have not got around to a good dataset or usecase with which I can test it.

MongoDB is another document oriented database, which I tried using for a pet project of mine. I don't remember exactly but there were some problems with respect to unicode handling. The project was done prior to go becoming 1.0, so the problem could be in any end.

Most of the new age databases are called NOSQL databases but what they really mean is that the database skips a lot of functions (such as datatype validation, stored procedures, etc.) and try to grow by scaling out instead of scaling up.

Cloud

OpenStack is a suite of open source projects that help you create a private cloud. DeltaCloud is a project which was initially started by RedHat, and now an apache top-level project, as a way to provide a single API layer which will work across any cloud in the backend. This project is done in ruby. I was initially interested in participating in its development, until I got introduced to Go and moved into a different tangent.

To start off a software company is a very easy task to do in today's world. The public clouds are becoming cheaper and cheaper everyday and their capacity can be provisioned instantly.

Amazon web services provides an umbrella of various public cloud offerings. I have used Amazon EC2 which is a way to create a Linux (and windows) VM that runs on Amazon's datacenters. The machines come on various sizes. Amazon S3 is a cloud offering that provides you way to store data in buckets. This is used by Dropbox heavily for storing all your data. There are various other services too. In some of our prototyping, we found the performance of Amazon EC2, to be consistent mostly, even in the free tier.

Google is not lagging behind with their cloud offerings either. When Google Reader was shut down, I used Google's Appengine to deploy an alternative FOSS product and I was blown away by the simplicity of creating applications on top of it. Google Compute is the way to get VMs running on the Google Cloud. As with Amazon, there are plenty of other services too.

There are plenty of other players like Microsoft Azure, Heroku etc. but I do not have any experience with their applications. While we are talking about Cloud, you should probably read about Orchestration and know about at least Zookeeper.

In-Process Databases

These are databases which you can embed into your application, without needing a dedicated server. They run on your process-space.

sqlite is the world's most deployed software and it competes with fopen to become the default way to store data for your desktop applications (if you are still writing them ;) ). A new branch is coming with the latest rage on storage datastructures, a log-structured merge tree as well.

leveldb is a database that is written by the eminent Googlers (and trendsetters of technology in the last decade or so) Jeff Dean and Sanjay Ghemawat who gave us MapReduce, GFS etc. It is forked by Facebook into RocksDB as well.

KyotoCabinet and LMDB are other projects on this space.

Linux Filesystems

Since we have covered GFS, HDFS, etc. earlier. We will look at other popular filesystems.

btrfs is a copy-on-write filesystem in Linux. It is intended to be the defacto linux filesystem in the future, possibly obsoleting ext series in the longer run.

XFS is a filesystem that initially came from SGI to Linux. This is my personal favorite and I have been using it on all my linux machines. In addition to good performance, this offers robustness and comes with a load of features that are useful to me, like defragmentation.

We also have the big daddy of filesystems zfs too on linux.

Ceph is another interesting distributed filesystem that works on the kernel space and is already merged in the linux kernel sources for a long time now. GlusterFS is another distributed filesystem which works in the userspace. Both of these filesystems focus on scaling out instead of scaling up.

Conclusion

Pick any of these technologies that you like and start writing a toy application on it, may be as simple as a ToDo application and learn through all the stages. This approach has helped me. It may help you also.

I have written this post from a Thinkpad T430 running openSUSE Factory and GNOME Shell with a bunch of KDE tools. I like this machine, However, in the past few months I have realized that, in today's world, If you are a developer, it is best if you run Linux on your server and Mac on your laptop.