The SUSE OpenSSL maintainers are hard at work to migrate openSUSE Tumbleweed and SUSE Linux Enterprise 15 to use OpenSSL 1.1 by default. Some work remains, see boo#1042629.

Here is how you can use OpenSSL 1.1 in parallel to the 1.0 version on openSUSE Leap for development and run-time:

zypper ar --refresh --check obs://security:tls security_tls
zypper in --from security_tls openssl libopenssl-devel

This replaces the following:

The following 3 NEW packages are going to be installed:
libopenssl-1_1_0-devel libopenssl1_1_0 openssl-1_1_0

The following 2 packages are going to be upgraded:
libopenssl-devel openssl

The following 2 packages are going to change architecture:
libopenssl-devel x86_64 -> noarch
openssl x86_64 -> noarch

The following 2 packages are going to change vendor:
libopenssl-devel openSUSE -> obs://build.opensuse.org/security
openssl openSUSE -> obs://build.opensuse.org/security

This will retain a working OpenSSL 1.0 run-time, and the two versions can be used in parallel. As you can see, the openssl and libopenssl-devel turn into non-architecture specific meta-packages.

Verify the installed command line version:

openssl version
OpenSSL 1.1.0f-fips 25 May 2017

To revert:

zypper in --from update --oldpackage openssl -libopenssl1_1_0

Accept the default conflict resolutions to switch back to the distribution packages.

Have the best of success porting your code to support OpenSSL 1.1. The upstream wiki has a good article on the OpenSSL 1.1 API changes.

Rust tries to follow the "make illegal states unrepresentable" mantra in several ways. In this post I'll show several things related to the process of building strings, from bytes in memory, or from a file, or from char * things passed from C.

Strings in Rust

The easiest way to build a string is to do it directly at compile time:

let my_string = "Hello, world!";

In Rust, strings are UTF-8. Here, the compiler checks our string literal is valid UTF-8. If we try to be sneaky and insert an invalid character...

let my_string = "Hello \xf0";

We get a compiler error:

error: this form of character escape may only be used with characters in the range [\x00-\x7f]
 --> foo.rs:2:30
  |
2 |     let my_string = "Hello \xf0";
  |                              ^^

Rust strings know their length, unlike C strings. They can contain a nul character in the middle, because they don't need a nul terminator at the end.

let my_string = "Hello \x00 zero";
println!("{}", my_string);

The output is what you expect:

$ ./foo | hexdump -C
00000000  48 65 6c 6c 6f 20 00 20  7a 65 72 6f 0a           |Hello . zero.|
0000000d                    ^ note the nul char here
$

So, to summarize, in Rust:

• Strings are encoded in UTF-8
• Strings know their length
• Strings can have nul chars in the middle

This is a bit different from C:

• Strings don't exist!

Okay, just kidding. In C:

• A lot of software has standardized on UTF-8.
• Strings don't know their length - a char * is a raw pointer to the beginning of the string.
• Strings conventionally have a nul terminator, that is, a zero byte that marks the end of the string. Therefore, you can't have nul characters in the middle of strings.

Building a string from bytes

Let's say you have an array of bytes and want to make a string from them. Rust won't let you just cast the array, like C would. First you need to do UTF-8 validation. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

fn convert_and_print(bytes: Vec<u8>) {
    let result = String::from_utf8(bytes);
    match result {
        Ok(string) => println!("{}", string),
        Err(e) => println!("{:?}", e)
    }
}

fn main() {
    convert_and_print(vec![0x48, 0x65, 0x6c, 0x6c, 0x6f]);
    convert_and_print(vec![0x48, 0x65, 0xf0, 0x6c, 0x6c, 0x6f]);
}

In lines 10 and 11, we call convert_and_print() with different arrays of bytes; the first one is valid UTF-8, and the second one isn't.

Line 2 calls String::from_utf8(), which returns a Result, i.e. something with a success value or an error. In lines 3-5 we unpack this Result. If it's Ok, we print the converted string, which has been validated for UTF-8. Otherwise, we print the debug representation of the error.

The program prints the following:

$ ~/foo
Hello
FromUtf8Error { bytes: [72, 101, 240, 108, 108, 111], error: Utf8Error { valid_up_to: 2, error_len: Some(1) } }

Here, in the error case, the Utf8Error tells us that the bytes are UTF-8 and are valid_up_to index 2; that is the first problematic index. We also get some extra information which lets the program know if the problematic sequence was incomplete and truncated at the end of the byte array, or if it's complete and in the middle.

And for a "just make this printable, pls" API? We can use String::from_utf8_lossy(), which replaces invalid UTF-8 sequences with U+FFFD REPLACEMENT CHARACTER:

fn convert_and_print(bytes: Vec<u8>) {
    let string = String::from_utf8_lossy(&bytes);
    println!("{}", string);
}

fn main() {
    convert_and_print(vec![0x48, 0x65, 0x6c, 0x6c, 0x6f]);
    convert_and_print(vec![0x48, 0x65, 0xf0, 0x6c, 0x6c, 0x6f]);
}

This prints the following:

$ ~/foo
Hello
He�llo

Reading from files into strings

Now, let's assume you want to read chunks of a file and put them into strings. Let's go from the low-level parts up to the high level "read a line" API.

Single bytes and single UTF-8 characters

When you open a File, you get an object that implements the Read trait. In addition to the usual "read me some bytes" method, it can also give you back an iterator over bytes, or an iterator over UTF-8 characters.

The Read.bytes() method gives you back a Bytes iterator, whose next() method returns Result<u8, io::Error>. When you ask the iterator for its next item, that Result means you'll get a byte out of it successfully, or an I/O error.

In contrast, the Read.chars() method gives you back a Chars iterator, and its next() method returns Result<char, CharsError>, not io::Error. This extended CharsError has a NotUtf8 case, which you get back when next() tries to read the next UTF-8 sequence from the file and the file has invalid data. CharsError also has a case for normal I/O errors.

Reading lines

While you could build a UTF-8 string one character at a time, there are more efficient ways to do it.

You can create a BufReader, a buffered reader, out of anything that implements the Read trait. BufReader has a convenient read_line() method, to which you pass a mutable String and it returns a Result<usize, io::Error> with either the number of bytes read, or an error.

That method is declared in the BufRead trait, which BufReader implements. Why the separation? Because other concrete structs also implement BufRead, such as Cursor — a nice wrapper that lets you use a vector of bytes like an I/O Read or Write implementation, similar to GMemoryInputStream.

If you prefer an iterator rather than the read_line() function, BufRead also gives you a lines() method, which gives you back a Lines iterator.

In both cases — the read_line() method or the Lines iterator, the error that you can get back can be of ErrorKind::InvalidData, which indicates that there was an invalid UTF-8 sequence in the line to be read. It can also be a normal I/O error, of course.

Summary so far

There is no way to build a String, or a &str slice, from invalid UTF-8 data. All the methods that let you turn bytes into string-like things perform validation, and return a Result to let you know if your bytes validated correctly.

The exceptions are in the unsafe methods, like String::from_utf8_unchecked(). You should really only use them if you are absolutely sure that your bytes were validated as UTF-8 beforehand.

There is no way to bring in data from a file (or anything file-like, that implements the Read trait) and turn it into a String without going through functions that do UTF-8 validation. There is not an unsafe "read a line" API without validation — you would have to build one yourself, but the I/O hit is probably going to be slower than validating data in memory, anyway, so you may as well validate.

C strings and Rust

For unfortunate historical reasons, C flings around char * to mean different things. In the context of Glib, it can mean

• A valid, nul-terminated UTF-8 sequence of bytes (a "normal string")
• A nul-terminated file path, which has no meaningful encoding
• A nul-terminated sequence of bytes, not validated as UTF-8.

What a particular char * means depends on which API you got it from.

Bringing a string from C to Rust

From Rust's viewpoint, getting a raw char * from C (a "*const c_char" in Rust parlance) means that it gets a pointer to a buffer of unknown length.

Now, that may not be entirely accurate:

• You may indeed only have a pointer to a buffer of unknown length
• You may have a pointer to a buffer, and also know its length (i.e. the offset at which the nul terminator is)

The Rust standard library provides a CStr object, which means, "I have a pointer to an array of bytes, and I know its length, and I know the last byte is a nul".

CStr provides an unsafe from_ptr() constructor which takes a raw pointer, and walks the memory to which it points until it finds a nul byte. You must give it a valid pointer, and you had better guarantee that there is a nul terminator, or CStr will walk until the end of your process' address space looking for one.

Alternatively, if you know the length of your byte array, and you know that it has a nul byte at the end, you can call CStr::from_bytes_with_nul(). You pass it a &[u8] slice; the function will check that a) the last byte in that slice is indeed a nul, and b) there are no nul bytes in the middle.

The unsafe version of this last function is unsafe CStr::from_bytes_with_nul_unchecked(): it also takes an &[u8] slice, but you must guarantee that the last byte is a nul and that there are no nul bytes in the middle.

I really like that the Rust documentation tells you when functions are not "instantaneous" and must instead walks arrays, like to do validation or to look for the nul terminator above.

Turning a CStr into a string-like

Now, the above indicates that a CStr is a nul-terminated array of bytes. We have no idea what the bytes inside look like; we just know that they don't contain any other nul bytes.

There is a CStr::to_str() method, which returns a Result<&str, Utf8Error>. It performs UTF-8 validation on the array of bytes. If the array is valid, the function just returns a slice of the validated bytes minus the nul terminator (i.e. just what you expect for a Rust string slice). Otherwise, it returns an Utf8Error with the details like we discussed before.

There is also CStr::to_string_lossy() which does the replacement of invalid UTF-8 sequences like we discussed before.

Conclusion

Strings in Rust are UTF-8 encoded, they know their length, and they can have nul bytes in the middle.

To build a string from raw bytes, you must go through functions that do UTF-8 validation and tell you if it failed. There are unsafe functions that let you skip validation, but then of course you are on your own.

The low-level functions which read data from files operate on bytes. On top of those, there are convenience functions to read validated UTF-8 characters, lines, etc. All of these tell you when there was invalid UTF-8 or an I/O error.

Rust lets you wrap a raw char * that you got from C into something that can later be validated and turned into a string. Anything that manipulates a raw pointer is unsafe; this includes the "wrap me this pointer into a C string abstraction" API, and the "build me an array of bytes from this raw pointer" API. Later, you can validate those as UTF-8 and build real Rust strings — or know if the validation failed.

Rust builds these little "corridors" through the API so that illegal states are unrepresentable.

For some tasks like compiling and QA tests I was looking for some more powerful additions to my SBC collection of RPI3 and Allwinner Pine64. Requirements: Aarch64, plenty of RAM, fast CPU, fast network, mainline kernel support (in a not to distant future).

Currently, many SBCs available in the 100 $US/€ price range only have 1 or 2 GByte of RAM (or even less) and often provide just 4 Cortex-A53 cores running at 1.2-1.4 GHz. There are some exceptions, at least regarding the CPU power, but still limited memory (some examples):

• BananaPi M3 (85€)
  • Pros: 8 Cores @ 1.8 GHz (Allwinner A83T), GBit Ethernet, 8 GByte eMMC
  • Cons: ARM A7 (32bit), 2 GByte RAM, only 2x USB 2.0, PowerVR graphics
• LeMaker HiKey 620 (109 $US)
  • Pros: 8 Cores @ 1.2 GHz (Kirin 620), 64bit (A53) 8 GByte eMMC
  • Cons: No ethernet, 2 GByte RAM, only 3x USB 2.0, discontinued
• Asus Tinkerboard (65€)
  • Pros: 4 Cores @ 2.0 GHz (Rockchip RK3288), GBit Ethernet
  • Cons: ARM A17 (32bit), 2GByte RAM, USB 2.0 only (4x)

There are more expensive options like the LeMaker HiKey 960 (240€, 3 GByte RAM, still no ethernet), or the Firefly RK3399 (200€, 4 GByte RAM), but these where clearly out of the 100€ price range.

The SBC board all sport a considerable number of of GPIOs, SPI and I2C ports and the like, but for hardware connectivity I already have the above mentioned bunch of RPIs (1 and 3) and Pine64 boards. So going into a slightly different direction, I investigated some current settop boxes.

Rockchip RK3399

The Rockchip RK3399 used on the Firefly is one of the available options, itt powers several different STBs with different RAM and eMMC configurations. I went with the the Yundoo Y8, for the following reasons:

• 4 GByte of RAM
• 32 GByte of eMMC
• 2 A72 Cores (2.0 GHz) + 4 A53 Cores (1.5 GHz)
• 1x GBit Ethernet
• 1x USB 3.0, standard A type receptable
• 1x USB 3.0, Type C (also provides DisplayPort video using alternate mode)
• 2x USB 2.0, Type A
• available for 105€, including EU power supply, IR Remote, short HDMI cable

Contrary to the above mentioned Firefly, you neither get PCIe expansion ports nor any GPIOs, so if this is a requirement the Y8 is no option for you.

In the next part, I will give some more details useful for running stock Linux instead of the provided Android 6.0 on the device – so stay tuned for UART info, MaskRom mode etc.

Update: UART/MaskRom info is now available: Yundoo Y8 RK3399 Settop Box (Part 2)

A good question about TDD with game development.

Really it depends what you mean by game development. If we are talking about creating a game engine then TDD absolutely has a place in the development process. It can be used like it is in any other application where you write tests using whatever framework is appropriate to express the business logic in logical chunks (individual tests that test a single requirement, not multiple requirements at once, this makes it easier to troubleshoot when a test fails). As far as I know a game engine is not much different from any other software so following TDD principles should be no different.

If however we are just talking about creating a game from an existing engine then things might be a bit tricky. A quick search shows that Unreal Engine supports some form of testing but I am not sure how easy to use it is. Another issue with writing tests for games is how can you express the logic of the game in a test? Sure, you can test things like the math or physics libraries, but what about more complex functionality? Overall I don’t think it would be worth trying to test games given the large amount of possible use cases, it would likely be too much of an effort to test the game as a whole. That however does not mean you should not test individual libraries/modules!

I am curious to know what developers who actually do game development use for doing their tests and what exactly they try to test when it comes to game functionality.

A good question about TDD with game development. was originally published in Information & Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.

During GUADEC this year I gave a presentation called Replacing C library code with Rust: what I learned with librsvg. This is the PDF file; be sure to scroll past the full-page presentation pages until you reach the speaker's notes, especially for the code sections!

You can also get the ODP file for the presentation. This is released under a CC-BY-SA license.

For the presentation, my daughter Luciana made some drawings of Ferris, the Rust mascot, also released under the same license:

As you may already know (if you don’t, please check these older posts) openQA, the automated testing system used by openSUSE runs daily tests on the latest KDE software from git. It works well and uncovered a number of bugs. However, it only tested X11. With Wayland starting to become usable, and some developers even switching to Wayland full time, it was a notable shortcoming. Until now.

Why would openQA not run on Wayland? The reason lies in the video driver. openQA runs its tests in a virtual machine (KVM) which uses the “cirrus” virtual hardware for video display. This virtualized video card does not expose an interface capable of interfacing with the kernel’s Direct Rendering Manager (DRM), which is required by kwin_wayland, causing a crash. To fix it, “virtio”, which presents a way for the guest OS to properly interface with the host’s hardware, is needed. Virtio can be used to emulate many parts of the VM’s hardware, including the video card. That would mean a working DRM interface, and a working Wayland session!

However, openQA tests did not handle the case where virtio was needed. This is where one of our previously-sung heroes, Fabian Vogt, comes into play. He added support for running Wayland tests in openQA using virtio. This means now that the Argon and Krypton live images now undergo an additional series of tests which are performed under Wayland. And of course, actual bugs have been found already.

Last but not least, now the groundwork has been laid down to do proper Wayland testing on vanilla Tumbleweed as well. openQA should be probably churning out results soon. Exciting times ahead!

This post has been migrated to my new blog that you can find here:

https://pureooze.com/blog/posts/2017-08-06-test-driven-development-in-javafx-with-testfx/

A dive into how to test JavaFX applications using TestFX.

If you just want to read about how to use TestFX skip to the “Setting Up Our Application” section.

I recently started working at a company that has a very heavy reliance on Java applications. As a primarily Python, Javascript and C++ developer, I had not used Java for a few years so given the obvious fact that I would have to get comfortable with using Java on a day to day basis, I decided to work on a small project in Java to get my feet wet.

The Application

While thinking about possible project ideas for this Java application I browsed through my Github repos and saw an all too familiar project, a podcast feed aggregator/player that I had worked on with some friends. The application was written using Qt 5.8 and was capable of fetching RSS feeds for any audio podcast on iTunes, stream episodes and render the HTML contents associated to an individual episode. The links shown below in the right hand side widget would also open in the default system browser when clicked on by a user.

While the application looks pretty simple (and it is) the fun of developing this came from working with two friends to learn Qt while also developing the application quickly and in a small amount of time. The reason I considered rewriting this application in Java was because the code we originally wrote had several flaws (keep in mind we developed it in a very short amount of time):

• We wrote zero tests for the application (Yikes!!)
• Classes were an after thought, they were not part of the initial design
• Due to the above the functions were very tightly coupled
• Tight Coupling made it very difficult to modify core processes of the application (like XML parsing) without breaking other features

The Approach — TDD

Given the flaws mentioned above it became clear to me that I would need to take a different approach to this new application if I wanted any hope of being able to maintain it for any reasonable amount of time. For help with how to approach this application design I turned to a book I had been reading recently: Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce.

This book is a fantastic read if you want to learn about what makes Test Driven Development a powerful tool for use in Object Oriented development. The authors provide a lot of deep insight on why tests are important, what kind of tests should be used when and how one writes expressive, easy to understand tests. I will spare the details on TDD but you can google it or read the book above to learn more about it.

The authors of the book continuously stress the importance of using End to End tests because of their ability to test the full development, integration and deployment processes in an organization. I myself do not have automated build systems such as Atlassian’s Bamboo system so my automation capabilities are limited, but I can make use of Unit tests to test individual functions and UI Tests to try and get as much End to End coverage (between the local application and the podcast data on the Internet) as possible.

To read the rest of this post it can be found on my new blog here:

https://pureooze.com/blog/posts/2017-08-06-test-driven-development-in-javafx-with-testfx/

Test Driven Development In JavaFX With TestFX was originally published in Information & Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.

Following up on my first DIY Net Player Post on this blog, I like to present another player that I recently built.

[caption id=“teakear1” align=“alignnone” width=“800”] TeakEar media player[/caption]

This is a Raspberry Pi based audiophile net player that decodes my mp3 collection and net radio to my Linn amplifier. It is called TeakEar, because it’s main corpus is made from teak wood. Obviously I do not want to waste rain forest trees just because of my funny ideas, the teak wood used here has been a table from the 1970ies, back when nobody cared about rainforests. I had the chance to safe parts of the table when it was sorted out, and now use it’s valuable wood for special things.

Upcycling is the idea here, and the central iron part has been a drawbar part from an old hay cart. The black central stone piece is schist, which I found outside in the fields, probably a roofer material.

[caption id=“attachment_902” align=“alignright” width=“220”] Backside view[/caption]

I wanted to combine these interesting and unique materials to something that stands for it’s own. And even though it covers a high tech sound device, I decided against any displays, lights or switches to not taint the warm and natural vibes of the heavy, dark and structured exotic wood, the rusty iron part, which obviously is the hand work of a blacksmith many years ago, and the black stone. Placed on my old school Linn Classic amplifier, it is an interesting object in my living room.

On it’s backside it has a steel sheet fitted into the wood body, held in place by two magnets to allow it to be quickly detached. On that, a Raspberry Pi 2 with a Hifiberry DAC is mounted. The Hifiberry DAC has a chinch output which is connected to the AUX input of the Linn Classic Amplifier. The energy supply is an ordinary external plug in power supply, something that could be improved.

On the software side, I use Volumio, the open audiophile music player. The Volumio project could not amaze me more. After a big redesign and -implementation last year, it is back with higher quality than before but still the same clear focus on music playing. It does nothing else, but with all features and extension points that are useful and needed.

[gallery ids=“921,920,922,923,924” type=“rectangular”]

I enjoy the hours in my workshop building these kind of devices, and I am sure this wont be the last one. Maybe next time with a bit more visible tech stuff.