Arduino para Niños
Legacy Systems as Old Cities
I just realized that I only tweeted about this a couple of months ago, but never blogged about it. Shame on me!
I wrote an article, Legacy Systems as Old Cities, for The Recompiler magazine. Is GNOME, now at 20 years old, legacy software? Is it different from mainframe software because "everyone" can change it? Does long-lived software have the same patterns of change as cities and physical artifacts? Can we learn from the building trades and urbanism for maintaining software in the long term? Could we turn legacy software into a good legacy?
You can read the article here.
Also, let me take this opportunity to recommend The Recompiler magazine. It is the most enjoyable technical publication I read. Their podcast is also excellent!
Update 2017/06/10 - Spanish version of the article, Los Sistemas Heredados como Ciudades Viejas
openSUSE Conference 2017
В конце прошлого месяца разработчики openSUSE снова собрались в Нюрнберге, чтобы обсудить дальнейшее развитие дистрибутива и просто пообщаться и весело провести время вместе. В последние пару лет проект стал очень быстро меняться. К проекту присоединилось очень много новых людей. Новые идеи, которые они приносят, находят свое место в новых проектах, разрабатываемых для openSUSE. Даже используя все информационные каналы проекта openSUSE, тяжело уследить за всеми новшествами, включаемыми в проект, и просто меняющейся стратегией совета. Поэтому вопрос о посещении конференции, тем более, если вы мейнтейнер, долгого размышления не потребует 

Конференция была открыта докладом Матиаса о LiMux. Краткая история проекта, его взлет, надежды и его падение… Явный пример того, на сколько опасны могут быть политики, пользующиеся властью для удовлетворения собственных предпочтений, и закрывающие глаза на интересы и желания населения. В настоящий момент я как раз работаю над этим проектом в администриции Мюнхена.
На конференции я снова встретился с Дмитрием, занимающимся проектом invis и продвижением Free Software решений на базе 1C в Германии. Беседовали о русскоязычном сообществе openSUSE, и о причинах его столь плачевного состояния. Я очень надеюсь, что сообществу получится преодалеть существующие проблемы и оно снова возьмет курс на развитие, как было в 2008-2010 годах.
Познакомился с Денисом Кондратенко из Киева и его женой. Очень приятная пара. Денис рассказывал о Ceph и EKG в openSUSE, а также о методах обработки метаданных в Elasticsearch. Мой личный опыт использования Elasticsearch ограничивается только BigData/Hadoop, поэтому и тут удалось узнать что-то новое.
Много новых иновационных идей было услышано от Ричарда. Он уже давно стал openSUSE evangelist’ом; его доклады об OBS или openQA можно услышать практически на каждой европейской Free Software конференции, посвященной GNU/Linux. Я всем советую посмотреть его доклад о Containerised Application.

Одним из спонсоров конференции в этом году стал fedora project. Fedora уже не первый год использует нашу систему openQA для автоматического тестирования linux-систем, а сотрудники RedHat уже во второй раз читают свои доклады на openSUSE Conference. В этом году это был доклад How semantic analysis of C and C++ ELF binaries can be used to analyze ABI changes, в прошлом году это были Enforcement of a system-wide crypto policy и Testing complex software in CI.
В общем, как и обычно, конферениция оставила приятное впечатление (пару фоток можно найти тут). И хотя в этом году она длилась всего 3 дня вместо 5, как в прошлом, информации для размышления я пролучил придостаточно. Её посещение в этом году не стоило мне ничего. Спасибо за это TSP. В следующем году она пройдет в Праге, куда я планирую поехать с семьей. Возможно там я встречусь и с теми, кто читает сейчас эти стоки 
Setting Alt-Tab behavior in gnome-shell
After updating my distro a few months ago, I somehow lost my tweaks to the Alt-Tab behavior in gnome-shell.
The default is to have Alt-Tab switch you between applications in the
current workspace. One can use Alt-backtick (or whatever key you
have above Tab) to switch between windows in the current application.
I prefer a Windows-like setup, where Alt-Tab switches between
windows in the current workspace, regardless of the application to
which they belong.
Many moons ago there was a gnome-shell extension to change this behavior, but these days (GNOME 3.24) it can be done without extensions. It is a bit convoluted.
With the GUI
If you are using X instead of Wayland, this works:
-
Unset the Switch applications command. To do this, run
gnome-control-center, go to Keyboard, and find the Switch applications command. Click on it, and hitBackspacein the dialog that prompts you for the keyboard shortcut. Click on the Set button. -
Set the Switch windows command. While still in the Keyboard settings, find the Switch windows command. Click on it, and hit
Alt-Tab. Click Set.
That should be all you need, unless you are in Wayland. In that case, you need to do it on the command line.
With the command line, or in Wayland
The kind people on #gnome-hackers tell me that as of GNOME
3.24, changing Alt-Tab doesn't work on Wayland as in (2) above,
because the compositor captures the Alt-Tab key when you type it
inside the dialog that prompts you for a keyboard shortcut. In that
case, you have to change the configuration keys directly instead of
using the GUI:
gsettings set org.gnome.desktop.wm.keybindings switch-applications "[]"
gsettings set org.gnome.desktop.wm.keybindings switch-applications-backward "[]"
gsettings set org.gnome.desktop.wm.keybindings switch-windows "['<Alt>Tab', '<Super>Tab']"
gsettings set org.gnome.desktop.wm.keybindings switch-windows-backward "['<Alt><Shift>Tab', '<Super><Shift>Tab']"
Of course the above also works in X, too.
Changing windows across all workspaces
If you'd like to switch between windows in all workspaces, rather than
in the current workspace, find the org.gnome.shell.window-switcher
current-workspace-only GSettings key and change it. You can do this
in dconf-editor, or on the command line with
gsettings set org.gnome.shell.window-switcher current-workspace-only true
Packaging MediaWiki extensions
As part of the work for the openSUSE wiki upgrade and move, I had to package a bunch of MediaWiki extensions. We'll use the MediaWiki 1.27.x LTS release, which means the extensions need to work with this version.
When it comes to packaging, there are three categories of extensions:
The Good
These extensions are hosted on phabricator.wikimedia.org, and you can easily download a tarball matching your MediaWiki version using the "Download snapshot" link on the extension page.
Packaging these extensions is easy - just unpack the tarball and copy/package everything to the extension directory.
These extensions are standardized enough to use a spec file template - usually I only had to adjust the extension name, tarball name and version. Speaking of the version - most extensions don't have explicit version numbers, so I decided to use the tarball date instead.
An example for this category is Auth_remoteuser (extension page, package) which we use to keep the "nice" wiki login form.
The Bad
These extensions are hosted on GitHub and typically only have a "master" branch. They usually still work with MediaWiki 1.27.x, but there's a small risk that they require features added in newer MediaWiki versions, and this risk will grow over time.
On the packaging side, they are as easy as the "good" extensions.
An example is the ParamProcessor extension (extension page, package) which is needed by the Maps extension
The Ugly
These extensions can be hosted on phabricator.mediawiki.org or GitHub, so there are "god ugly" and "bad ugly" extensions ;-) The thing that makes packaging really ugly is that they don't include all the code they need. Instead, you have to download the missing parts with composer.
composer works fine in a "real" system, but makes packaging hard. Running it from the spec will obviously fail because OBS doesn't allow network connections while building a package (and even if it's annoying in this case, not having network access during build is a good thing[tm]).
My solution is a little script that unpacks the extension tarball and runs "composer install --no-dev" inside the extension directory. The most important part is the "--no-dev" parameter because that avoids lots of superfluous things. Afterwards, I build a tarball from the "vendor" directory and add it to the package.
Yeah, I know that's not nice - guess why I named this section "The Ugly" ;-)
One of the packages that need a "composer install" run is the GitHub extension (extension page, package including script to run composer).
Luckily, "ugly" only applies to packaging. The extensions and their maintainers are for sure not ugly - for example, the maintainer of the GitHub extension was very fast in fixing a bug :-)
golang range Tickers
Yesterday Praveen sent me an interesting piece of golang code. Read the following code and tell what the answer will be:
===
type LED struct {
state bool
ticker *time.Ticker
}
func toggle(led *LED) {
led.state = !led.state
}
func looper(led *LED) {
for range led.ticker.C {
toggle(led)
}
}
func main() {
fmt.Println("Initial number of GoRoutines: ", runtime.NumGoroutine())
led := &LED{state: true, ticker: time.NewTicker(time.Millisecond * 500)}
go looper(led)
fmt.Println("Number of GoRoutines after a call to looper: ", runtime.NumGoroutine())
time.Sleep(2 * time.Second)
led.ticker.Stop()
fmt.Println("Number of GoRoutines after stopping the ticker: ", runtime.NumGoroutine())
runtime.GC()
fmt.Println("Number of GoRoutines after gc: ", runtime.NumGoroutine())
}
===
Golang playground URL: https://play.golang.org/p/1as5QN1r2c
Gist URL: https://gist.github.com/psankar/8af76ba183b0203ec141bca8156f5955
I will explain roughly what the code is doing.
There is a LED struct which has a Ticker and a state variable. While creating an instance of the led struct, we initialise the state and the Ticker. There is a looper function will toggle the state, whenever the Ticker fires an event.
Now when the program is launched, there will be one goroutine (the initial main thread). After we call looper in a goroutine, the goroutineCount will be 2. Now, comes the tricky part. We stop the Ticker, after a particular amount of time. We even call the gc.
It was observed by Praveen that this piece of code was leaking go routines and the number of go routines was never going down, inspite of the Ticker getting stopped.
The reason why the leakage is happening is because, the "range" loop is never exiting. If the range loop was on a channel, you could "close" it. The ticker.C channel however is a receive only channel and you cannot close it.
How do we fix this, so that none of the goroutines are leaking ? If you have watched the talks, golang concurrency patterns by Rob Pike and Advanced golang concurrency patterns by Sameer Ajmani, then you will realise that it is quite easy to add another parameter to the looper function, which could just exit the loop. So the updated code will be:
===
type LED struct {
state bool
ticker *time.Ticker
}
func toggle(led *LED) {
led.state = !led.state
}
func looper(led *LED) {
for range led.ticker.C {
toggle(led)
}
}
func looper2(led *LED, q chan bool) {
for {
select {
case <-led.ticker.C:
toggle(led)
case <-q:
fmt.Println("Exiting the goroutine")
return
}
}
}
func main() {
fmt.Println("Initial number of GoRoutines: ", runtime.NumGoroutine())
led := &LED{state: true, ticker: time.NewTicker(time.Millisecond * 500)}
q := make(chan bool)
go looper2(led, q)
// go looper(led)
fmt.Println("Number of GoRoutines after a call to looper: ", runtime.NumGoroutine())
time.Sleep(2 * time.Second)
led.ticker.Stop()
fmt.Println("Number of GoRoutines after stopping the ticker: ", runtime.NumGoroutine())
q <- true
fmt.Println("Number of GoRoutines after sending a message on the quit channel: ", runtime.NumGoroutine())
}
===
Playground URL: https://play.golang.org/p/NlWbyHLHvA
Gist URL: https://gist.github.com/psankar/4e5b2e563038ce3e9c17eb208c76168a
Let me know if you have any comments.
Langkah Sederhana Upgrade Leap 42.1 ke 42.2
Pada kesempatan kali ini Saya mencoba untuk membuat langkah sederhana untuk memutakhirkan versi (upgrade) openSUSE Leap 42.1 menjadi 42.2. Untuk penjelasan lebih detail, Anda bisa mengunjungi panduan upgrade official dari openSUSE di link ini. Sedangkan pada tutorial ini, Saya hanya akan mencoba untuk menjelaskan langkah yang sangat sederhana untuk melakukan proses upgrade. Langkah ini telah saya lakukan dan berhasil. Berikut langkah – langkahnya.
- Pastikan sistem Anda telah dilakukan update, jika belum Anda bisa melakukannya dengan melalui Software Manager atau dengan menggunakan terminal (akses root) dengan mengetik zypper up.
- Lakukan backup repository sekarang dengan mengetik cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.old.
- Hapus repositori yang tidak diperlukan melalui aplikasi YaST2 kemudian pilih Software Repositories. Cara ini bisa dilakukan juga dengan melalui terminal, namun sebagaimana pada kesepakatan awal bahwa Saya hanya akan menjelaskan langkah yang sederhana dan mudah.

Agar lebih mudah ketika melakukan upgrade, berikut repositori yang tidak Saya hapus.
* OSS
* Non OSS
* OSS Updates
* Packman - Ubah angka versi rilis pada repositori yang ada menjadi 42.2 (sebelumnya 42.1)
- Kembali ke terminal, lakukan refresh dengan mengetik zypper ref.
- Unduh paket dari repositori terbaru dengan mengetik zypper dup –download-only.
- Setelah proses unduh selesai, logout dari sistem.
- Tekan Ctrl+Alt+F1 untuk masuk ke mode teks dan login sebagai root
- Ubah runlevel ke 3 dengan mengetik init 3.
- Terakhir ketik zypper –no-refresh dup untuk melakukan proses upgrade.
Proses upgrade membutuhkan beberapa menit. setelah proses upgrade selesai dan tidak terjadi kendala Anda bisa melakukan reboot pada sistem kemudian setelah login Anda bisa mengecek versi dari sistem dengan melalui terminal dengan mengetik lsb_release -a.
Keyword : openSUSE 42.1, openSUSE 42.2, Leap, Update, Upgrade, Instalasi
Exploring Rust's standard library: system calls and errors
In this post I'll show you the code path that Rust takes inside its
standard library when you open a file. I wanted to learn how Rust
handles system calls and errno, and all the little subtleties of the
POSIX API. This is what I learned!
The C side of things
When you open a file, or create a socket, or do anything else that
returns an object that can be accessed like a file, you get a file
descriptor in the form of an int.
/* All of these return a int with a file descriptor, or
* -1 in case of error.
*/
int open(const char *pathname, int flags, ...);
int socket(int domain, int type, int protocol);
You get a nonnegative integer in case of success, or -1 in case of an
error. If there's an error, you look at errno, which gives you an
integer error code.
int fd;
retry_open:
fd = open ("/foo/bar/baz.txt", 0);
if (fd == -1) {
if (errno == ENOENT) {
/* File doesn't exist */
} else if (errno == ...) [
...
} else if (errno == EINTR) {
goto retry_open; /* interrupted system call; let's retry */
}
}
Many system calls can return EINTR, which means "interrupted system
call", which means that something interrupted the kernel while it
was doing your system call and it returned control to userspace, with
the syscall unfinished. For example, your process may have received a
Unix signal (e.g. you send it SIGSTOP by pressing Ctrl-Z on a
terminal, or you resized the terminal and your process got a
SIGWINCH). Most of the time EINTR means simply that you must
retry the operation: if you Control-Z a program to suspend it, and
then fg to continue it again; and if the program was in the middle
of open()ing a file, you would expect it to continue at that exact
point and to actually open the file. Software that doesn't check for
EINTR can fail in very subtle ways!
Once you have an open file descriptor, you can read from it:
ssize_t
read_five_bytes (int fd, void *buf)
{
ssize_t result;
retry:
result = read (fd, buf, 5);
if (result == -1) {
if (errno == EINTR) {
goto retry;
} else {
return -1; /* the caller should cherk errno */
}
} else {
return result; /* success */
}
}
... and one has to remember that if read() returns 0, it means we
were at the end-of-file; if it returns less than the number of bytes
requested it means we were close to the end of file; if this is a
nonblocking socket and it returns EWOULDBLOCK or EAGAIN then one
must decide to retry the operation or actually wait and try again
later.
There is a lot of buggy software written in C that tries to use the POSIX API directly, and gets these subtleties wrong. Most programs written in high-level languages use the I/O facilities provided by their language, which hopefully make things easier.
I/O in Rust
Rust makes error handling convenient and safe. If you decide to
ignore an error, the code looks like it is ignoring the error
(e.g. you can grep for unwrap() and find lazy code). The
code actually looks better if it doesn't ignore the error and
properly propagates it upstream (e.g. you can use the ? shortcut to
propagate errors to the calling function).
I keep recommending this article on error models to people; it discusses POSIX-like error codes vs. exceptions vs. more modern approaches like Haskell's and Rust's - definitely worth studying over a few of days (also, see Miguel's valiant effort to move C# I/O away from exceptions for I/O errors).
So, what happens when one opens a file in Rust, from the toplevel API down to the system calls? Let's go down the rabbit hole.
You can open a file like this:
use std::fs::File;
fn main () {
let f = File::open ("foo.txt");
...
}
This does not give you a raw file descriptor; it gives you an
io::Result<fs::File, io::Error>, which you must pick apart to see if
you actually got back a File that you can operate on, or an error.
Let's look at the implementation of File::open() and File::create().
impl File {
pub fn open<P: AsRef<Path>>(path: P) -> io::Result<File> {
OpenOptions::new().read(true).open(path.as_ref())
}
pub fn create<P: AsRef<Path>>(path: P) -> io::Result<File> {
OpenOptions::new().write(true).create(true).truncate(true).open(path.as_ref())
}
...
}
Here, OpenOptions is an auxiliary struct that implements a "builder"
pattern. Instead of passing bitflags for the various
O_CREATE/O_APPEND/etc. flags from the open(2) system call, one
builds a struct with the desired options, and finally calls .open()
on it.
So, let's look at the implementation of OpenOptions.open():
pub fn open<P: AsRef<Path>>(&self, path: P) -> io::Result<File> {
self._open(path.as_ref())
}
fn _open(&self, path: &Path) -> io::Result<File> {
let inner = fs_imp::File::open(path, &self.0)?;
Ok(File { inner: inner })
}
See that fs_imp::File::open()? That's what we want: it's the
platform-specific wrapper for opening files. Let's look
at its implementation for Unix:
pub fn open(path: &Path, opts: &OpenOptions) -> io::Result<File> {
let path = cstr(path)?;
File::open_c(&path, opts)
}
The first line, let path = cstr(path)? tries to convert a Path
into a nul-terminated C string. The second line calls the following:
pub fn open_c(path: &CStr, opts: &OpenOptions) -> io::Result<File> {
let flags = libc::O_CLOEXEC |
opts.get_access_mode()? |
opts.get_creation_mode()? |
(opts.custom_flags as c_int & !libc::O_ACCMODE);
let fd = cvt_r(|| unsafe {
open64(path.as_ptr(), flags, opts.mode as c_int)
})?;
let fd = FileDesc::new(fd);
...
Ok(File(fd))
}
Here, let flags = ... converts the OpenOptions we had in the
beginning to an int with bit flags.
Then, it does let fd = cvt_r (LAMBDA), and that lambda function
calls the actual open64() from libc (a Rust wrapper for the system's
libc): it returns a file descriptor, or -1 on error. Why is this
done in a lambda? Let's look at cvt_r():
pub fn cvt_r<T, F>(mut f: F) -> io::Result<T>
where T: IsMinusOne,
F: FnMut() -> T
{
loop {
match cvt(f()) {
Err(ref e) if e.kind() == ErrorKind::Interrupted => {}
other => return other,
}
}
}
Okay! Here f is the lambda that calls open64(); cvt_r() calls
it in a loop and translates the POSIX-like result into something
friendly to Rust. This loop is where it handles EINTR, which gets
translated into ErrorKind::Interrupted. I suppose cvt_r() stands
for convert_retry()? Let's look at
the implementation of cvt(), which fetches the error code:
pub fn cvt<T: IsMinusOne>(t: T) -> io::Result<T> {
if t.is_minus_one() {
Err(io::Error::last_os_error())
} else {
Ok(t)
}
}
(The IsMinusOne shenanigans are just a Rust-ism to help convert
multiple integer types without a lot of as casts.)
The above means, if the POSIX-like result was -1, return an Err() from
the last error returned by the operating system. That should surely
be errno internally, correct? Let's look at
the implementation for io::Error::last_os_error():
pub fn last_os_error() -> Error {
Error::from_raw_os_error(sys::os::errno() as i32)
}
We don't need to look at Error::from_raw_os_error(); it's just a
conversion function from an errno value into a Rust enum value.
However, let's look at sys::os::errno():
pub fn errno() -> i32 {
unsafe {
(*errno_location()) as i32
}
}
Here, errno_location() is an extern function defined in GNU libc
(or whatever C library your Unix uses). It returns a pointer to the
actual int which is the errno thread-local variable. Since non-C
code can't use libc's global variables directly, there needs to be a
way to get their addresses via function calls - that's what
errno_location() is for.
And on Windows?
Remember the internal File.open()? This is what it looks
like on Windows:
pub fn open(path: &Path, opts: &OpenOptions) -> io::Result<File> {
let path = to_u16s(path)?;
let handle = unsafe {
c::CreateFileW(path.as_ptr(),
opts.get_access_mode()?,
opts.share_mode,
opts.security_attributes as *mut _,
opts.get_creation_mode()?,
opts.get_flags_and_attributes(),
ptr::null_mut())
};
if handle == c::INVALID_HANDLE_VALUE {
Err(Error::last_os_error())
} else {
Ok(File { handle: Handle::new(handle) })
}
}
CreateFileW() is the Windows API function to open files. The
conversion of error codes inside Error::last_os_error() happens
analogously - it calls GetLastError() from the Windows API and
converts it.
Can we not call C libraries?
The Rust/Unix code above depends on the system's libc for open() and
errno, which are entirely C constructs. Libc is what actually does
the system calls. There are efforts to make the Rust standard library
not use libc and use syscalls directly.
As an example, you can look at the Rust standard library for Redox. Redox is a new operating system kernel entirely written in Rust. Fun times!
Update: If you want to see what a C-less libstd would look like, take a look at steed, an effort to reimplement Rust's libstd without C dependencies.
Conclusion
Rust is very meticulous about error handling, but it succeeds in
making it pleasant to read. I/O functions give you back an
io::Result<>, which you piece apart to see if it succeeded or got an
error.
Internally, and for each platform it supports, the Rust standard
library translates errno from libc into an io::ErrorKind Rust
enum. The standard library also automatically handles Unix-isms like
retrying operations on EINTR.
I've been enjoying reading the Rust standard library code: it
has taught me many Rust-isms, and it's nice to see how the
hairy/historical libc constructs are translated into clean Rust
idioms. I hope this little trip down the rabbit hole for the
open(2) system call lets you look in other interesting places, too.
Don't defer Close() on writable files
fsync() added to the end of the post.It’s an idiom that quickly becomes rote to Go programmers: whenever
you conjure up a value that implements the io.Closer interface,
after checking for errors you immediately defer its Close()
method. You see this most often when making HTTP requests:
resp, err := http.Get("https://joeshaw.org")
if err != nil {
return err
}
defer resp.Body.Close()
or opening files:
f, err := os.Open("/home/joeshaw/notes.txt")
if err != nil {
return err
}
defer f.Close()
But this idiom is actually harmful for writable files because
deferring a function call ignores its return value, and the Close()
method can return errors. For writable files, Go programmers should
avoid the defer idiom or very infrequent, maddening bugs will
occur.
Why would you get an error from Close() but not an earlier Write()
call? To answer that we need to take a brief, high-level detour into
the area of computer architecture.
Generally speaking, as you move outside and away from your CPU, actions get orders of magnitude slower. Writing to a CPU register is very fast. Accessing system RAM is quite slow in comparison. Doing I/O on disks or networks is an eternity.
If every Write() call committed the data to the disk synchronously,
the performance of our systems would be unusably slow. While
synchronous writes are very important for certain types of software
(like databases), most of the time it’s overkill.
The pathological case is writing to a file one byte at a time. Hard drives – brutish, mechanical devices – need to physically move a magnetic head to the position on the platter and possibly wait for a full platter revolution before the data could be persisted. SSDs, which store data in blocks and have a finite number of write cycles for each block, would quickly burn out as blocks are repeatedly written and overwritten.
Fortunately this doesn’t happen because multiple layers within
hardware and software implement caching and write buffering. When you
call Write(), your data is not immediately being written to media.
The operating system, storage controllers and the media itself are all
buffering the data in order to batch smaller writes together,
organizing the data optimally for storage on the medium, and deciding
when best to commit it. This turns our writes from slow, blocking
synchronous operations to quick, asynchronous operations that don’t
directly touch the much slower I/O device. Writing a byte at a time
is never the most efficient thing to do, but at least we are not
wearing out our hardware if we do it.
Of course, the bytes do have to be committed to disk at some point. The operating system knows that when we close a file, we are finished with it and no subsequent write operations are going to happen. It also knows that closing the file is its last chance to tell us something went wrong.
On POSIX systems like Linux and macOS, closing a file is handled by
the close system call. The BSD man page for close(2) talks about
the errors it can return:
ERRORS
The close() system call will fail if:
[EBADF] fildes is not a valid, active file descriptor.
[EINTR] Its execution was interrupted by a signal.
[EIO] A previously-uncommitted write(2) encountered an input/output
error.
EIO is exactly the error we are worried about. It means that we’ve
lost data trying to save it to disk, and our Go programs should
absolutely not return a nil error in that case.
The simplest way to solve this is simply not to use defer when
writing files:
func helloNotes() error {
f, err := os.Create("/home/joeshaw/notes.txt")
if err != nil {
return err
}
if err = io.WriteString(f, "hello world"); err != nil {
f.Close()
return err
}
return f.Close()
}
This does mean additional bookkeeping of the file in the case of
errors: we must explicitly close it in the case where
io.WriteString() fails (and ignore its error, because the write
error takes precedence). But it’s clear, straightforward, and
properly checks the error from the f.Close() call.
There is a way to handle this case with defer by using named
return values and a closure:
func helloNotes() (err error) {
var f *os.File
f, err = os.Create("/home/joeshaw/notes.txt")
if err != nil {
return
}
defer func() {
cerr := f.Close()
if err == nil {
err = cerr
}
}()
err = io.WriteString(f, "hello world")
return
}
The main benefit of this pattern is that it’s not possible to forget
to close the file because the deferred closure always executes. In
longer functions with more if err != nil conditional branches, this
pattern can also result in fewer lines of code and less repetition.
Still, I find this pattern to be a little too magical. I dislike using named return values, and modifying the return value after the core function finishes is not intuitively clear even to experienced Go programmers.
I am willing to accept the tradeoff of more readable and easily understandable code for needing to obsessively review code to ensure that the file is closed in all cases, and that’s the approach I recommend in code reviews I give to others.
Update
On Twitter, Ben Johnson
suggested
that Close() may be safe to run multiple times on files, like so:
func doSomething() error {
f, err := os.Create("foo")
if err != nil {
return err
}
defer f.Close()
if _, err := f.Write([]byte("bar"); err != nil {
return err
}
if err := f.Close(); err != nil {
return err
}
return nil
}
The Go docs on io.Closer explicitly say that
at an interface level behavior after the first call is unspecificed,
but specific implementations may document their own behavior.
The docs for *os.File unfortunately aren’t clear
on its behavior, saying only, “Close closes the File, rendering it
unusable for I/O. It returns an error, if any.” The implemenation as
of 1.8, however, shows:
func (f *File) Close() error {
if f == nil {
return ErrInvalid
}
return f.file.close()
}
func (file *file) close() error {
if file == nil || file.fd == badFd {
return syscall.EINVAL
}
var err error
if e := syscall.Close(file.fd); e != nil {
err = &PathError{"close", file.name, e}
}
file.fd = -1 // so it can't be closed again
// no need for a finalizer anymore
runtime.SetFinalizer(file, nil)
return err
}
For clarity, badFd is defined as -1, so subsequent attempts to close
an *os.File will do nothing and return syscall.EINVAL. But since
we are ignoring the error from the defer, this doesn’t matter. It’s
not idempotent, exactly, but as Ben put later in the Twitter thread,
it “won’t blow shit up if you call it
twice.”
The implementation is a good, common-sense one and it seems unlikely to change in the future and cause problems. But the lack of documentation about this outcome makes me a little nervous. Maybe a doc update to codify this behavior would be a good task for Go 1.10.
Update 2
Closing the file is the last chance the OS has to tell us about problems, but the buffers are not necessarily going to be flushed when you close the file. It’s entirely possible that flushing the write buffer to disk will happen after you close the file, and a failure there cannot be caught. If this happens, it usually means you have something seriously wrong, like a failing disk.
However, you can force the write to disk with the Sync() method on
*os.File, which calls the fsync system call. You should check for
errors from that call, but then I think it’s safe to ignore an error
from Close(). Calling fsync has serious implications on
performance: it’s flushing write buffers out to slow disks. But if
you really, really want the data on disk, the best pattern to follow
is probably:
func helloNotes() error {
f, err := os.Create("/home/joeshaw/notes.txt")
if err != nil {
return err
}
defer f.Close()
if err = io.WriteString(f, "hello world"); err != nil {
return err
}
return f.Sync()
}
Synology PhotoStation password vulnerability
Using a simple shell loop to run "ps ax | grep synophoto_dsm_user", it was possible to get user and password credentials for user on the NAS who had PhotoStation enabled with their DSM credentials.
Fortunately, by default, shell access on the NAS is not available (by ssh or telnet), it has to be enabled by the admin.
Still, it is a bad practise to pass credentials to process using command line, which can be intercepted.
PhotoStation version 6.7.1-3419 or earlier is vulnerable. I've contacted Synology and they should release a security fix really shortly, as well as a CVE for it.
Update (June 13, 2017): Synology has released a CVE and the vulnerability is fixed in PhotoStation 6.7.2-3429 or later. Remember to update this package on your NAS !
