In the past months I’ve set up LDAP at home, to avoid having different user accounts for the services that I run on my home hardware. Rather than the venerable OpenLDAP I settled for 389 Directory Server, commercially known as Red Hat Directory Server, mainly because I was more familiar with it. Rather than describing how to set that up (Red Hat’s own documentation is excellent on that regard), this post will focus on the steps required to enable encryption using Let’s Encrypt certificates.

The problem

Even the LDAP server (they’re actually two, but for the purpose of this post it does not matter) was just operating in my LAN, I wanted to reduce the amount of information going around in clear, including LDAP queries. That meant setting up encryption, of course.

The problem was that Red Hat’s docs only cover the “traditional” way of obtaining certificates, that is obtaining a Certificate Authority (CA) certificate, request a server certificate, obtain it and set it up in the server. There is (obviously) no mention of Let’s Encrypt anywhere. I’ve found some guides, but they were either too complicated (lots of fuzzing around with certutil) or they weren’t clear for some steps. Hence, this post.

NOTE: I’ve focused on 389-ds version 2.0 and above, which has a different CLI set of commands than the venerable 1.3 series. All of the steps shown here can also be carried out via the Cockpit Web interface as well, if your distribution carries it (spoiler: openSUSE doesn’t).

Importing the CA

This is arguably one of the most important steps of the process. 389-ds needs also to store the CA for your certificates. As you may (or may not) know, Let’s Encrypt’s CA certificates are two:

• The actual “root” certificate, by Internet Security Research Group (ISRG);
• An intermediate certificate, signed by the above root, called “R3”.

Let’s Encrypt certificates, like the one powering this website, are signed by R3. But since you have to follow a “chain of trust”, to validate the certificate you follow these steps (excuse me, security people; this is probably a bad semplification):

1. Check the actual certificate (e.g. the one on dennogumi.org)
2. The certificate is signed by “R3”, so move up the chain and check the R3 certificate
3. The R3 certificate is signed by the ISRG root certificate, so move up the chain and check the ISRG root
4. The ISRG certificate is trusted by the OS / application using it, so everything stops there.

If any of the steps fails, the whole validation fails.

This long winding explanation is to tell you that 389-ds needs the whole certificate chain for its CA (so ISRG root + R3) in order to properly validate the Let’s Encrypt certificate you’ll use. If you don’t do that, chances are that some software which uses the system CA will work (for example ldapsearch) but others, like SSSD will fail with “Unknown CA” errors (buried deep into debug logs, so at the practical level they’ll just fail and you won’t know why).

Let’s get to business. Access the Chain of Trust page for Let’s Encrypt and download the relevant certificates. I’m not sure if 389-ds supports ECDSA certificates, so I downloaded the RSA ones: ISRG Root X1 and Let’s Encrypt R3 (both in PEM format). Put them somewhere in your server. Then, as root, import the two CA certificates into 389-ds (substitute LDAP_ADDRESS with the LDAP URI of your server):

# ISRG Root

dsconf -v -D "cn=Directory Manager"  LDAP_ADDRESS security ca-certificate \
    add --file /path/to/certificate --name "ISRG"

# Let's Encrypt R3

dsconf -v -D "cn=Directory Manager"  LDAP_ADDRESS security ca-certificate \
    add --file /path/to/certificate --name "R3"

NOTE: This step may not be necessary if you use Let’s Encrypt’s “full chain” certificates, but I did not test that.

Importing the certificates

Then, you have to import a Let’s Encrypt certificate, which means you have to obtain one. There are hundreds of guides and clients that can do the job nicely, so I won’t cover that part. If you use certbot, Let’s Encrypt’s official client, you will have the certificate and the private key for it in /etc/letsencrypt/live/YOURDOMAIN/fullchain.pem and /etc/letsencrypt/live/YOURDOMAIN/privkey.pem.

You need to import the private key first (substitute LDAP_ADDRESS and DOMAIN with the LDAP URI of your server and the Let’s Encrypt domain, respectively):

dsctl LDAP_ADDRESS tls import-server-key-cert \
    /etc/letsencrypt/live/DOMAIN/fullchain.pem \
        /etc/letsencrypt/live/DOMAIN/privkey.pem

Note that you pass the certificate as well as the key to import it (in doubt, check’s Red Hat documentation).

Once the key is done, it is time to import the actual certificate:

dsconf -v -D "cn=Directory Manager" LDAP_ADDRESS security certificate add \
    --file /etc/letsencrypt/live/DOMAIN/fullchain.pem \
        --primary-cert \
        --name "LE"

--primary-cert sets the certificate as the server’s primary certificate.

Then, we switch on TLS in the server:

dsconf -v -D "cn=Directory Manager" LDAP_ADDRESS config replace \
	nsslapd-securePort=636 nsslapd-security=on

And finally, we restart our instance (replace INSTANCE with your configured instance name):

systemctl restart dirsrv@INSTANCE

Testing everything out

You can use ldapsearch to check whether the SSL connection is OK (I’ve used Directory Manager, but you can use any user you want):

# STARTTLS

ldapsearch -H ldap://your.ldap.hostname -W -x -D "cn=Directory Manager" -ZZ "search filter here"

# TLS

ldapsearch -H ldaps://your.ldap.hostname -W -x -D "cn=Directory Manager" "search filter here"

If everything is OK, you should get a result: otherwise, you’ll get an error like “Can’t contact the LDAP server”.

Alternatively, you can use openssl:

openssl s_client -connect your.ldap.hostname:636
[...]
---
SSL handshake has read 5188 bytes and written 438 bytes
Verification: OK

Don’t forget to adjust your applications to connect via ldaps rather than ldap after everything is done.

Renewing the certificates

To renew the certificates you repeat the steps outlined above for the certificates (without the CA part, of course). Make sure you always import your private key: if there is a mismatch between it and the certificate, 389-ds will refuse to start.

If you use certbot you can use a post-renewal hook to trigger the import of the certificate into 389-ds. This is what I’ve been using: bear in mind it’s customized to my setup and does a few more things than needed. Also it does only import the certificate, and not the full chain.

I messed up and 389-ds won’t start! What do I do?

You can disable encryption by editing /etc/dirsrv/slapd-INSTANCE/dse.ldif and changing nsslapd-security to off, then start 389-ds again. Then you can review everything and see what went wrong. But if you can, I recommend the Cockpit Web UI: it makes the first-time setup a breeze.

Wrap up

Importing the certificates is surprisingly simple, but my Internet searches have been frustrating because at least half of what I found was either not applicable, incomplete, or did not work. I hope this small tutorial can be useful for those who want a bit more security in their LDAP setup.

Have you ever edited a document in LibreOffice in more than one window? Right, neither have I. Who'd think about LibreOffice and more than one user at the same time, right? Except ... somebody did and that's how collaborative editing based on LibreOffice works. For whatever strange reason, somewhen in the past somebody thought that implementing multiple views for one document in OpenOffice (StarOffice?) was a good idea. Just select Window->New Window in the menu and you can edit your favourite document in 50 views that each show a different part of the document and update in real time. And that, in fact, is how collaborative editing such as with Collabora Online works - open a document, create a new view for every user, and there you go.

But, given that this has never really been used that much, how well did the original relevant code perform and scale for more users? Well, not much, it turns out. Not a big surprise, considering that presumably back when that code was written nobody thought the same document could be edited by numerous users at the same time. But I've been looking exactly into this recently as part of optimizing Collabora Online performance, and boy, are there were gems in there. You thought that showing the same document in more views would just mean more painting also in those views? Nah, think again, this is OpenOffice code, the land of programming wonders.

Profiling the code

When running Online's perf-test, which simulates several users typing in the same document, most of the time is actually spent in SwEditShell::EndAllAction(). It's called whenever Writer finishes an action such as adding another typed characters, and one of the things it does is telling other views about the change. So here LO spends a little time adding the character and then the rest of the time is spent in various code parts "talking" about it. A good part of that is that whenever an action is finished, that view tells the others about it happening, and then all those views tell all other views about how they reacted to it, making every change O(n^2) with the number of views. That normally does not matter, since on the desktop n generally tends to be 1, but hey, add few more views, and it can be a magnitude slower or more.

Redrawing, for example, is rather peculiar. When a part of the document changes, relevant areas of the view need redrawing. So all views get told about the rectangles that need repainting. In the desktop case those can be cropped by the window area, but for tiled rendering used by Online the entire document is the "window" area, so every view gets told about every change. And each view collects such rectangles, and later on it processes them and tells all other views about the changes. Yes, again. And it seems that in rare cases each view really needs its own repaint changes (even besides the cropping, as said before). So there's a lot of repeated processing of usually the same rectangles over and over again.

One of the functions prominently taking place in CPU costs is SwRegionRects::Compress(), which ironically is supposed to make things faster by compressing a group of rectangles into a smaller set of rectangles. I guess one of the cases in OpenOffice where the developer theoretically heard about optimizations being supposed to make things faster, but somehow the practical side of things just wasn't there. What happens here is that the function compares each rectangle with each other, checking if they can be merged ... and then, if yes and that changes the set of rectangles, it restarts the entire operation. Which easily makes the entire thing O(n^3). I have not actually found out why the restarting is there. I could eventually think of a rather rare case where restarting makes it possible to compress the rectangles more, but another possibility is that the code dates back to the time when it was not safe to continue after modifying whichever container SwRegionRects was using back then, and it has stayed there even though that class has been using to std::vector since a long time ago.

Another kind of interesting take on things is the SwRegionRects::operator-= in there. Would you expect that rectangles would be collected by simply, well, collecting them and then merging them together? Maybe you would, but that's not how it's done here. See, somebody apparently thought that it'd be better to use the whole area, and then remove rectangles to paint from it, and then at the end invert the whole thing. The document area is limited, so maybe this was done to "easily" crop everything by the area? It works, except, of course, this is way slower. Just not slow enough to really notice when n is 1.

Other code that works fine with small numbers but fails badly with larger ones is VCLEventListeners, a class for getting notified about changes to VCL objects such as windows. It's simply a list of listener objects, and normally there aren't that many of those. But if LO core gets overloaded, this may grow. And since each listener may remove itself from the list at any point, the loop calling all of them always checks for each of them if the listener is still in the list. So, again, O(n^2). And, of course, it's only rarely that any listener removes itself, so the code spends a lot of time doing checks just in case.

But so that I do not talk only about old code, new code can do equally interesting things. Remote rendering uses LOK (LibreOfficeKit), which uses text-based messages to send notifications about changes. And the intuitive choice for writing text are C++ iostreams, which are flexible, and slow. So there will be a lot of time spent in creating text messages, because as said above, there are many changes happening, repeatedly. And since there are so many repeated messages, it makes sense to write extra class CallbackFlushHandler that collects these messages and drops duplicates. Except ... for many of the checks it first needs to decode text mesages back to binary data, using C++ iostreams. And in most cases, it will find out that it can drop some message duplicates, so all these string conversions were done for nothing. Oops.

And there are more ways in which things can get slower rather than faster. CallbackFlushHandler uses an idle timer to first process all data in bulk and flush the data at once only when idle. Except if it gets too busy to keep up, and it can easily get too busy because of all the things pointed out above, it may take a very long time before any data is flushed. To make things even worse, the queue of collected messages will be getting longer and longer, which means searching for duplicates and compressing it will get longer. Which in turn will make everything even slower, which again in turn will make everything even slower. Bummer.

All in all, if unlucky, it may not take that much for everything to slow down very noticeably. Online's perf-test, which simulates only 6 users typing, can easily choke itself for a long time. Admitedly, it simulates them all typing at the same time and rather fast, which is not very a realistic scenario, but typing hitting the keyboard randomly and quickly is exactly how we all test things, right? So I guess it could be said that Collabora Online's perf-test simulates users testing Collabora Online performance :). Realistic scenarios are not going to be this bad.

Anyway. In this YT video you can see in the top part how perf-test performs without any optimizations. The other 5 simulated users are typing elsewhere in the document, so it's not visible, but it affects performance.

Improved performance

But as you can see in the other two parts, this poor performance is actually already a thing of the past. The middle part shows how big a difference can even one change make. In this specific case, the only difference is adding an extra high-priority timer to CallbackFlushHandler, which tries to flush the message queue before it becomes too big.

The bottom part is all the improvements combined, some of them already in git, some of them I'm still cleaning up. That includes changes like:

• SwRegionRects::Compress() is now roughly somewhere at O(n*log(n)) I think. I've fixed the pointless restarts on any change, and implemented further optimizations such as Noel's idea to first sort the rectangles and not compare ones that cannot possibly overlap.
• I have also changed the doubly-inverted paint rectangles handling to simply collecting them, cropping them at the end and compressing them.
• One of the things I noticed when views collect their paint rectangles is that often they are adjacent and together form one large rectangle. So I have added a rather simple optimization of detecting this case and simply growing the previous rectangle.
• Since it seems each Writer view really needs to collect its own paint rectangles, I have at least changed it so that they do not keep telling each other about them all the time in LOK mode. Now they collect them, and only once at end they are all combined together and compressed, and often thousands of rectangles become just tens of them.
• Another thing Writer views like to announce all the time in LOK mode are cursor and selection positions. Now they just set a flag and compute and send the data only once at the end if needed.
  
• VCLEventListeners now performs checks only if it knows that a listener has actually removed itself. Which it knows, because it manages the list.
  
• Even though LOK now uses tiled rendering to send the view contents to clients, LO core was still rendering also to windows, even though those windows are never shown. That's now avoided by ignoring window invalidations in LOK mode.
• Noel has written a JSON writer which is faster then the Boost one, and made non-JSON parts use our OString, which is faster and better suited for the fixed-format messages.
  
• I have converted the from-message conversions to also use our strings, but more importantly I have changed internal LOK communications to be binary rather than text based, so that they usually do not even have to be converted. Only at the end those relatively few non-duplicated text messages are created.
• Noel has optimized some queue handling in CallbackFlushHandler and then I have optimized it some more. Including the high-priority timer mentioned above.
• There have been various other improvements from others from Collabora as part of the recent focus on improving performance.
  
• While working on all of this, I noticed that even though we have support for Link Time Optimization (LTO), we do not use it, probably because it was broken on Windows. I've fixed this and sorted out few other small problems, and releases in the future should get a couple percent better performance across the board from this.
  

This is still work in progress, but it already looks much better, as now most of the time is actually spent doing useful things like doing the actual document changes or drawing and sending document tiles to clients. LOK and Collabora Online performance should improve noticeably, recent (Collabora) versions 6.4.x should include already some improvements, and the upcoming Collabora Online 2021 should have all of them.

And even though this's been an exercise in profiling LibreOffice performance for something nobody thought of back when the original OpenOffice code was written, some of these changes should matter even for desktop LibreOffice and will be included starting with LO 7.3.

Tumbleweed es una distribución “Rolling Release” de actualización contínua. Aquí puedes estar al tanto de las últimas novedades.

openSUSE Tumbleweed es la versión “rolling release” o de actualización continua de la distribución de GNU/Linux openSUSE.

Hagamos un repaso a las novedades que han llegado hasta los repositorios esta semana.

El anuncio original lo puedes leer en el blog de Dominique Leuenberger, publicado bajo licencia CC-by-sa, en este este enlace:

• http://dominique.leuenberger.net/blog/2021/10/opensuse-tumbleweed-review-of-the-week-2021-42

Está semana los test openQA han echado para atrás 6 snapshots por diversos problemas y solo se han llegado a publicar 2 nuevas snapshots.

Eso está bien. Quiere decir que los sistemas de testeo funcionan y cazan los errores antes de que estos lleguen a los usuarios y causen problemas en sus sistemas.

Como he dicho se han publicado 2 snapshotso, 1016 y 1019), que contenían entre otras, estas actualizaciones:

• KDE Gear 21.08.2
• KDE Plasma 5.23.0
• KDE Frameworks 5.87.0
• Apparmor
• Gimp 2.10.28
• Linux kernel 5.14.11
• Mesa 21.2.4

Y en próximas actualizaciones podrás encontrar:

• KDE Plasma 5.23.1
• Systemd 249.5
• Linux kernel 5.14.14
• Bison 3.8.2
• RPM 4.1Co
• openSSL 3.0.0

Si quieres estar a la última con software actualizado y probado utiliza openSUSE Tumbleweed la opción rolling release de la distribución de GNU/Linux openSUSE.

Mantente actualizado y ya sabes: Have a lot of fun!!

Enlaces de interés

• • • ¿Por qué deberías utilizar openSUSE Tumbleweed?
    • zypper dup en Tumbleweed hace todo el trabajo al actualizar
    • ¿Cual es el mejor comando para actualizar Tumbleweed?
    • Comprueba la valoración de las “snapshots” de Tumbleweed
    • ¿Qué es el test openQA?
    • http://download.opensuse.org/tumbleweed/iso/
    • https://es.opensuse.org/Portal:Tumbleweed

——————————–

Dear Tumbleweed users and hackers,

This week has been overshadowed a lot with Snapshots that did not pass openQA – in total, we have had 6 snapshots in QA – of which only 2 made it through and have been published. Of course, all the issues identified have resulted in relevant bug reports which the maintainers will be working on, fixing, resubmitting the packages that needed to be reverted, and then we move forward. After all, openQA does exactly what we want it to do: it protects you, the users, from getting broken snapshots.

So, as said, we published two snapshots (1016 and 1019), containing those updates:

• KDE Gear 21.08.2
• KDE Plasma 5.23.0
• KDE Frameworks 5.87.0
• Apparmor profile update for Samba printing subsystem
• Gimp 2.10.28
• Linux kernel 5.14.11 (5.14.12 was responsible for one of the snapshots being blocked)
• Mesa 21.2.4

The staging projects are currently filled with:

• KDE Plasma 5.23.1
• Systemd 249.5
• Linux kernel 5.14.14
• Bison 3.8.2: breaks gdb
• RPM 4.17: one fix for suse-hpc missing (Fix submitted to devel prj)
• Coreutils 9.0
• gpg 2.3.x: some openQA issues detected, https://progress.opensuse.org/issues/101358
• openSSL 3.0.0: no active progress

Apenas hablo de Debian en el blog, y es que me parece un sistema demasiado elevado para mis conocimientos. No obstante hoy me apetece porque se ha anunciado que ya está disponible Plasma 5.23 para Debian 11, un hecho inusual ya que esta distribución comunitaria no suele estar tan al día con el Software de la Comunidad KDE.

Disponible Plasma 5.23 para Debian 11

Como comenté ayer, la mayor parte de la Comunidad del Software Libre se está uniendo a la celebración de los 25 años de KDE, contribuyendo a los eventos, conferencias, vídeos, camisetas especiales o lanzamientos de software conmemorativos.

De esta forma me complace anunciar que ya está á disponible Plasma 5.23 para Debian 11, y que para conseguir estar a la última simplemente debéis seguir las instrucciones que nos indican en el blog de Norbert Preining’s.

Así que ya tenemos una extensa lista de distribuciones que han dado el salto a esta versión del escritorio Plasma (KDE Neon, ArchLinux y derivadas, Kubuntu, OpenSUSE Thumbleweed, PCLinuxOS, FreeBSD Ports, etc).

Es más, esta circunstancia ha hecho que conozca una página que nos informa sobre la versión Plasma disponible en un increíble número de distribuciones: https://repology.org/project/plasma-desktop/versions

Más información: There and Back Again

Las novedades básicas de Plasma 5.22

Aún tengo pendiente el resumen de las novedades pero he aquí la lista preliminar de ellas.

• Nuevo fondo para el escritorio
• Nuevo tema: Brisa — Océano azul que mejora el aspecto de Plasma y aclara su cometido.
• Nuevos efectos visuales permanentes: los elementos activos de una ventana de diálogo, por ejemplo, se «iluminan» cuando la ventana obtiene el foco, las casillas de verificación muestran marcas reales y los botones de radio se encienden como bombillas.
• Las barras de desplazamiento y los controles numéricos son más grandes, haciéndolos más accesibles y fáciles de usar en las pantallas táctiles
• Nueva opción que le permite escoger los colores de acentuación del escritorio.
• Diálogo con una cuenta atrás al cambiar la resolución de pantalla.
• Nuevas opciones para Activar, desactivar o recordar el estado del Bluetooth.
• Reescrito grandes partes del código del lanzador de aplicaciones para hacerlo más rápido y fácil de usar.
• Mejoras en la bandeja del sistema.
• Mejoras en Wayland.

Más información: KDE

The past week produced two openSUSE Tumbleweed snapshots and both included a lot of updates for users of KDE.

Plasma, Gear and Frameworks weren’t the only packages to update in the snapshots.

Snapshot 20211019 offered quite a bit of updated packages. Remote access package remmina 1.4.21 provided updates for the GNOME 40 runtime and made some backward compatibility with WebKit versions under 2.32.0. The refreshable braille display package brltty updated to version 6.4 and made sysusers.d a new package while also removing some old SUSE RPM constructs. The PDF rendering library poppler 21.10.0 fixed the rendering of some odd splash patterns and added support for setting custom stamp annotations. Mesa 21.2.4 had several fixes and rolled out patches for both PowerPC and LLVM i386 compiling. A package to get a major update in the snapshot was rdma-core 37.1, which focuses on userspace components for the Linux Kernel’s drivers/infiniband subsystem; this new major version fixed cmake flags to correct paths for pkg-config. KDE’s 5.87.0 Frameworks version had an enormous amount of updated packages in the snapshot. Among the updates was the removal of defunct Python and Ruby script engines with Plasma Framework. The 5.87.0 Breeze Icons package added 22px variants of the preferences icons and the same version of the UI framework package Kirigami fixed Breadcrumb Control on mobile when using layers.

The previous snapshot this week, 20211016, updated the 25th Anniversary Edition of KDE’s Plasma; the 5.23.0 version dives into a cool blue default color scheme and provides System Settings that lets users pick accent colors to make their system pop out with a personal touch. The Kickoff launcher for Plasma, which is located in the bottom left-hand corner of the screen, had some large rewritten code to make it faster and easier to use. Plasma also made changes to let users drag-and-drop stuff between native Wayland and XWayland apps. KDE’s 21.08.2 version of Gear restored the ability to quickly switch between “Zoom to Fit” and “Actual Size” with a keyboard shortcut for the App Gwenview and video editing app Kdenlive made a minor timing delay for startup crash detection. The sound editing app Audacity updated to version 3.0.5 had a single bug fix and gimp 2.10.28 improved the dark theme by adding a white border on the mouse-hover. The graphics editor also set a light background for selected text in layers and removed a 3D shadow box. The updated version of git 2.33.1 fixed git pull and git rebase -r for various corner cases and bugs. LibreOffice 7.2.2.2 updated translations and the container command-line utility package skopeo updated to version 1.5.0, which introduced an --ignore option to allow sync command to continue syncing even after a particular image sync fails. Other packages to update in the snapshot were GNOME’s video player totem 3.38.2, python-kiwi 9.24.0 and several 4.2.17 versions of the widget abstraction library libyui.

Me complace compartir con todos vosotros que ya está disponible Plasma 5.23 para Kubuntu 21.10. Recordemos que el escritorio de la Comunidad KDE apareció la semana pasada y que esta misma semana acaba de recibir su primera gran actualización. En distribuciones como KDE Neon o ArchLinux también está disponible (de momento, que yo sepa).

Disponible Plasma 5.23 para Kubuntu 21.10

He de reconocer que no me esperaba (pero me alegra) que la gente de la Comunidad Kubuntu pudiera ponerse al día con tanta rapidez con la última versión de Plasma 5.23. Estoy seguro que esto es para conmemorar el 25 aniversario de la Comunidad KDE que todo el mundo del Software Libre está celebrando.

En otras palabras, ayer 20 de octubre la Comunidad Kubuntu realizó el anuncio de que ya está disponible Plasma 5.23.1 para Kubuntu 21.10.

Esto significa cambiar los repositorios inicialmente preparados para un entorno de escritorio, lo cual puede afectar un poco a lo programado inicialmente por los desarrolladores, y por este motivo, el usuario que quiera tener el último escritorio de la Comunidad KDE debe realizar algunas acciones no habituales y, por tanto, un poco más elaboradas.

De esta forma, para tenerlo en nuestro ordenador debemos realizar básicamente tres pasos: añadir un repositorio, actualizar los repositorios y actualizar el sistema.

En otras palabras y paso a paso se trata de:

1. Abrir una sesión de consola
2. Escribir en la consola: $sudo add-apt-repository ppa:kubuntu-ppa/backports
3. Poner la contraseña de superusuario.
4. Escribir en la consola: $ sudo apt update
5. Escribir en la consola: $ sudo apt  full-upgrade
6. Y ya lo tenemos.

Lamentablemente no lo he probado, y no tengo tiempo para hacerlo, así­ que no os puedo decir que tal funciona, pero creo irá bastante bien.

Además, creo que no está de más recordar como nos cuenta la Comunidad Kubuntu, la Comunidad KDE tiene programadas más versiones de corrección de errores programadas para Plasma 5.23, por lo que, aunque creen que estos backports serán beneficiosos para los entusiastas, los usuarios que deseen utilizar una versión de Plasma más estable que se queden con Plasma 5.22 como se incluye en la versión original 21.10 (Impish Indri).

Más información: Kubuntu

El comando uniq sirve para encontrar texto duplicado adyacente en un archivo

os comandos de GNU coreutils son las herramientas básicas de nuestros sistemas GNU con kernel Linux similares a las de sistemas Unix, para gestionar desde la línea de comandos el sistema y operaciones con archivos de texto.

Vamos a repasar las opciones y uso del comando uniq que sirve para encontrar cadenas de texto duplicado en un archivo y mostrar diferentes resultados.

Antes de empezar para que el comando uniq considere a un texto repetido, este debe estar en líneas adyacentes, consecutivas, juntas o como quieras llamarles.

Es por eso que normalmente se utiliza junto con el comando sort que lo que hace es justamente ordenar el texto y queda así agrupado por líneas similares. (¿os apetece un próximo artículo sobre ese comando?)

Ejecutando uniq

Consideremos que tenemos un archivo que tiene las siguientes 11 líneas. Vemos el contenido con el comando cat:

cat archivo.txt                           
Yo vivo en América
Yo vivo en África
Yo vivo en Oceanía
Yo vivo en Europa
Yo Vivo en Asia
Yo vivo en la luna
Yo vivo en Europa
Yo vivo en Marte
Yo vivo en Asia
Yo vivo en el Polo Norte
Yo vivo en europa

Podemos ordenarlo con el comando sort y ejecutar el comando uniq sin más argumentos. Y nos muestra 10 líneas únicas, ya que “Yo vivo en Europa” está duplicado:

sort archivo.txt | uniq         
Yo vivo en África
Yo vivo en América
Yo vivo en Asia
Yo Vivo en Asia
Yo vivo en el Polo Norte
Yo vivo en europa
Yo vivo en Europa
Yo vivo en la luna
Yo vivo en Marte
Yo vivo en Oceanía

Ignorando mayúsculas y minúsculas

De manera predeterminada, el comando uniq de GNU coreutils diferencia entre mayúsculas y minúsculas. Podemos impedir esa diferenciación con la opción -i lo que nos deja 8 resultados:

sort archivo.txt | uniq -i             
Yo vivo en África
Yo vivo en América
Yo vivo en Asia
Yo vivo en el Polo Norte
Yo vivo en europa
Yo vivo en la luna
Yo vivo en Marte
Yo vivo en Oceanía

Número de ocurrencias encontradas

El comando uniq nos puede mostrar el número de ocurrencias encontradas para cada cadena con la opción -c y además con la opción de sin tener en cuenta la diferencia entre mayúsculas y minúsculas:

sort archivo.txt | uniq -ic     
      1 Yo vivo en África
      1 Yo vivo en América
      2 Yo vivo en Asia
      1 Yo vivo en el Polo Norte
      3 Yo vivo en europa
      1 Yo vivo en la luna
      1 Yo vivo en Marte
      1 Yo vivo en Oceanía

Mostrar solo las líneas duplicadas

Podemos hacer que uniq nos muestre únicamente las líneas que ha encontrado duplicadas con la opción -d

sort archivo.txt | uniq -id        
Yo vivo en Asia
Yo vivo en europa

Solo muestra una vez cada una de las línea duplicadas, es decir, si hay 5 líneas iguales, solo muestra 1 como muestra. Si quieres ver todas las líneas duplicadas, no solo una por cada grupo utiliza la opción -D

sort archivo.txt | uniq -iD        
Yo vivo en Asia
Yo Vivo en Asia
Yo vivo en europa
Yo vivo en Europa
Yo vivo en Europa

Filtrando los caracteres o campos a comprobar

De manera predeterminada uniq comprueba la totalidad de las líneas para compararlas, pero quizás debido a la manera en la que guardamos los datos en nuestro archivo queremos filtrar el texto que queremos que compare.

Compararemos caracteres con la opción -w y un número que indicará los primeros caracteres a comparar. Si buscamos las líneas iguales ignorando mayúsculas y minúsculas de los 7 primeros caracteres lo haremos:

sort archivo.txt | uniq -iw 7     
Yo vivo en África

Solo encuentra una ocurrencia, porque todas empiezan con el mismo texto.

Ahora en vez de decirle que compruebe los primeros 7 caracteres, vamos a decirle que ignore los primeros 11 caracteres y empiece a comparar con el resto mediante la opción -s

sort archivo.txt | uniq -cs 11     
      1 Yo vivo en África
      1 Yo vivo en América
      2 Yo vivo en Asia
      1 Yo vivo en el Polo Norte
      1 Yo vivo en europa
      2 Yo vivo en Europa
      1 Yo vivo en la luna
      1 Yo vivo en Marte
      1 Yo vivo en Oceanía

En vez de indicar el número de caracteres a ignorar, también podemos hacerlo por campos. Un campo es un grupo de caracteres separados por un espacio en blanco. Con la opción -f vamos a indicarle al comando que ignore los primeros 3 campos. Lo que dará un resultado idéntico al anterior:

sort archivo.txt | uniq -cf 3          
      1 Yo vivo en África
      1 Yo vivo en América
      2 Yo vivo en Asia
      1 Yo vivo en el Polo Norte
      1 Yo vivo en europa
      2 Yo vivo en Europa
      1 Yo vivo en la luna
      1 Yo vivo en Marte
      1 Yo vivo en Oceanía

Estás serían las maneras de utilizar este comando escrito para GNU por Richard M. Stallman y David MacKenzie. Quizás útil a la hora de incluirlo en algún script. ¿Te ha parecido instructivo?

Up to now, text support in librsvg has been fairly limited. The text chapter in the SVG spec is pretty big and it contains features that are very much outside of my experience (right-to-left languages, vertical text). But now I think I have a plan for how to improve the text features.

Bidirectional text

Bidirectional text, or "bidi" for short, happens when one has a right-to-left language like Arabic or Hebrew intermixed with things like Arabic numerals, or Latin text.

This is an interesting little example from the SVG spec:

<?xml version="1.0" encoding="utf-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
     width="100%" height="100%" viewBox="0 0 600 72"
     direction="rtl" xml:lang="he">

  <title direction="ltr" xml:lang="en">Right-to-left Text</title>
  <desc direction="ltr" xml:lang="en">
    An example for using the 'direction' and 'unicode-bidi' properties
    in documents that predominantly use right-to-left languages.
  </desc>

  <text x="300" y="50" text-anchor="middle" font-size="36"> כתובת MAC:&#x200F;
    <tspan direction="ltr" unicode-bidi="embed">00-24-AF-2A-55-FC</tspan> 
  </text>

</svg>

It is supposed to render like this; it says "MAC address: xxx-yyy-zzz":

However, until librsvg 2.52.2 this did not work correctly. You see, librsvg takes each piece of text inside a <text> and creates a Span for it. If there is a <tspan> element, it also gets a Span created. However, librsvg did not handle the case where an inner <tspan> changes the direction property from its parent <text> element.

Another bug:

<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xml:lang="fa" direction="rtl" width="200" height="100">
    <g font-family="Dana-FaNum" font-size="16">
        <text x="50%" y="50%"><tspan font-weight="bold">نام: </tspan><tspan>مهدی</tspan></text>
    </g>
</svg>

It is supposed to render more or less like this (the line says, "Name: Mehdi"):

This was broken, too!. I did not realize that within <text>, the spans need to be laid out exactly in the text direction, in this case rtl. I thought that it was sufficient to let Pango deal with each group of characters within a span.

Together with that, I fixed text-anchor for right-to-left text. This is what one uses to align a text block with respect to its start, middle, or end. It was also broken for RTL languages before.

One remaining bug is that there should be a space to the left of the :, but librsvg is eating it. I think it needs to implement the SVG2 white-space behavior, which replaces the ambiguous definition of xml:space from SVG1.1.

A test that makes me exceedingly happy

This family of images from Wikimedia Commons has been broken for many years. Someone has been painstakingly making SVG2 maps of German rail by writing SVG code by hand; they use features that became available only after Madds Holland's Outreachy internship from last Summer. Also, the maps use text offsetting features that were just broken.

The little arrowheads were broken because librsvg didn't support orient="auto-start-reverse" in markers (arrowheads get done with SVG's "marker" feature). Madds added support for that during their Outreachy internship.

Then, most of the the multi-line text objects in there were laid out incorrectly. Librsvg was laying out spans inside text chunks incorrectly when they had dx/dy offsets and the text also had text-anchor different from the default.

Previously:

Now:

One last detail which I haven't figured out is that the positioning of glyphs is a bit off. If you look at the slanted label for the LU-Rheingönheim station, the "heim" at the end looks a bit misaligned with the rest of the label. Firefox renders it correctly. Advice is appreciated!

Future plans

All those fixes will appear in librsvg 2.52.3, due in a few days.

I want to add more tests for right-to-left and bidi text; they can be affected by many properties for which there are no tests right now.

After bidi text works reasonably well, I want to add support for positioning individual glyphs with the x/y/dx/dy properties. People from Wikimedia Commons really want this, to be able to lay out equations and such.

Once individual glyphs can be positioned independently, maybe textPath support, which cartographers really like for curved labels.

A handy tool

I recently discovered the Ahem font. It is part of the CSS fonts tests, and consequently part of the Web Platform Tests. Quoting from its description:

The font’s em-square is exactly square. Its ascent and descent combined is exactly the size of the em square; this means that the font’s extent is exactly the same as its line-height, meaning that it can be exactly aligned with padding, borders, margins, and so forth.

That is, you can typeset something with the Ahem font and it will appear as contiguous squares. If you render the string ABCD, it will appear as a rectangle four-ems wide by one-em tall. This is wonderful for reproducible tests!

<?xml version="1.0" encoding="utf-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="600" height="300">
  <rect x="0" y="0" width="100%" height="100%" fill="white"/>
  <line x1="50%" y1="0" x2="50%" y2="100%" stroke-width="2" stroke="blue"/>
  <line x1="0" y1="100" x2="100%" y2="100" stroke-width="2" stroke="blue"/>

  <text x="50%" y="100" font-family="Ahem" font-size="20" text-anchor="end"><tspan dy="-1em">A</tspan><tspan dx="-40" dy="1em">BC</tspan></text>
</svg>

That renders like this:

Text rendering clearly means spending days trying to get three black squares to draw in the correct position.

Hoy tengo poco tiempo y nada en el congelador de noticias, así que echo mano de un clásico los temas de iconos para tu escritorio. Hoy tengo el gusto de presentaros Grade icon theme, un tema de iconos que destaca por su colorido diseño que combinan perfectamente tanto con temas claros como oscuros.

Grade icon theme, iconos coloridos para Plasma

Para el escritorio Plasma de la Comunidad KDE hay cientos de temas de todo tipo disponibles para los usuarios: iconos, cursores, emoticonos, etc, Y como me gusta cambiar de vez en cuando, en el blog le he dedicado muchos artículos a cada uno de los packs.

De la mente y la mano digital de mayurzambare nos llega un nuevo tema de iconos que recibe el nombre de Grade icon theme, un tema que encaja a la perfección con cualquier tipo de escritorio, bien sea claro u oscuro por su colorido y desenfadado estilo.

Por cierto, no es la primera vez que este diseñador aparece en el blog, hace relativamente poco hablé de otro de sus temas, concretamente de Beauty, un precioso tema de iconos para tu Plasma que habitó en mi portátil hace unos meses.

Y como siempre digo, si os gusta el pack de iconos podéis “pagarlo” de muchas formas en la nueva página de KDE Store, que estoy seguro que el desarrollador mayurzambare lo agradecerá: puntúale positivamente, hazle un comentario en la página o realiza una donación. Ayudar al desarrollo del Software Libre también se hace simplemente dando las gracias, ayuda mucho más de lo que os podéis imaginar, recordad la campaña I love Free Software Day 2017 de la Free Software Foundation donde se nos recordaba esta forma tan sencilla de colaborar con el gran proyecto del Software Libre y que en el blog dedicamos un artículo.

Más información: KDE Store