Ish Sookun posted at 17:25

Where to obtain COVID-19 updates?

Six days ago the Prime Minister of Mauritius, P. K. Jugnauth, announced to the population that we have recorded the first cases of COVID-19 in Mauritius. Since then much has happened.

The number of COVID-19 cases jumped from 3 less than a week ago to become 42 today.

A National Communication Committee on COVID-19 was set up to provide daily updates on MBC channels at 11h30. The committee provides information on the number of positive COVID-19 cases being identified, status of people in quarantine, additional measures being taken by the government, etc.

I suggest that the one source of information to trust right now should be the National Communication Committee.

Media outlets are picking bits of information from the daily press conference and reporting those individually. That's fine but refrain from re-sharing or posting un-trusted or un-verified information on social media networks. People claiming on social media that hospital staff or relatives working there are giving alarming facts are NOT FACTS! They just contribute to the pile of fake news and rumours.

The Ministry of Health and Wellness updates covid19.mu which is a page that provides the number of active COVID-19 cases in Mauritius and health advice to stay safe.

My developer friends published COVID-19 Mauritius, a webpage that provides COVID-19 statistics for Mauritius. Their source is the local newspapers and radio stations. The source code of the webpage is available on GitHub and contributors are most welcome.

L'express has a dossier of articles on COVID-19.

Update

The Ministry of Health and Wellness together with Mauritius Telecom launched another website, besafemoris.mu, to provide updates on COVID-19 in Mauritius.

The website provides news updates and communiques issued by the ministry. It has a map of health services currently operational.

beSafeMoris is also available as a mobile app for Android and iOS. At the time of writing this post it wasn't available on the Google Play Store though. Instead, the apk file was downloadable from the beSafeMoris website. The app is available on Apple's App Store.

Multi-line-timeout: making sure your last multi-line message is not lost

When your application has a problem that it cannot handle, then Java, PHP and other environments often write multi-line error messages. These long messages include many information useful for developers but it might be difficult to handle for logging systems. A multi-line log message is usually saved when the beginning of the next message is detected, which made handling the last message problematic.

This is where multi-line-timeout can help. After a pre-configured timeout, the last message is saved by syslog-ng. Losing a message when there are thousands of them is probably not a big deal. But if you are lucky, the error messages are rare. Maybe just a single message in a file. In either case, you can lose most or all of the messages. If you do not want to lose these messages, configure multi-line-timeout.

Note: this problem does not affect some less common multi-line modes, like multi-line-garbage or multi-line-suffix.

In this blog, you will find a simple configuration and a test to see how multi-line-timeout works. You can use it as a basis for your own use case.

Preparations for testing multi-line-timeout

It is hard to predict when your Java application writes a stack trace, so instead of that I prepared you an easy way to test multi-line-timeout using synthetic log messages. All you need for testing:

  • syslog-ng 3.26 (or later) installed

  • two open terminal windows

  • a browser window, from where you can copy & paste commands

If your operating system does not have syslog-ng 3.26 (or later) available, check our third party repositories page. On most Linux distributions, you can create configuration snippets under /etc/syslog-ng/conf.d with a .conf extension. Otherwise append the following to syslog-ng.conf:

source s_multi {
    file("/var/log/events"
        multi-line-mode("prefix-garbage")
        multi-line-prefix('^EVENT: ')
        multi-line-timeout(10)
        flags("no-parse")
    );
};

destination d_multi {
    file("/var/log/multi");
};

log {source(s_multi); destination(d_multi); };

The above configuration expects multi-line messages to arrive into a file called /var/log/events and writes the results to /var/log/multi. These files do not exist by default, so you should create them:

touch /var/log/multi /var/log/events

This way you can follow log output right from the beginning. Once you restarted / reloaded syslog-ng and the configuration took effect, you are ready for testing.

Testing

For testing you will need two terminal windows and a browser window. From the browser, you will need to copy & paste commands into one of the terminal windows. You will use the second terminal window to follow the content of /var/log/multi, the file where syslog-ng saves multi-line logs when using the above configuration. In the first terminal, enter the following command, which you can then follow in the second terminal window:

tail -f /var/log/multi

Use the commands below in the first terminal one after the other as follows. First, copy and paste the first three lines from the commands below to the first terminal and hit enter. You will see, that the first two lines are displayed immediately. This is because there syslog-ng knows for sure that the log messages arrived successfully as the beginning pattern of the third message is also already there. The third message only arrives after the 10 seconds time-out, as there is no next message there (yet), which could indicate that the previous message is over.

Finally, copy & paste the last command to the first terminal and hit enter. After ten seconds, you should see the log message on the second terminal.

echo EVENT: bla1 >> /var/log/events ; echo bla 1 >> /var/log/events ; echo blabla 1 >> /var/log/events
echo EVENT: bla2 >> /var/log/events ; echo bla 2 >> /var/log/events ; echo blabla 2 >> /var/log/events
echo EVENT: bla3 >> /var/log/events ; echo bla 3 >> /var/log/events ; echo blabla 3 >> /var/log/events
echo EVENT: bla4 >> /var/log/events ; echo bla 4 >> /var/log/events ; echo blabla 4 >> /var/log/events

What is next?

As you could see from the above using multi-line-timeout resolves problems around reading multi-line log messages from files. Now it is time to custom tailor the above configuration example to your environment.

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.

eLearning mit Feedback

Lange wurde über eLearning diskutiert, aber wenig gemacht. Jetzt, zu Zeiten der Corona Krise, brauchen wir dringend eine Lösung.

Dieser Blog soll zeigen, was diesbezüglich mit einer normalen ownCloud Installation zu erreichen ist, ohne dass langes Kennenlernen der Plattform für alle Beteiligten nötig ist.

Es wird eine wirklich einfache Variante gezeigt, die schnell an den Start gebracht werden kann, mit der aber schon erstaunlich viel möglich ist.

Das interessante ist, dass nicht nur Inhalte zur Verfügung gestellt werden können (das ist, was auf Youtube passiert), sondern dass es zusätzlich einen Weg für Schüler gibt, ihre Ergebnisse zur Ansicht und Korrektur zurückzuliefern.

Darüberhinaus gibt es noch eine einfache Dialog-Möglichkeit, um einzelne Dateien zu kommentieren.

Damit besteht eine Feedback Schleife zwischen Schüler und Lehrer, die natürlich nicht an persönlichen Unterricht heranreicht, aber besser als z.B. Mailkommunikation ist.

Dieses System kann nicht nur Schul-Lehrern helfen, sondern auch anderen Berufsgruppen wie Instrumenten-Lehrern oder Physiotherapeuten, die damit den Patienten per Video etwas zeigen und dann Hinweise auf Basis der zurückgeschickten Dateien geben können.

ownCloud als Trainings-Platform

ownCloud ist eine open source Infrastruktur zum Betreiben von sog. privaten Cloud-Speicher, mit denen Daten zwischen Rechnern synchronisiert und mit anderen sehr einfach geteilt werden können.

Mit seinen Möglichkeiten zur Zusammenarbeit bietet es alles, was für die Aufgabenstellung eines einfachen eTrainings notwendig ist.

Dateien teilen

ownCloud stellt als zentralen Baustein der ganzen Idee das sog. *File Sharing* zur Verfügung. Das bedeutet, dass ein Benutzer in der Cloud einem anderen sehr einfach Zugriff auf seine Dateien geben kann, ohne die zu kopieren oder hin- und herzuschicken.

sharing

Frau Teacher teilt ein Verzeichnis mit Schülerin Felizitas

Dazu legt der Lehrer in seiner ownCloud für jeden seiner Schüler ein Verzeichnis an, in das Dokumente oder Videos für einen Schüler gespeichert werden. Mittels des *File Sharings* kann der Lehrer dieses Verzeichnis nun per Klick mit dem entsprechenden Schüler teilen. Das bedeutet, dass das Verzeichnis des Lehrers im ownCloud-Zugang des Schülers „auftaucht“.

Der Schüler hat damit Zugriff auf den Inhalt, den der Lehrer für ihn zur Verfügung gestellt hat. Er kann Dokumente herunterladen und zB. Videos ansehen.

Mit ownClouds Fähigkeit, Benutzer in Gruppen zu organisieren, können Inhalte genauso einfach zB. gleich für eine ganze Klasse zur Verfügung gestellt werden. Dazu muss das entsprechende Verzeichnis nicht mit einem individuellen Benutzer wie Felizitas geteilt werden, sondern mit einer Gruppe, in der zum Beispiel alle Geigenschüler zusammengefasst sind.

Feedback

kommentare

Kommentarfunktion

Nun ist es aber auch möglich, dass der Schüler eine Datei in den vom Lehrer geteilten Folder zurücklegt. Das geschieht durch Hochladen einer Datei in das Verzeichnis. Das kann z.B. ein bearbeitetes und gescanntes Papier sein oder auch ein kurzes Video.

Der Lehrer kann nun nachvollziehen, was der Schüler mit den Lerninhalten erreicht hat und seinerseits Feedback geben.

Kommentare

Ein in diesem Zusammenhang nützliches Feature ist die Kommentar-Funktion, die ownCloud auf Datei-Ebene zur Verfügung stellt. Damit können von jedem Benutzer Kommentare zu einer Datei geschrieben werden.

Im Falle der geteilten Inhalte ist dann ein einfacher Dialog um die verschiedenen Dateien leicht möglich.

Ausblick

Dies ist erst der Anfang – ownCloud hat als erfolgreiches open source Projekt noch eine Vielzahl von mehr Möglichkeiten, aber mit diesen wenigen Schritten sollte schon ein rudimentärer eTraining Ablauf möglich sein, der Schüler und Lehrer schnell und relativ unkompliziert wieder zusammenbringt, wenn persönlicher Kontakt nicht möglich ist.

Weitere Features wie File-Tags, Gruppenmanagement, Ablaufdaten etc. können im weiteren Verlauf hinzugenommen werden.

Leap 15.2 Beta: mariadb upgrade woes

I'm running a server at home with openSUSE Leap, and since Leap 15.2 is now in beta, I thought it was a good idea to update and take part in the beta testing effort.
This machine is running an Owncloud instance, serving some internal NFS shares and used as a development machine for (cross-)compiling stuff, packaging etc.

The update went pretty smooth, only the mariadb instance used by Owncloud was hosed afterwards. There was no login possible and generally database users were gone.
Fortunately, I always have recent backups available, both a mysqldump and a complete file system backup.

So I tried to just restore the mysqldump on the updated database. This did not work, Bug#1166786.
Then I did just restore the filesystem backup of /var/lib/mysql and the database worked again.
Unfortunately, as I found out reproducing and investigating the issue, it would just get killed again by the next update, Bug#1166781. (Extra kudos to openSUSE Product Management which decided that this is not a bug, but instead regarded a feature!).

Finally I found the upstream bug in mariadb JIRA bugtracker, (which also does not look like there is much interest in fixing this), but with the information given there, I was able to fix this for me.

So all of you who are stuck with
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
after updating a mariadb instance to 10.4, this might help:

  1. restart mysqld with option --skip-grant-tables, to disable user authentication
  2. in the mysql database, execute
  • ALTER TABLE user CHANGE COLUMN `auth_string` `authentication_string` text;
  • DROP TABLE global_priv;
  • now run the mariadb-upgrade command
  • restart mariadb.service with default options
  • This fixed my instance and owncloud is working again.

    Note that I am by no means a database expert. Take backups before performing these steps.

    Saturday

    21 March, 2020

    Ish Sookun posted at 16:58

    How will people cope with the COVID-19 confinement financially?

    People of Mauritius are under a nationwide confinement since 19 March 2020. This confinement is to last for about two weeks, during which people are to remain at home and special permits are given to certain essential services workers to be able to travel only between their home and workplace.

    As per the Cabinet Decisions of 20 March 2020, a COVID-19 Wage Support Scheme will provide financial support to  employees who would become technically unemployed on a temporary basis due to the impact of COVID-19. The scheme will cover formal sectors (mainly Travel and Tourism Sector, Export Oriented Enterprises, ICT/BPO Sector and SMEs). This support will be extended to other formal sectors of the economy.

    I could not find a proper reference on the Government portal that highlights what are formal sectors but I assume they mean companies duly registered and operating within a defined industry (like the examples given above, Tourism, ICT/BPO etc).

    Therefore, I also assume that my neighbourhood dhollpuri seller, carpenters, construction workers, plumbers and labourers will not benefit from the COVID-19 Wage Support Scheme.

    My self-employed friends will be left on their own to pay their rents, utility bills and mortgages.

    Confinement means no work, which means no income for them. How will they cope financially during this confinement period? What happens if the confinement is extended after the two weeks?

    I am a proponent of this confinement which I feel is a necessary step to stop the propagation of COVID-19 on the island. I know the economy will be severely hit but I am also putting my trust in the capable authorities to sustain its people and help them cope financially.

    I sincerely hope that the authorities extend the support to more people currently affected by the COVID-19 confinement.

    Reducing memory consumption in librsvg, part 2: SpecifiedValues

    To continue with last time's topic, let's see how to make librsvg's DOM nodes smaller in memory. Since that time, there have been some changes to the code; that is why in this post some of the type names are different from last time's.

    Every SVG element is represented with this struct:

    pub struct Element {
        element_type: ElementType,
        element_name: QualName,
        id: Option<String>,
        class: Option<String>,
        specified_values: SpecifiedValues,
        important_styles: HashSet<QualName>,
        result: ElementResult,
        transform: Transform,
        values: ComputedValues,
        cond: bool,
        style_attr: String,
        element_impl: Box<dyn ElementTrait>,
    }
    

    The two biggest fields are the ones with types SpecifiedValues and ComputedValues. These are the sizes of the whole Element struct and those two types:

    sizeof Element: 1808
    sizeof SpecifiedValues: 824
    sizeof ComputedValues: 704
    

    In this post, we'll reduce the size of SpecifiedValues.

    What is SpecifiedValues?

    If we have an element like this:

    <circle cx="10" cy="10" r="10" stroke-width="4" stroke="blue"/>
    

    The values of the style properties stroke-width and stroke get stored in a SpecifiedValues struct. This struct has a bunch of fields, one for each possible style property:

    pub struct SpecifiedValues {
        baseline_shift:              SpecifiedValue<BaselineShift>,
        clip_path:                   SpecifiedValue<ClipPath>,
        clip_rule:                   SpecifiedValue<ClipRule>,
        /// ...
        stroke:                      SpecifiedValue<Stroke>,
        stroke_width:                SpecifiedValue<StrokeWidth>,
        /// ...
    }
    

    Each field is a SpecifiedValue<T> for the following reason. In CSS/SVG, a style property can be unspecified, or it can have an inherit value to force the property to be copied from the element's parent, or it can actually have a specified value. Librsvg represents these as follows:

    pub enum SpecifiedValue<T>
    where
        T: // some trait bounds here
    {
        Unspecified,
        Inherit,
        Specified(T),
    }
    

    Now, SpecifiedValues has a bunch of fields, 47 of them to be exact — one for each of the style properties that librsvg supports. That is why SpecifiedValues has a size of 824 bytes; it is the largest sub-structure within Element, and it would be good to reduce its size.

    Not all properties are specified

    Let's go back to the chunk of SVG from above:

    <circle cx="10" cy="10" r="10" stroke-width="4" stroke="blue"/>
    

    Here we only have two specified properties, so the stroke_width and stroke fields of SpecifiedValues will be set as SpecifiedValue::Specified(something) and all the other fields will be left as SpecifiedValue::Unspecified.

    It would be good to store only complete values for the properties that are specified, and just a small flag for unset properties.

    Another way to represent the set of properties

    Since there is a maximum of 47 properties per element (or more if librsvg adds support for extra ones), we can have a small array of 47 bytes. Each byte contains the index within another array that contains only the values of specified properties, or a sentinel value for properties that are unset.

    First, I made an enum that fits in a u8 for all the properties, plus the sentinel value, which also gives us the total number of properties. The #[repr(u8)] guarantees that this enum fits in a byte.

    #[repr(u8)]
    enum PropertyId {
        BaselineShift,
        ClipPath,
        ClipRule,
        Color,
        // ...
        WritingMode,
        XmlLang,
        XmlSpace,
        UnsetProperty, // the number of properties and also the sentinel value
    }
    

    Also, since before these changes there was the following monster to represent "which property is this" plus the property's value:

    pub enum ParsedProperty {
        BaselineShift(SpecifiedValue<BaselineShift>),
        ClipPath(SpecifiedValue<ClipPath>),
        ClipRule(SpecifiedValue<ClipRule>),
        Color(SpecifiedValue<Color>),
        // ...
    }
    

    I changed the definition of SpecifiedValues to have two arrays, one to store which properties are specified, and another only with the values for the properties that are actually specified:

    pub struct SpecifiedValues {
        indices: [u8; PropertyId::UnsetProperty as usize],
        props: Vec<ParsedProperty>,
    }
    

    There is a thing that is awkward in Rust, or which I haven't found how to solve in a nicer way: given a ParsedProperty, find the corresponding PropertyId for its discriminant. I did the obvious thing:

    impl ParsedProperty {
        fn get_property_id(&self) -> PropertyId {
            use ParsedProperty::*;
    
            match *self {
                BaselineShift(_) => PropertyId::BaselineShift,
                ClipPath(_)      => PropertyId::ClipPath,
                ClipRule(_)      => PropertyId::ClipRule,
                Color(_)         => PropertyId::Color,
                // ...
            }
        }
    }
    

    Initialization

    First, we want to initialize an empty SpecifiedValues, where every element of the the indices array is set to the sentinel value that means that the corresponding property is not set:

    impl Default for SpecifiedValues {
        fn default() -> Self {
            SpecifiedValues {
                indices: [PropertyId::UnsetProperty.as_u8(); PropertyId::UnsetProperty as usize],
                props: Vec::new(),
            }
        }
    }
    

    That sets the indices field to an array full of the same PropertyId::UnsetProperty sentinel value. Also, the props array is empty; it hasn't even had a block of memory allocated for it yet. That way, SVG elements without style properties don't use any extra memory.

    Which properties are specified and what are their indices?

    Second, we want a function that will give us the index in props for some property, or that will tell us if the property has not been set yet:

    impl SpecifiedValues {
        fn property_index(&self, id: PropertyId) -> Option<usize> {
            let v = self.indices[id.as_usize()];
    
            if v == PropertyId::UnsetProperty.as_u8() {
                None
            } else {
                Some(v as usize)
            }
        }
    }
    

    (If someone passes id = PropertyId::UnsetProperty, the array access to indices will panic, which is what we want, since that is not a valid property id.)

    Change a property's value

    Third, we want to set the value of a property that has not been set, or change the value of one that was already specified:

    impl SpecifiedValues {
        fn replace_property(&mut self, prop: &ParsedProperty) {
            let id = prop.get_property_id();
    
            if let Some(index) = self.property_index(id) {
                self.props[index] = prop.clone();
            } else {
                self.props.push(prop.clone());
                let pos = self.props.len() - 1;
                self.indices[id.as_usize()] = pos as u8;
            }
        }
    }
    

    In the first case in the if, the property was already set and we just replace its value. In the second case, the property was not set; we add it to the props array and store its resulting index in indices.

    Results

    Before:

    sizeof Element: 1808
    sizeof SpecifiedValues: 824
    

    After:

    sizeof Element: 1056
    sizeof SpecifiedValues: 72
    

    The pathological file from the last time used 463,412,720 bytes in memory before these changes. After the changes, it uses 314,526,136 bytes.

    I also measured memory consumption for a normal file, in this case one with a bunch of GNOME's symbolic icons. The old version uses 17 MB; the new version only 13 MB.

    How to keep fine-tuning this

    For now, I am satisfied with SpecifiedValues, although it could still be made smaller:

    • The crate tagged-box converts an enum like ParsedProperty into an enum-of-boxes, and codifies the enum's discriminant into the box's pointer. This way each variant occupies the minimum possible memory, although in a separately-allocated block, and the container itself uses only a pointer. I am not sure if this is worth it; each ParsedProperty is 64 bytes, but the flat array props: Vec<ParsedProperty> is very appealing in a single block of memory. I have not checked the sizes of each individual property to see if they vary a lot among them.

    • Look for a crate that lets us have the properties in a single memory block, a kind of arena with variable types. This can be implemented with a bit of unsafe, but one has to be careful with the alignment of different types.

    • The crate enum_set2 represents an array of field-less enums as a compact bit array. If we changed the representation of SpecifiedValue, this would reduce the indices array to a minimum.

    If someone wants to dedicate some time to implement and measure this, I would be very grateful.

    Next steps

    According to Massif, the next thing is to keep making Element smaller. The next thing to shrink is ComputedValues. The obvious route is to do exactly the same as I did for SpecifiedValues. I am not sure if it would be better to try to share the style structs between elements.

    Reducción del consumo de memoria en librsvg, parte 2: SpecifiedValues

    Para continuar con el tema de la vez pasada, vamos a ver cómo reducir el tamaño en memoria de los nodos del DOM en librsvg. Desde entonces ha habido cambios en el código; por eso es que en este artículo los nombres de algunos tipos han cambiado con respecto al artículo anterior.

    Cada elemento del SVG se representa con esta estructura:

    pub struct Element {
        element_type: ElementType,
        element_name: QualName,
        id: Option<String>,
        class: Option<String>,
        specified_values: SpecifiedValues,
        important_styles: HashSet<QualName>,
        result: ElementResult,
        transform: Transform,
        values: ComputedValues,
        cond: bool,
        style_attr: String,
        element_impl: Box<dyn ElementTrait>,
    }
    

    Los dos campos más grandes son los que tienen tipos SpecifiedValues y ComputedValues. He aquí los tamaños de la estructura completa Element y los de esos tipos:

    sizeof Element: 1808
    sizeof SpecifiedValues: 824
    sizeof ComputedValues: 704
    

    En este artículo vamos a reducir el tamaño de SpecifiedValues.

    ¿Qué es SpecifiedValues?

    Si tenemos un elemento así:

    <circle cx="10" cy="10" r="10" stroke-width="4" stroke="blue"/>
    

    Los valores de propiedades de estilos stroke-width y stroke se guardan en un SpecifiedValues; esta estructura tiene un montón de campos, uno para cada propiedad de estilos:

    pub struct SpecifiedValues {
        baseline_shift:              SpecifiedValue<BaselineShift>,
        clip_path:                   SpecifiedValue<ClipPath>,
        clip_rule:                   SpecifiedValue<ClipRule>,
        /// ...
        stroke:                      SpecifiedValue<Stroke>,
        stroke_width:                SpecifiedValue<StrokeWidth>,
        /// ...
    }
    

    Cada campo es un SpecifiedValue<T> por la siguiente razón. En CSS/SVG, una propiedad de estilos puede estar no especificada, o ser inherit para forzar que se copie la propiedad del elemento padre, o un valor específico. Librsvg lo representa así:

    pub enum SpecifiedValue<T>
    where
        T: // algunos requerimientos de traits aquí
    {
        Unspecified,
        Inherit,
        Specified(T),
    }
    

    Ahora bien, SpecifiedValues tiene un montón de campos, 47 para ser exactos — uno por cada una de las propiedades de estilos que soporta librsvg. Por eso es que el tamaño de SpecifiedValues es de 824 bytes. Es la sub-estructura más grande dentro de Element, y sería bueno reducirle el tamaño.

    No todas las propiedades se especifican

    Volvamos a ver el pedacito de SVG de arriba.

    <circle cx="10" cy="10" r="10" stroke-width="4" stroke="blue"/>
    

    Aquí sólo se especifican dos de las propiedades, de modo que los campos stroke_width y stroke de SpecifiedValues van a quedar como SpecifiedValue::Specified(algo) y todos los demás van a quedar como SpecifiedValue::Unspecified.

    Sería bueno sólo almacenar los valores completos para las propiedades que están especificadas, y una bandera más pequeña para las propiedades que están sin especificar.

    Otra forma de representar el conjunto de propiedades

    Como hay un máximo de 47 propiedades por elemento (o más si librsvg añade soporte para adicionales), podemos tener un arreglito de 47 bytes. Cada byte contiene el índice en otro arreglo que sólo contiene el valor de una propiedad especificada, o un valor centinela para indicar que la propiedad no está especificada.

    Primero hice una enumeración que quepa en un u8 para todas las propiedades, más el valor centinela al final, que además nos da el número total de propiedades. El #[repr(u8)] nos garantiza que ese enum cabe en un byte.

    #[repr(u8)]
    enum PropertyId {
        BaselineShift,
        ClipPath,
        ClipRule,
        Color,
        // ...
        WritingMode,
        XmlLang,
        XmlSpace,
        UnsetProperty, // el número de propiedades y el valor centinela
    }
    

    Además, desde antes ya había este monstruo para representar "cuál propiedad" además del valor de la propiedad:

    pub enum ParsedProperty {
        BaselineShift(SpecifiedValue<BaselineShift>),
        ClipPath(SpecifiedValue<ClipPath>),
        ClipRule(SpecifiedValue<ClipRule>),
        Color(SpecifiedValue<Color>),
        // ...
    }
    

    Cambié la definición de SpecifiedValues para que tenga dos arreglos, uno que indica qué propiedades están especificadas, y otro sólo con las propiedades especificadas:

    pub struct SpecifiedValues {
        indices: [u8; PropertyId::UnsetProperty as usize],
        props: Vec<ParsedProperty>,
    }
    

    Hay una cosa que es incómoda en Rust, o que no he sabido resolver mejor: dada una ParsedProperty, hay que encontrar el PropertyId correspondiente para su discriminante. Puse lo obvio:

    impl ParsedProperty {
        fn get_property_id(&self) -> PropertyId {
            use ParsedProperty::*;
    
            match *self {
                BaselineShift(_) => PropertyId::BaselineShift,
                ClipPath(_)      => PropertyId::ClipPath,
                ClipRule(_)      => PropertyId::ClipRule,
                Color(_)         => PropertyId::Color,
                // ...
            }
        }
    }
    

    Inicialización

    Primero, queremos inicializar un SpecifiedValues vacío, en donde todo los elementos del arreglo indices están puesto al valor sentinela que indica que la propiedad correspondiente no está especificada:

    impl Default for SpecifiedValues {
        fn default() -> Self {
            SpecifiedValues {
                indices: [PropertyId::UnsetProperty.as_u8(); PropertyId::UnsetProperty as usize],
                props: Vec::new(),
            }
        }
    }
    

    Eso pone el campo indices a un arreglo lleno del valor sentinela PropertyId::UnsetProperty. Además, el arreglo props está vacío; ni siquiera se ha pedido un bloque de memoria para él. Así, los elementos del SVG que no tienen propiedades de estilos no ocupan memoria extra.

    ¿Qué propiedades están especificadas y cuáles son sus índices?

    Segundo, queremos una función que nos dé el índice en props de alguna propiedad, o que nos diga si esa propiedad no está especificada aún:

    impl SpecifiedValues {
        fn property_index(&self, id: PropertyId) -> Option<usize> {
            let v = self.indices[id.as_usize()];
    
            if v == PropertyId::UnsetProperty.as_u8() {
                None
            } else {
                Some(v as usize)
            }
        }
    }
    

    (Si alguien pasa id = PropertyId::UnsetProperty, el acceso al arreglo indices va a mandar un panic, que es lo que queremos, pues ese no es identificador válido para una propiedad.)

    Cambiar el valor de una propiedad

    Tercero, queremos poner el valor de una propiedad que no estaba especificada, o cambiar el valor de una que ya lo estaba:

    impl SpecifiedValues {
        fn replace_property(&mut self, prop: &ParsedProperty) {
            let id = prop.get_property_id();
    
            if let Some(index) = self.property_index(id) {
                self.props[index] = prop.clone();
            } else {
                self.props.push(prop.clone());
                let pos = self.props.len() - 1;
                self.indices[id.as_usize()] = pos as u8;
            }
        }
    }
    

    En el primer caso del if, la propiedad ya estaba puesta y nada más remplazamos su valor. En el segundo caso, la propiedad no estaba puesta; la añadimos al arreglo props y guardamos su índice resultante en indices.

    Resultados

    Antes:

    sizeof Element: 1808
    sizeof SpecifiedValues: 824
    

    Después:

    sizeof Element: 1056
    sizeof SpecifiedValues: 72
    

    El archivo patológico de la vez anterior consumía 463,412,720 bytes en memoria antes de estos cambios. Después de los cambios, consume 314,526,136 bytes.

    También medí el consumo de memoria de un un archivo normal, en este caso uno con muchos de los iconos simbólicos de GNOME. La versión anterior consume 17 MB; la versión nueva sólo 13 MB.

    Cómo seguir ajustando esto

    Por ahora, estoy satisfecho con SpecifiedValues, aunque todavía se podría hacer más pequeño:

    • El crate tagged-box convierte un enum como el ParsedProperty en un enum-de-boxes, y codifica el discriminante del enum en el puntero que apunta al box. De esta forma cada variante ocupa el mínimo posible de memoria, aunque se le suma un bloque de memora extra, y el contenedor en sí ocupa un solo puntero. No estoy seguro si valga la pena; cada ParsedProperty ocupa 64 bytes, pero el arreglo plano de props: Vec<ParsedProperty> queda muy lindo en un solo bloque de memoria. No he visto los tamaños de cada propiedad individual para ver si varían mucho entre sí.

    • Buscar un crate para poder tener las propiedades en un sólo bloque de memoria, una especie de arena de tipos variables. Esto se puede implementar con un poquito de unsafe, pero hay que tener cuidado con la alineación de los elementos de diferentes tipos.

    • El crate enum_set2 representa un arreglo de enums sin campos como un arreglo de bits compacto. Si se cambiara la representación de SpecifiedValue, esto reduciría el arreglo indices al mínimo.

    Si alguien quiere dedicarle tiempo a implementar y medir algo así, le estaría muy agradecido.

    Siguientes pasos

    Según Massif, lo siguiente es seguir haciendo que Element sea más pequeño. Lo que sigue de reducir de tamaño es ComputedValues. La opción obvia es hacerle exactamente lo mismo que a SpecifiedValues. No estoy seguro de si vale más la pena intentar compartir las estructuras de estilos entre varios elementos.

    openSUSE Tumbleweed – Review of the week 2020/11 & 12

    Dear Tumbleweed users and hackers,

    Last week I missed, for personal reasons, to write up the report. So, slacking in one week means I have to catch up the other week. Of course, you are all eager to hear/read what is happening in Tumbleweed. In the period since covered, we have released 7 Snapshots (0305, 0306, 0307, 0309, 0311, 0312 and 0314). The major changes were:

    • Linux kernel 5.5.7
    • Python 3.8.2, with a lot of python modules being updated
    • Mesa 20.0.1
    • KDE Applications 19.12.3
    • KDE Plasma 5.18.3

    Thins currently being staged or close to be shipped:

    • RPM: change of database format to ndb
    • Linux kernel 5.5.10
    • Qt 5.15.0 (currently betas being tested)
    • Ruby 2.7 – possibly paired with the removal of Ruby 2.6
    • GCC 10 as the default compiler
    • Removal of Python 2
    • GNU Make 4.3
    Ish Sookun posted at 18:04

    Nationwide confinement announced in Mauritius

    The Prime MInister of Mauritius had a press conference scheduled at 14h30 today. 1.5 hours later it was canceled and journalists were told that the PM will address the nation directly later.

    At around 21h30 Prime Minister P. K. Jugnauth addressed the population through the national TV, Mauritius Broadcasting Corporation. It appeared to be a recorded message.

    P. K. Jugnauth announced that four new cases of COVID-19 have been registered today. This makes the total cases of COVID-19 rise to seven in Mauritius. This number is high enough to raise the alarms now.

    He announced nationwide confinement starting 06h00 tomorrow to last for two weeks. He assured the population that there won't be shortage of staple food and medicine. He also assured people working in the private sector that their salary won't be affected.

    YaST Team posted at 16:00

    Highlights of YaST Development Sprint 95

    Contents

    Due to recent events, many companies all over the world are switching to a remote working model, and SUSE is not an exception. The YaST team is distributed so, for many members, it is not a big deal because they are already used to work in this way. For other folks it might be harder. Fortunately, SUSE is fully supporting us in this endeavor, so the YaST team has been able to deliver quite some stuff during this sprint, and we will keep doing our best in the weeks to come.

    Before jumping into what the team has recently done, we would also like to bring your attention to the migration of our blog from the good old openSUSE Lizards blog platform to the YaST website. So, please, if you use some feeds reader, update the YaST blog URL to the new one.

    Now, as promised, let’s talk only about software development. These days we are mainly focused on fixing bugs to make the upcoming (open)SUSE releases shine. However, we still have time to introduce some important improvements. Among all the changes, we will have a look at the following ones:

    Expanding the Possibilities of Pervasive Encryption

    Some months ago, in this dedicated blog post, we introduced the joys and benefits of the so-called pervasive encryption available for s390 mainframes equipped with a Crypto Express cryptographic coprocessor. As you may remember (and you can always revisit the post if you don’t), those dedicated pieces of hardware ensure the information at-rest in any storage device can only be read in the very same system where that information was encrypted.

    But, what is better than a cryptographic coprocessor? Several cryptographic coprocessors! An s390 logical partition (LPAR) can have access to multiple crypto express adapters, and several systems can share every adapter. To configure all that, the concept of cryptographic domains is used. Each domain is protected by a master key, thus preventing access across domains and effectively separating the contained keys.

    Now YaST detects when it’s encrypting a device in a system with several cryptographic domains. If that’s the case, the dialog for pervasive encryption allows specifying which adapters and domains must be used to generate the new secure key.

    To succeed, all the used adapters/domains must be set with the same master key. If that’s not the case, YaST detects the circumstance and displays the corresponding information.

    Install Missing Packages during Storage System Analysis

    As our reader surely knows, YaST always ensures the presence of all the needed utilities when performing any operation in the storage devices, like formatting and/or encrypting them. If some necessary tool is missing in the system, YaST has always shown the following dialog to alert the user and to allow to install the missing packages with a single click.

    But the presence of those tools was only checked at the end of the process, when YaST needed them to modify the devices. For example, in the screenshot above, YaST asked for btrfsprogs & friends because it wanted to format a new partition with that file system.

    If the needed utility was already missing during the initial phase in which the storage devices are analyzed, the user had no chance to install the corresponding package. For example, if a USB stick formatted with Btrfs would have been inserted, the user would get an error like this when executing the YaST Partitioner or when opening the corresponding YaST module to configure the bootloader.

    Btrfs old error message

    Now that intimidating error is replaced by this new pop-up that allows to install the missing packages and restart the hardware probing. As usual, with YaST, expert users can ignore the warning and continue the process if they understand the consequences outlined in the new pop-up window.

    Probe callback

    We took the opportunity to fix other small details in the area, like better reporting when the YaST Partitioner fails to install some package, a more up-to-date list of possibly relevant packages per technology, and improvements in the source code organization and the automated tests.

    Reporting Conflicting Storage Attributes in AutoYaST Profiles

    If you are an AutoYaST user, you undoubtely know that it is often too quiet and offers little information about inconsistencies or potential problems in the profile. For simple sections, it is not a problem at all, but for complicated stuff, like partitioning, it is far from ideal.

    In (open)SUSE 15 and later versions, and given that we had to reimplement the partitioning support using the new storage layer, we decided to add a mechanism to report some of those issues like missing attributes or invalid values. There is a high chance that, using an old profile in newer AutoYaST versions, you have seen some of those warnings.

    Recently, a user reported a problem that caused AutoYaST to crash. While debugging the problem, we found both raid_name and lvm_group attributes defined in one of the partition sections. Obviously, they are mutually exclusive, but it is quite easy to overlook this situation. Not to mention that AutoYaST should not crash.

    From now on, if AutoYaST detects such an inconsistency, it will automatically select one of the specified attributes, informing the user about the decision. You can see an example in the screenshot below.

    AutoYaST conflicting attributes warning

    For the time being, this check only applies to those attributes which determine how a device is going to be used (mount, raid_name, lvm_name, btrfs_name, bcache_backing_for, and bcache_caching_for), but we would like to extend this check in the future.

    Usability Improvements in iSCSI-LIO-server Module

    Recently, one of our developers detected several usability problems in the iSCSI LIO Server module, and he summarized them in a bug report. Apart from minor things, like some truncated and misaligned texts, he reported the UI to be quite confusing: it is not clear when authentication credentials are needed, and some labels are misleading. To add insult to injury, we found a potential crash when clicking the Edit button while we were addressing those issues.

    As usual, a image is worth a thousand words. Below you can see how the old and confusing UI looked like.

    Old iSCSI LIO Server Module UI

    Now, let’s compare it with the new one, which is better organized and more approachable. Isn’t it?

    New iSCSI LIO Server Module UI

    Conclusion

    It is possible that, during the upcoming weeks, we need to make some further adjustments to our workflow, especially when it comes to video meetings. But, at this point, everything is working quite well, and we are pretty sure that we will keep delivering at a good pace.

    So, take care, and stay tuned!