Renaming or changing a schema is not an easy task in Oracle operation but if you really want to rename the schema go for the traditional way of exporting the existing schema and import into a new schema.

Step shown in this tutorial is using Oracle 11g, it maybe won’t work for newer version.

> select * from v$version;
|================================================================================|
|BANNER                                                                          |
|================================================================================|
|Oracle Database 11g Release 11.2.0.4.0 - 64bit Production                       |
|PL/SQL Release 11.2.0.4.0 - Production                                          |
|CORE 11.2.0.4.0 Production                                                      |
|TNS for Linux: Version 11.2.0.4.0 - Production                                  |
|NLSRTL Version 11.2.0.4.0 - Production                                          |

Data Pump Mapping to the imp Utility

Please take note, Data Pump import often doesn’t have a one-to-one mapping of the legacy utility parameter. Data Pump import automatically provides many features of the old imp utility.

For example, COMMIT=Y isn’t required because Data Pump import automatically commits after each table is imported. Table below describes how legacy import parameters map to Data Pump import.

How I gonna rename my ORACLE schema

Let say, I accidently create user schema HOST_USER as suppose to be host1

-- Create user schema (mistaken username here..shit)
create user HOST_USER IDENTIFIED BY password4sk default TABLESPACE host_sk;
-- creating table space
create TABLESPACE host_sk datafile 'host_sk.dbf' size 1G autoextend on maxsize 8G;
-- create database role
create role HOST_SK_ROLE;
-- granting some privilage to role we created
grant  
   CREATE SESSION, ALTER SESSION, CREATE MATERIALIZED VIEW, CREATE PROCEDURE, 
   CREATE SEQUENCE, CREATE SYNONYM, CREATE TABLE, CREATE TRIGGER, CREATE TYPE, 
   CREATE VIEW, DEBUG CONNECT SESSION
to HOST_SK_ROLE;
-- grant that role to user (that I mistaken created previously) 
-- and give tablespace quota to them
GRANT HOST_SK_ROLE TO HOST_USER;
ALTER USER HOST_USER QUOTA unlimited ON host_sk;
-- create table
CREATE TABLE "HOST_USER"."STOCK_BALANCE_WS" 
   ("TRANSFERID" NUMBER(9,0), "ARTICLE_ID" VARCHAR2(14), 
  "QUANTITY" NUMBER(6,0)) SEGMENT CREATION IMMEDIATE 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
 NOCOMPRESS LOGGING
  STORAGE(INITIAL 1048576 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
  BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
  TABLESPACE "HOST_SK" ;
-- insert some data inside
INSERT INTO HOST_USER.ARTICLE_SW 
       (TRANSFERID, ARTICLE_ID, ARTICLE_NAME, DESCRIPTION, WEIGHT) 
VALUES (1, '1003', 'CONDOM DUREX', 'Super studs', 10);

As you see, I already do lot of thing with my database, then I just realize schema should be HOST1 instead of HOST_USER! I want to rename the schema. Unfortunately, oracle don’t allow to change schema name easily.

There is a trick by importing and map to HOST1 schema (if you follow my step, please don’t just copy paste. create the target user it if you don’t have and as long that user have same privilage and tablespace it will be fine)

Export with oracle data pump

$ expdp HOST_USER/password4sk directory=tmp schemas=HOST_USER dumpfile=old_schema_to_remap.dmp  LOGFILE=exp_schema_to_remap.log
Export: Release 11.2.0.4.0 - Production on Thu Apr 2 04:17:08 2020

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Connected to: Oracle Database 11g Release 11.2.0.4.0 - 64bit Production
Starting "HOST_USER"."SYS_EXPORT_SCHEMA_01":  HOST_USER/******** directory=tmp schemas=HOST_USER dumpfile=old_schema_to_remap.dmp LOGFILE=exp_schema_to_remap.log
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 0 KB
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
. . exported "HOST_USER"."ARTICLE_SW"                     0 KB       0 rows
Master table "HOST_USER"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
******************************************************************************
Dump file set for HOST_USER.SYS_EXPORT_SCHEMA_01 is:
  /tmp/old_schema_to_remap.dmp
Job "HOST_USER"."SYS_EXPORT_SCHEMA_01" successfully completed at Thu Apr 2 04:17:21 2020 elapsed 0 00:00:13

Import to other target user via remap_schema parameter

$ impdp userid=host1/password4sk directory=tmp dumpfile=old_schema_to_remap remap_schema=HOST_USER:host1 LOGFILE=imp_schema_to_remap.log
Import: Release 11.2.0.4.0 - Production on Thu Apr 2 04:19:07 2020

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Connected to: Oracle Database 11g Release 11.2.0.4.0 - 64bit Production
Master table "HOST1"."SYS_IMPORT_FULL_01" successfully loaded/unloaded
Starting "HOST1"."SYS_IMPORT_FULL_01":  userid=host1/******** directory=tmp dumpfile=old_schema_to_remap remap_schema=HOST_USER:host1 LOGFILE=imp_schema_to_remap.log
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
. . imported "HOST1"."ARTICLE_SW"                            0 KB       0 rows
Job "HOST1"."SYS_IMPORT_FULL_01" successfully completed at Thu Apr 2 04:19:09 2020 elapsed 0 00:00:01

after import via data pump, the next step is just to drop the old schema.

This step is much easier than redo everything IMHO. Anyway, please becareful. I am novice oracle dba, my step maybe not suitable for you :smile:

April 1st seems like as good a time as any for a new Chafa release — though note that Chafa is no joke. At least not anymore, what with the extremely enterprise-ready sixel pipeline and all.

As usual, you can get it from the download page or from Github. There are also release notes. Here are the highlights:

Sixel output

Thanks to this 90s-era technology, you can print excellent-looking graphics directly in the terminal with no need for character cell mosaics or hacky solutions like w3mimagedisplay (from w3m) or Überzug. It works entirely using ANSI escape sequence extensions, so it's usable over ssh, telnet and that old 2400 baud modem you found in grandma's shed.

The most complete existing implementation is probably Hayaki Saito's libsixel, but I chose to write one from scratch for Chafa, since sixel output is remarkably intensive computationally, and I wanted to employ a combination of advanced techniques (parallelism, quantization using a PCA approach, SIMD scaling) and corner-cutting that wouldn't have been appropriate in that library. This gets me fast animation playback and makes it easier to phase out the ImageMagick dependency in the long term.

There are at least two widely available virtual terminals that support sixels: One is XTerm (when compiled with --enable-sixel), and the other is mlterm. Unfortunately, I don't think either is widely used compared to distribution defaults like GNOME Terminal and Konsole, so here's hoping for more mainstream support for this feature.

Glyph import

If sixels aren't your cup of tea, symbol mode has a new trick for you too. It's --glyph-file, which allows you to load glyphs from external fonts into Chafa's symbol map. This can give it a better idea of what your terminal font looks like and allows support for more exotic symbols or custom fonts to suit any respectable retro graphics art project.

Keep in mind that you still need to select the appropriate symbol ranges with --symbols and/or --fill. These options now allow specifying precise Unicode ranges, e.g. --symbols 20,41..5a to emit only ASCII spaces and uppercase letters.

Color extraction

In symbol mode, each cell's color pair is now based on the median color of the underlying pixels instead of the average. Now this isn't exactly a huge feature, at least not in terms of effort, but it can make a big difference for certain images, especially line art. You can get the old behavior back with --color-extractor average.

इस प्रश्न का उत्तर जानने के लिए हमें जुलाई के महीने तक इंतजार करना होगा।

१७ मार्च को openSUSE Board की बैठक हुई। यह निर्णय लिया गया कि इस समय हम केवल COVID-19 की स्थिति देख सकते हैं और जुलाई तक इंतजार कर सकते हैं जब बोर्ड द्वारा openSUSE.Asia Summit और oSLO 2020 को लेकर कुछ निर्णय लिया जाएगा।

तब तक सुरक्षित रहें और यदि आप का देश लॉकडाउन में हैं तो कृपया घर पर रहें।

In this article, we will discover a softwares trio that will allow you to manage and listen to your music from your terminal:

• mpd: the music player daemon
• mpc: a CLI interface to mpd
• ncmpcpp: a mpd client, written in C++ with a ncurses interface

We will see how to install, configure and use it, as well as configuring smoothly integrated desktop notifications.

Installation

As they are available in the official openSUSE repositories installing them is as easy as:

zypper in mpd mpclient ncmpcpp

Onward to the configuration part !

Configuring mpd, the music player daemon

In this article, we will configure and run mpd as an user instance. If needed, it can of course run as a system instance. In that case, you should configure it in /etc/mpd.conf.

First of all, let’s create some configuration directories:

mkdir ~/.config/mpd
mkdir ~/.config/mpd/playlists 

Then, we will use the package template:

cp /usr/share/doc/packages/mpd/mpdconf.example ~/.config/mpd/mpd.conf

and edit it to our needs. Below is an extract of the important settings (the example user is geeko):

grep -v "^#|^$" ~/.config/mpd/mpd.conf

music_directory                 "~/Music"
playlist_directory              "~/.config/mpd/playlists"
db_file                         "~/.config/mpd/mpd.db"
log_file                        "~/.config/mpd/log"
pid_file                        "~/.config/mpd/mpd.pid"
state_file                      "~/.config/mpd/mpdstate"
sticker_file                    "~/.config/mpd/sticker.sql"
user                            "geeko"
bind_to_address                 "localhost"
port                            "6600"
log_level                       "default"
restore_paused                  "yes"
input {
        plugin "curl"
}
audio_output {
        type            "pulse"
        name            "Pulse MPD Output"
}
audio_output {
    type        "fifo"
    name        "mpd_fifo"
    path        "/tmp/mpd.fifo"
    format      "44100:16:2"
}
filesystem_charset              "UTF-8"

It is crucial to set properly the path to your audio files and sound output. Here we use PulseAudio as it is installed and used by default on most openSUSE desktops. If you are attentive, you will notice that we added a second audio output which will not be used for audio but to display a graphical visualizer in ncmpcpp (yes, in a terminal emulator !).

Activate mpd as an user space systemd service

Now we will create a systemd service that will start mpd with the user settings. This way mpd will start with your session.

Let’s create the needed directories structure:

mkdir -p ~/.config/systemd/user

and create the service file:

$EDITOR ~/.config/systemd/user/mpd.service

add the following content (remember to adapt the $USER variable to your need):

[Unit]
Description=Music Player Daemon

[Service]
ExecStart=/usr/bin/mpd --no-daemon /home/$USER/.config/mpd/mpd.conf
ExecStop=/usr/bin/mpd --kill
PIDFile=/home/$USER/.config/mpd/mpd.pid

[Install]
WantedBy=default.target

finally, let’s start and enable this service:

systemctl --user start mpd
systemctl --user enable mpd

ncmpcpp’s configuration

Now that mpd is up and running, we will configure the ncurses client we installed : ncmpcpp.

mkdir ~/.ncmpcpp
cp /usr/share/doc/packages/ncmpcpp/config ~/.ncmpcpp/
$EDITOR ~/.ncmpcpp/config

Here is an example of a nice colorized configuration with a split view:

grep -v "^#|^$" ~/.ncmpcpp/config

ncmpcpp_directory = ~/.ncmpcpp
lyrics_directory = ~/.ncmpcpp/lyrics
mpd_host = localhost
mpd_port = 6600
mpd_connection_timeout = 5
mpd_music_dir = ~/Music
mpd_crossfade_time = 5
visualizer_fifo_path = /tmp/mpd.fifo
visualizer_output_name = mpd_fifo
visualizer_in_stereo = yes
visualizer_sync_interval = 30
visualizer_type = ellipse
visualizer_look = ▮●
visualizer_color = 41, 83, 119, 155, 185, 215, 209, 203, 197, 161
system_encoding = "UTF-8"
playlist_disable_highlight_delay = 5
message_delay_time = 5
song_list_format = {%a - }{%t}|{$8%f$9}$R{$3(%l)$9}
song_status_format = {{%a{ "%b"{ (%y)}} - }{%t}}|{%f}
song_library_format = {%n - }{%t}|{%f}
alternative_header_first_line_format = $b$1$aqqu$/a$9 {%t}|{%f} $1$atqq$/a$9$/b
alternative_header_second_line_format = {{$4$b%a$/b$9}{ - $7%b$9}{ ($4%y$9)}}|{%D}
now_playing_prefix = $b
now_playing_suffix = $/b
song_window_title_format = {%a - }{%t}|{%f}
browser_sort_mode = name
browser_sort_format = {%a - }{%t}|{%f} {(%l)}
song_columns_list_format = (20)[]{a} (6f)[green]{NE} (50)[white]{t|f:Title} (20)[cyan]{b} (7f)[magenta]{l}
playlist_show_mpd_host = no
playlist_show_remaining_time = yes
playlist_shorten_total_times = no
playlist_separate_albums = no
playlist_display_mode = columns
browser_display_mode = classic
search_engine_display_mode = classic
playlist_editor_display_mode = classic
incremental_seeking = yes
seek_time = 1
volume_change_step = 2
autocenter_mode = yes
centered_cursor = yes
progressbar_look = =>
default_place_to_search_in = database
user_interface = alternative
media_library_primary_tag = genre
default_find_mode = wrapped
header_visibility = yes
statusbar_visibility = yes
titles_visibility = yes
header_text_scrolling = yes
cyclic_scrolling = yes
lines_scrolled = 2
follow_now_playing_lyrics = yes
fetch_lyrics_for_current_song_in_background = yes
store_lyrics_in_song_dir = yes
allow_for_physical_item_deletion = no
screen_switcher_mode = browser, media_library, visualizer
startup_screen = playlist
startup_slave_screen = "visualizer"
startup_slave_screen_focus = no
locked_screen_width_part = 50
jump_to_now_playing_song_at_start = yes
ask_before_clearing_playlists = yes
clock_display_seconds = no
display_volume_level = yes
display_bitrate = no
display_remaining_time = yes
ignore_leading_the = no
mouse_support = yes
enable_window_title = yes
external_editor = vim
use_console_editor = yes
colors_enabled = yes

With these settings you will have a split view with the current playlist of the left and the visualizer on the right. All options are well documented in ncmpcpp’s man page.

Use it

Here are a few shortcuts that will help you getting started

• F1 : show help
• 1 : show playlist ;
• 2 : show directory browser
• 3 : show search
• 4 : show library
• 5 : playlist editor
• 6 : tags editor
• 8 : visualizer
• p : toggle play/pause
• a : add selection to playlist
• > : play next track
• < :play previous track

Desktop Environment shortcut

By default, pressing p will toggle pause, but what if you are not in front of your terminal running ncmpcpp ? That is where mpc enters the game. Let’s open our favorite desktop environment settings and add some keyboard shortcuts:

• MPD Pause: use the mpc toggle command
• MPD Previous Song: use the mpc prev command
• MPD Next Song: use the mpc next command

Extra: get notified when song changes

There is a configuration parameter in ncmpcpp that makes it trigger a command each time the song changes, we will use it to execute a small Python3 script in order to pop a nice desktop notification.

In order to use this script, you will have to make sure that two small libs are installed:

zypper in python3-notify2 python3-python-mpd2

Then add the following code in a file called mpd_notify.py in your $HOME/bin:

#!/usr/bin/env python3
# -*- coding: UTF8 -*-

import gi
import notify2
from gi.repository import GLib
from mpd import MPDClient

client = MPDClient()
client.timeout = 10
client.idletimeout = None
client.connect("localhost", 6600)

mpd_song = MPDClient.currentsong(client)

s_artist = mpd_song['artist']
s_title = mpd_song['title']
s_album = mpd_song['album']

s_notification = s_artist + " - " + s_title + " - (" + s_album +")"

notify2.init("Music Player Demon")
show_song = notify2.Notification("Music Player Demon", s_notification,
    icon="/usr/share/icons/Adwaita/scalable/emblems/emblem-music-symbolic.svg")

show_song.set_hint("transient", True)

show_song.show()

Now, we will just add the corresponding parameter in ncmpcpp configuration’s file:

execute_on_song_change = "/usr/bin/python3 /home/$USER/bin/mpd_notify.py"

In the end

We hope that you liked this discovery and that you will enjoy managing your music with those tools. ncmpcpp is quite powerful, it includes search capabilities, different views of your music library, a tag editor and everything you need to tweak it to your taste !

Dear Tumbleweed users and hackers,

During this week, we have released 6 snapshots to the public (0318, 0319, 0320, 0322, 0324, 0325). The changes were more under the hood than spectacular, but here they are:

• RPM: change of database format to ndb (Read on – there were some issues)
• Mozilla Firefox 74.0
• Linux kernel 5.5.9 & 5.5.11
• VirtualBox 6.1.4
• KDE Frameworks 5.68.0
• Mesa 20.0.2
• Coreutils 8.32
• Samba 4.12.0

This was week 13. Are you superstitious? Any special fear of ’13’? We could, of course, blame week 13 for the minor issue we had not caught with openQA, but I’d rather claim it would have happened any week. We simply lack an upgrade test for MicroOS. What am I talking about? You most likely have not even realized there was an issue. The RPM switch of the database format to ndb (any other DB format would have exposed the same issue, so no flaming here) had a glitch on MicroOS and transactional-update based installations in that the RPM database was not accessible for a moment. Nothing broke (unless you rolled back after seeing it – this might have been more problematic). See Bug 1167537 if you’re interested in more details. Of course, fixed RPM packages are already out (we even published it asap in the :Update channel) – and the affected systems could self heal.

Staging projects are still busy with changes around these topics:

• Kubernetes 1.18.0
• Linux kernel 5.5.13
• Rust 1.41.1
• LLVM 10
• Qt 5.15.0 (currently beta2 being tested)
• Ruby 2.7 – possibly paired with the removal of Ruby 2.6
• GCC 10 as the default compiler
• Removal of Python 2
• GNU Make 4.3

Dot what ???

What is commonly referred to as dotfiles are all those small plain text files that contain your softwares’ configuration. Most of the time they reside in your $HOME directory but are hidden as they are prefixed with a dot, hence their name. For some apps, you can find them as well in the $HOME/.config directory.

When it comes to managing them (i.e keep track of the changes, moving them around between different workstations, backing them up,…), there are different solutions:

• the good old USB stick
• rsyncing them
• syncing them in the “cloud”
• copying them in a central folder and symlinking them to where they are supposed to be found

In this article, we will focus on how to manage them in an efficient and simple way with Git.

Git comes to rescue

In this example, we will use $HOME/Dotfiles as the Git repository, but feel free to change it to your needs.

First of all, we will initialize this repository

git init --bare $HOME/Dotfiles

Then, as all the git commands that we will use will refer to this repository, it is advised to create an alias, such as:

alias dotfiles='/usr/bin/git --git-dir=$HOME/Dotfiles --work-tree=$HOME'

You can add this line to your $SHELL configuration file ($HOME/.bashrc if you use Bash or $HOME/.zshrc if you use zsh).

Next, we will configure Git so it will not show all the untracked files. This is required as we use the entire $HOME as work tree.

dotfiles config --local status.showUntrackedFiles no

At that point, you should be able to check the state of this repository:

dotfiles status

Then you can add your configuration files and commit as you wish. For example, let’s add our .bashrc :

dotfiles add .bashrc
dotfiles commit -m "Added .bashrc"

Now just add a remote repository (your self-hosted Git or a public one) and push your changes to it:

dotfiles remote add origin git@yourgit.example.com/dotfiles.git
dotfiles push

Setup a new machine

Now that you have it all set, let’s configure a new system with the dotfiles you have in your repository.

First, clone locally your online repository:

git clone --bare git@yourgit.example.com/dotfiles.git $HOME/Dotfiles

Again, you have to defined the same alias as before:

alias dotfiles='/usr/bin/git --git-dir=$HOME/Dotfiles --work-tree=$HOME'

Remember to put it in your $SHELL configuration file. Now, just apply the changes from the repository you have just cloned to your system:

dotfiles checkout

If some of the files already exist, you will get an error. This will probably happen with files created by default during the openSUSE installation and user account creation, such as the $HOME/.bashrc file, no worries, just rename or delete them.

Now, each time you change your configuration files tracked by Git, remember to commit and push your changes.

Sources

The following articles where used as sources of this article. Thanks a lot to their authors:

• Easily manage your dotfiles with git
• How to make your Dotfile management a painless affair
• How to manage your dotfiles with git

Let's continue with the enormous SVG from the last time, a map extracted from OpenStreetMap.

According to Massif, peak memory consumption for that file occurs at the following point during the execution of rsvg-convert. I pasted only the part that refers to Bézier paths:

    --------------------------------------------------------------------------------
      n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
    --------------------------------------------------------------------------------
1    33 24,139,598,653    1,416,831,176    1,329,943,212    86,887,964            0
2   ->24.88% (352,523,448B) 0x4A2727E: alloc (alloc.rs:84)
    | ->24.88% (352,523,448B) 0x4A2727E: alloc (alloc.rs:172)
    |   ->24.88% (352,523,448B) 0x4A2727E: allocate_in<rsvg_internals::path_builder::PathCommand,alloc::alloc::Global> (raw_vec.rs:98)
    |     ->24.88% (352,523,448B) 0x4A2727E: with_capacity<rsvg_internals::path_builder::PathCommand> (raw_vec.rs:167)
    |       ->24.88% (352,523,448B) 0x4A2727E: with_capacity<rsvg_internals::path_builder::PathCommand> (vec.rs:358)
    |         ->24.88% (352,523,448B) 0x4A2727E: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter (vec.rs:1992)
    |           ->24.88% (352,523,448B) 0x49D212C: from_iter<rsvg_internals::path_builder::PathCommand,smallvec::IntoIter<[rsvg_internals::path_builder::PathCommand; 32]>> (vec.rs:1901)
    |             ->24.88% (352,523,448B) 0x49D212C: collect<smallvec::IntoIter<[rsvg_internals::path_builder::PathCommand; 32]>,alloc::vec::Vec<rsvg_internals::path_builder::PathCommand>> (iterator.rs:1493)
    |               ->24.88% (352,523,448B) 0x49D212C: into_vec<[rsvg_internals::path_builder::PathCommand; 32]> (lib.rs:893)
    |                 ->24.88% (352,523,448B) 0x49D212C: smallvec::SmallVec<A>::into_boxed_slice (lib.rs:902)
3   |                   ->24.88% (352,523,016B) 0x4A0394C: into_path (path_builder.rs:320)
    |
4   ->03.60% (50,990,328B) 0x4A242F0: realloc (alloc.rs:128)
    | ->03.60% (50,990,328B) 0x4A242F0: realloc (alloc.rs:187)
    |   ->03.60% (50,990,328B) 0x4A242F0: shrink_to_fit<rsvg_internals::path_builder::PathCommand,alloc::alloc::Global> (raw_vec.rs:633)
    |     ->03.60% (50,990,328B) 0x4A242F0: shrink_to_fit<rsvg_internals::path_builder::PathCommand> (vec.rs:623)
    |       ->03.60% (50,990,328B) 0x4A242F0: alloc::vec::Vec<T>::into_boxed_slice (vec.rs:679)
    |         ->03.60% (50,990,328B) 0x49D2136: smallvec::SmallVec<A>::into_boxed_slice (lib.rs:902)
5   |           ->03.60% (50,990,328B) 0x4A0394C: into_path (path_builder.rs:320)

Line 1 has the totals, and we see that at that point the program uses 1,329,943,212 bytes on the heap.

Lines 3 and 5 give us a hint that into_path is being called; this is the function that converts a temporary/mutable PathBuilder into a permanent/immutable Path.

Lines 2 and 4 indicate that the arrays of PathCommand, which are inside those immutable Paths, use 24.88% + 3.60% = 28.48% of the program's memory; between both they use 352,523,448 + 50,990,328 = 403,513,776 bytes.

That is about 400 MB of PathCommand. Let's see what's going on.

What is in a PathCommand?

A Path is a list of commands similar to PostScript, which get used in SVG to draw Bézier paths. It is a flat array of PathCommand:

pub struct Path {
    path_commands: Box<[PathCommand]>,
}

pub enum PathCommand {
    MoveTo(f64, f64),
    LineTo(f64, f64),
    CurveTo(CubicBezierCurve),
    Arc(EllipticalArc),
    ClosePath,
}

Let's see the variants of PathCommand:

• MoveTo: 2 double-precision floating-point numbers.
• LineTo: same.
• CurveTo: 6 double-precision floating-point numbers.
• EllipticalArc: 7 double-precision floating-point numbers, plus 2 flags (see below).
• ClosePath: no extra data.

These variants vary a lot in terms of size, and each element of the Path.path_commands array occupies the maximum of their sizes (i.e. sizeof::<EllipticalArc>).

A more compact representation

Ideally, each command in the array would only occupy as much space as it needs.

We can represent a Path in a different way, as two separate arrays:

• A very compact array of commands without coordinates.
• An array with coordinates only.

That is, the following:

pub struct Path {
    commands: Box<[PackedCommand]>,
    coords: Box<[f64]>,
}

The coords array is obvious; it is just a flat array with all the coordinates in the Path in the order in which they appear.

And the commands array?

PackedCommand

We saw above that the biggest variant in PathCommand is Arc(EllipticalArc). Let's look inside it:

pub struct EllipticalArc {
    pub r: (f64, f64),
    pub x_axis_rotation: f64,
    pub large_arc: LargeArc,
    pub sweep: Sweep,
    pub from: (f64, f64),
    pub to: (f64, f64),
}

There are 7 f64 floating-point numbers there. The other two fields, large_arc and sweep, are effectively booleans (they are just enums with two variants, with pretty names instead of just true and false).

Thus, we have 7 doubles and two flags. Between the two flags there are 4 possibilities.

Since no other PathCommand variant has flags, we can have the following enum, which fits in a single byte:

#[repr(u8)]
enum PackedCommand {
    MoveTo,
    LineTo,
    CurveTo,
    ArcSmallNegative,
    ArcSmallPositive,
    ArcLargeNegative,
    ArcLargePositive,
    ClosePath,
}

That is, simple values for MoveTo/etc. and four special values for the different types of Arc.

Packing a PathCommand into a PackedCommand

In order to pack the array of PathCommand, we must first know how many coordinates each of its variants will produce:

impl PathCommand {
    fn num_coordinates(&self) -> usize {
        match *self {
            PathCommand::MoveTo(..) => 2,
            PathCommand::LineTo(..) => 2,
            PathCommand::CurveTo(_) => 6,
            PathCommand::Arc(_) => 7,
            PathCommand::ClosePath => 0,
        }
    }
}

Then, we need to convert each PathCommand into a PackedCommand and write its coordinates into an array:

impl PathCommand {
    fn to_packed(&self, coords: &mut [f64]) -> PackedCommand {
        match *self {
            PathCommand::MoveTo(x, y) => {
                coords[0] = x;
                coords[1] = y;
                PackedCommand::MoveTo
            }

            // etc. for the other simple commands

            PathCommand::Arc(ref a) => a.to_packed_and_coords(coords),
        }
    }
}

Let's look at that to_packed_and_coords more closely:

impl EllipticalArc {
    fn to_packed_and_coords(&self, coords: &mut [f64]) -> PackedCommand {
        coords[0] = self.r.0;
        coords[1] = self.r.1;
        coords[2] = self.x_axis_rotation;
        coords[3] = self.from.0;
        coords[4] = self.from.1;
        coords[5] = self.to.0;
        coords[6] = self.to.1;

        match (self.large_arc, self.sweep) {
            (LargeArc(false), Sweep::Negative) => PackedCommand::ArcSmallNegative,
            (LargeArc(false), Sweep::Positive) => PackedCommand::ArcSmallPositive,
            (LargeArc(true), Sweep::Negative) => PackedCommand::ArcLargeNegative,
            (LargeArc(true), Sweep::Positive) => PackedCommand::ArcLargePositive,
        }
    }
}

Creating the compact Path

Let's look at PathBuilder::into_path line by line:

impl PathBuilder {
    pub fn into_path(self) -> Path {
        let num_commands = self.path_commands.len();
        let num_coords = self
            .path_commands
            .iter()
            .fold(0, |acc, cmd| acc + cmd.num_coordinates());

First we compute the total number of coordinates using fold; we ask each command cmd its num_coordinates() and add it into the acc accumulator.

Now we know how much memory to allocate:

        let mut packed_commands = Vec::with_capacity(num_commands);
        let mut coords = vec![0.0; num_coords];

We use Vec::with_capacity to allocate exactly as much memory as we will need for the packed_commands; adding elements will not need a realloc(), since we already know how many elements we will have.

We use the vec! macro to create an array of 0.0 repeated num_coords times; that macro uses with_capacity internally. That is the array we will use to store the coordinates for all the commands.

        let mut coords_slice = coords.as_mut_slice();

We get a mutable slice out of the whole array of coordinates.

        for c in self.path_commands {
            let n = c.num_coordinates();
            packed_commands.push(c.to_packed(coords_slice.get_mut(0..n).unwrap()));
            coords_slice = &mut coords_slice[n..];
        }

For each command, we see how many coordinates it will generate and we put that number in n. We get a mutable sub-slice from coords_slice with only that number of elements, and pass it to to_packed for each command.

At the end of each iteration we move the mutable slice to where the next command's coordinates will go.

        Path {
            commands: packed_commands.into_boxed_slice(),
            coords: coords.into_boxed_slice(),
        }
    }

At the end, we create the final and immutable Path by converting each array into_boxed_slice like the last time. That way each of the two arrays, the one with PackedCommands and the one with coordinates, occupy the minimum space they need.

An iterator for Path

This is all very well, but we also want it to be easy to iterate on that compact representation; the PathCommand enums from the beginning are very convenient to use and that's what the rest of the code already uses. Let's make an iterator that unpacks what is inside a Path and produces a PathCommand for each element.

pub struct PathIter<'a> {
    commands: slice::Iter<'a, PackedCommand>,
    coords: &'a [f64],
}

We need an iterator over the array of PackedCommand so we can visit each command. However, to get elements of coords, I am going to use a slice of f64 instead of an iterator.

Let's look at the implementation of the iterator:

impl<'a> Iterator for PathIter<'a> {
    type Item = PathCommand;

    fn next(&mut self) -> Option<Self::Item> {
        if let Some(cmd) = self.commands.next() {
            let cmd = PathCommand::from_packed(cmd, self.coords);
            let num_coords = cmd.num_coordinates();
            self.coords = &self.coords[num_coords..];
            Some(cmd)
        } else {
            None
        }
    }
}

Since we want each iteration to produce a PathCommand, we declare it as having the associated type Item =  PathCommand.

If the self.commands iterator has another element, it means there is another PackedCommand available.

We call PathCommand::from_packed with the self.coords slice to unpack a command and its coordinates. We see how many coordinates the command consumed and re-slice self.coords according to the number of commands, so that it now points to the coordinates for the next command.

We return Some(cmd) if there was an element, or None if the iterator is empty.

The implementation of from_packed is obvious and I'll just paste a bit from it:

impl PathCommand {
    fn from_packed(packed: &PackedCommand, coords: &[f64]) -> PathCommand {
        match *packed {
            PackedCommand::MoveTo => {
                let x = coords[0];
                let y = coords[1];
                PathCommand::MoveTo(x, y)
            }

            // etc. for the other variants in PackedCommand

            PackedCommand::ArcSmallNegative => PathCommand::Arc(EllipticalArc::from_coords(
                LargeArc(false),
                Sweep::Negative,
                coords,
            )),

            PackedCommand::ArcSmallPositive => // etc.

            PackedCommand::ArcLargeNegative => // etc.

            PackedCommand::ArcLargePositive => // etc.
        }
    }
}

Results

Before the changes (this is the same Massif heading as above):

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 33 24,139,598,653    1,416,831,176    1,329,943,212    86,887,964            0
                                       ^^^^^^^^^^^^^
                                           boo

After:

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 28 26,611,886,993    1,093,747,888    1,023,147,907    70,599,981            0
                                       ^^^^^^^^^^^^^
                                          oh yeah

We went from using 1,329,943,212 bytes down to 1,023,147,907 bytes, that is, we knocked it down by 300 MB.

However, that is for the whole program. Above we saw that Path data occupies 403,513,776 bytes; how about now?

->07.45% (81,525,328B) 0x4A34C6F: alloc (alloc.rs:84)
| ->07.45% (81,525,328B) 0x4A34C6F: alloc (alloc.rs:172)
|   ->07.45% (81,525,328B) 0x4A34C6F: allocate_in<f64,alloc::alloc::Global> (raw_vec.rs:98)
|     ->07.45% (81,525,328B) 0x4A34C6F: with_capacity<f64> (raw_vec.rs:167)
|       ->07.45% (81,525,328B) 0x4A34C6F: with_capacity<f64> (vec.rs:358)
|         ->07.45% (81,525,328B) 0x4A34C6F: rsvg_internals::path_builder::PathBuilder::into_path (path_builder.rs:486)

Perfect. We went from occupying 403,513,776 bytes to just 81,525,328 bytes. Instead of Path data amounting to 28.48% of the heap, it is just 7.45%.

I think we can stop worrying about Path data for now. I like how this turned out without having to use unsafe.

References

• Refactoring to use an iterator
• Adding tests for Path/PathBuilder
• Using a more compact representation for Path

Four openSUSE Tumbleweed snapshots were released so far this week.

Kismet, KDE Frameworks, sudo, LibreOffice and ImageMagick were just a few of the packages that received updates in the snapshots.

The most recent snapshot, 20200322 brougth the 1.3.6 version of the Bluetooth configuration tool, blueberry. Full featured Command Line Interface (CLI) system information tool inxi 3.0.38 fixed a Perl issue where perl treats 000 as a string and not 0. General purpose VPN package WireGuard removed dead code. The snapshot also updated several YaST packages. Fixes were made to help with text icons displayed during installations in yast2 4.2.74 package and some cosmetic changes were made in the yast2-ntp-client 4.2.10 package to not show check-boxes for saving configuration and starting the deamon. The snapshot is currently trending at a rating of 84, according to the Tumbleweed snapshot reviewer.

Just three packages were updated in snapshot 20200320. Python 2 compatibility was removed in the urlscan 0.9.4 package. Both elementary-xfce-icon-theme and perl-Encode 3.05 were updated in the snapshot, which is trending at a rating of 99.

The other two snapshots also recorded a stable rating of 99.

ImageMagick 7.0.10.0 provided an update that prevent heap overflow in snapshot 20200319. KDE’s Frameworks 5.68.0 fixed a memory leak in ConfigView and Dialog. Multiple additions and fixes were made to the Breeze Icons package of the new Frameworks version and Kirigami improved support Qt 5.14 on Android. LibreOffice 6.4.2.2 brought some translations and sudo had a minor change regarding an update that affected Linux containers and ignored a failure to restore the to RLIMIT_CORE resource limit.

Both the GNU Compiler Collection 9 and 10 were updated in the 20200318 snapshot. The updated versions includes fixes for binutils version parsing. The new major version of Mozilla Firefox 74.0 landed in the snapshot and fixed a dozen Common Vulnerability and Exposures, which included a fix for CVE-2020-6809 that addressed the Web Extensions that had the all-urls permission and made a fetch request with a mode set to ‘same-origin’ possible for the Web Extension to read local files. The Advanced Linux Sound Architecture (alsa) 1.2.2 package added multiple patches and the same version alsa-plugins package provided an update for m4 files affecting macro processors. Apparmor 2.13.4 provided several abstraction updates and fixed log parsing for logs with an embedded newline. Developers will be happy to see a new cscope 15.9 for source code searches as it adds parentheses and vertical bar metacharacters in regex searches. SecureTransport and WinCrypt implementations for sha256 were added in the curl 7.69.1. A maintenance release for virtualbox 6.1.4 was in the snapshot; the update supports Linux Kernel 5.5. Linux Kernel 5.5.9 was also released in the snapshot and xfsprogs 5.5.0 fixed broken unit conversions in the xfs_repair. Wireless network and device detector, sniffer, wardriving tool Kismet had its first full release for 2020, which was primarily a bugfix release; the 2020_03_R1 version had a fix for buffer size calculations, which could impact gps handling, and had updates for the ultra-low-power, highly-integrated single-chip device kw41z capture code.

• resetting windows update
• uninstalling various pieces of software
• in general, random changing of different settings ;-)

Im letzten Beitrag wurde die Idee zu einem auf der verbreiteten privaten Cloud Infrastruktur ownCloud basierenden System zum Unterricht erläutert, wenn die persönliche Begegnung wie zur Zeit nicht möglich ist.

Um das noch etwas zugänglicher zu machen, haben wir ein Video erstellt, das die Interaktion zwischen einer Musiklehrerin und ihrer Schülerin Felizitas beispielhaft zeigt. Dabei werden Videos geteilt, die einen eingeschränkten Lehrbetrieb ermöglichen und die Schülerin bei der Sache hält.

Die beschriebenen Schritte lassen sich ohne weiteres sofort mit jeder ownCloud oder wohl auch Nextcloud durchführen. Dabei können trotz der besonderen Situation Datenschutzaspekte beachtet werden.

Hoffentlich trägt das dazu bei, dass ohne lange Verzögerung das Beste aus der Situation gemacht werden kann.