Refactoring the Length type
CSS length values have a number and a unit, e.g. 5cm
or 6px. Sometimes the unit is a percentage, like 50%, and SVG
says that lengths with percentage units should be resolved with
respect to a certain rectangle. For example, consider this circle
element:
<circle cx="50%" cy="75%" r="4px" fill="black"/>
This means, draw a solid black circle whose center is at 50% of the width and 75% of the height of the current viewport. The circle should have a 4-pixel radius.
The process of converting that kind of units into absolute pixels for the final drawing is called normalization. In SVG, percentage units sometimes need to be normalized with respect to the current viewport (a local coordinate system), or with respect to the size of another object (e.g. when a clipping path is used to cut the current shape in half).
One detail about normalization is that it can be with respect to the horizontal dimension of the current viewport, the vertical dimension, or both. Keep this in mind: at normalization time, we need to be able to distinguish between those three modes.
The original C version
I have talked about the original C code for lengths before; the following is a small summary.
The original C code had this struct to represent lengths:
typedef struct {
double length;
char factor;
} RsvgLength;
The parsing code would set the factor field to a
character depending on the length's unit: 'p' for percentages, 'i'
for inches, etc., and '\0' for the default unit, which is pixels.
Along with that, the normalization code needed to know the direction (horizontal, vertical, both) to which the length in question refers. It did this by taking another character as an argument to the normalization function:
double
_rsvg_css_normalize_length (const RsvgLength * in, RsvgDrawingCtx * ctx, char dir)
{
if (in->factor == '\0') /* pixels, no need to normalize */
return in->length;
else if (in->factor == 'p') { /* percentages; need to consider direction */
if (dir == 'h') /* horizontal */
return in->length * ctx->vb.rect.width;
if (dir == 'v') /* vertical */
return in->length * ctx->vb.rect.height;
if (dir == 'o') /* both */
return in->length * rsvg_viewport_percentage (ctx->vb.rect.width,
ctx->vb.rect.height);
} else { ... }
}
The original post talks about how I found a couple of bugs with
how the directions are identified at normalization time. The function
above expects one of 'h'/'v'/'o' for horizontal/vertical/both, and one or
two places in the code passed the wrong character.
Making the C version cleaner
Before converting that code to Rust, I removed the pesky characters and made the code use proper enums to identify a length's units.
+typedef enum {
+ LENGTH_UNIT_DEFAULT,
+ LENGTH_UNIT_PERCENT,
+ LENGTH_UNIT_FONT_EM,
+ LENGTH_UNIT_FONT_EX,
+ LENGTH_UNIT_INCH,
+ LENGTH_UNIT_RELATIVE_LARGER,
+ LENGTH_UNIT_RELATIVE_SMALLER
+} LengthUnit;
+
typedef struct {
double length;
- char factor;
+ LengthUnit unit;
} RsvgLength;
Then, do the same for the normalization function, so it will get the direction in which to normalize as an enum instead of a char.
+typedef enum {
+ LENGTH_DIR_HORIZONTAL,
+ LENGTH_DIR_VERTICAL,
+ LENGTH_DIR_BOTH
+} LengthDir;
double
-_rsvg_css_normalize_length (const RsvgLength * in, RsvgDrawingCtx * ctx, char dir)
+_rsvg_css_normalize_length (const RsvgLength * in, RsvgDrawingCtx * ctx, LengthDir dir)
Making the C version easier to get right
While doing the last change above, I found a place in the code that used the wrong direction by mistake, probably due to a cut&paste error. Part of the problem here is that the code was specifying the direction at normalization time.
I decided to change it so that each direction value carried its own
direction since initialization, so that subsequent
code wouldn't have to worry about that. Hopefully, initializing a
width field should make it obvious that it needed
LENGTH_DIR_HORIZONTAL.
typedef struct {
double length;
LengthUnit unit;
+ LengthDir dir;
} RsvgLength;
That is, so that instead of
/* at initialization time */
foo.width = _rsvg_css_parse_length (str);
...
/* at rendering time */
double final_width = _rsvg_css_normalize_length (&foo.width, ctx, LENGTH_DIR_HORIZONTAL);
we would instead do this:
/* at initialization time */
foo.width = _rsvg_css_parse_length (str, LENGTH_DIR_HORIZONTAL);
...
/* at rendering time */
double final_width = _rsvg_css_normalize_length (&foo.width, ctx);
This made the drawing code, which deals with a lot of coordinates at the same time, a lot less noisy.
Initial port to Rust
To recap, this was the state of the structs after the initial refactoring in C:
typedef enum {
LENGTH_UNIT_DEFAULT,
LENGTH_UNIT_PERCENT,
LENGTH_UNIT_FONT_EM,
LENGTH_UNIT_FONT_EX,
LENGTH_UNIT_INCH,
LENGTH_UNIT_RELATIVE_LARGER,
LENGTH_UNIT_RELATIVE_SMALLER
} LengthUnit;
typedef enum {
LENGTH_DIR_HORIZONTAL,
LENGTH_DIR_VERTICAL,
LENGTH_DIR_BOTH
} LengthDir;
typedef struct {
double length;
LengthUnit unit;
LengthDir dir;
} RsvgLength;
This ported to Rust in a straightforward fashion:
pub enum LengthUnit {
Default,
Percent,
FontEm,
FontEx,
Inch,
RelativeLarger,
RelativeSmaller
}
pub enum LengthDir {
Horizontal,
Vertical,
Both
}
pub struct RsvgLength {
length: f64,
unit: LengthUnit,
dir: LengthDir
}
It got a similar constructor that took the
direction and produced an RsvgLength:
impl RsvgLength {
pub fn parse (string: &str, dir: LengthDir) -> RsvgLength { ... }
}
(This was before using Result; remember that the original
C code did very little error checking!)
The initial Parse trait
It was at that point that it seemed convenient to introduce a Parse
trait, which all CSS value types would implement to
parse themselves from string.
However, parsing an RsvgLength also needed an extra piece of data,
the LengthDir. My initial version of the Parse trait had an
associated called Data, through which one could pass an extra piece
of data during parsing/initialization:
pub trait Parse: Sized {
type Data;
type Err;
fn parse (s: &str, data: Self::Data) -> Result<Self, Self::Err>;
}
This was explicitly to be able to pass a LengthDir to the parser for
RsvgLength:
impl Parse for RsvgLength {
type Data = LengthDir;
type Err = AttributeError;
fn parse (string: &str, dir: LengthDir) -> Result <RsvgLength, AttributeError> { ... }
}
This was okay for lengths, but very noisy for everything else that
didn't require an extra bit of data. In the rest of the code, the
helper type was Data = () and there was a pair of extra parentheses ()
in every place that parse() was called.
Removing the helper Data type
Introducing one type per direction
Over a year later, that () bit of data everywhere was driving me
nuts. I started refactoring the Length module to remove it.
First, I introduced three newtypes to wrap Length, and indicate
their direction at the same time:
pub struct LengthHorizontal(Length);
pub struct LengthVertical(Length);
pub struct LengthBoth(Length);
This was done with a macro because now each wrapper type
needed to know the relevant LengthDir.
Now, for example, the declaration for the <circle> element looked
like this:
pub struct NodeCircle {
cx: Cell<LengthHorizontal>,
cy: Cell<LengthVertical>,
r: Cell<LengthBoth>,
}
(Ignore the Cell everywhere; we got rid of that later.)
Removing the dir field
Since now the information about the length's direction is embodied in
the LengthHorizontal/LengthVertical/LengthBoth types, this made it
possible to remove the dir field from the inner
Length struct.
pub struct RsvgLength {
length: f64,
unit: LengthUnit,
- dir: LengthDir
}
+pub struct LengthHorizontal(Length);
+pub struct LengthVertical(Length);
+pub struct LengthBoth(Length);
+
+define_length_type!(LengthHorizontal, LengthDir::Horizontal);
+define_length_type!(LengthVertical, LengthDir::Vertical);
+define_length_type!(LengthBoth, LengthDir::Both);
Note the use of a define_length_type! macro to generate code for
those three newtypes.
Removing the Data associated type
And finally, this made it possible to remove the Data associated
type from the Parse trait.
pub trait Parse: Sized {
- type Data;
type Err;
- fn parse(parser: &mut Parser<'_, '_>, data: Self::Data) -> Result<Self, Self::Err>;
+ fn parse(parser: &mut Parser<'_, '_>) -> Result<Self, Self::Err>;
}
The resulting mega-commit removed a bunch of stray parentheses ()
from all calls to parse(), and the code ended up a lot easier to
read.
Removing the newtypes
This was fine for a while. Recently, however, I figured out that it would be possible to embody the information for a length's direction in a different way.
But to get there, I first needed a temporary refactor.
Replacing the macro with a trait with a default implementation
Deep in the guts of length.rs, the key function that does something
different based on LengthDir is its scaling_factor method:
enum LengthDir {
Horizontal,
Vertical,
Both,
}
impl LengthDir {
fn scaling_factor(self, x: f64, y: f64) -> f64 {
match self {
LengthDir::Horizontal => x,
LengthDir::Vertical => y,
LengthDir::Both => viewport_percentage(x, y),
}
}
}
That method gets passed, for example, the width/height of the
current viewport for the x/y arguments. The method decides whether
to use the width, height, or a combination of both.
And of course, the interesting part of the define_length_type! macro
was to generate code for calling LengthDir::Horizontal::scaling_factor()/etc. as
appropriate depending on the LengthDir in question.
First I made a trait called Orientation with a scaling_factor
method, and three zero-sized types that implement that trait. Note
how each of these three implementations corresponds to one of the
match arms above:
pub trait Orientation {
fn scaling_factor(x: f64, y: f64) -> f64;
}
pub struct Horizontal;
pub struct Vertical;
pub struct Both;
impl Orientation for Horizontal {
fn scaling_factor(x: f64, _y: f64) -> f64 {
x
}
}
impl Orientation for Vertical {
fn scaling_factor(_x: f64, y: f64) -> f64 {
y
}
}
impl Orientation for Both {
fn scaling_factor(x: f64, y: f64) -> f64 {
viewport_percentage(x, y)
}
}
Now most of the contents of the define_length_type! macro can go in
the default implementation of a new trait
LengthTrait. Crucially, this trait has an
Orientation associated type, which it uses to call into the
Orientation trait:
pub trait LengthTrait: Sized {
type Orientation: Orientation;
...
fn normalize(&self, values: &ComputedValues, params: &ViewParams) -> f64 {
match self.unit() {
LengthUnit::Px => self.length(),
LengthUnit::Percent => {
self.length() *
<Self::Orientation>::scaling_factor(params.view_box_width, params.view_box_height)
}
...
}
}
Note that the incantation is
<Self::Orientation>::scaling_factor(...) to call that method on the
associated type.
Now the define_length_type! macro is shrunk a lot, with the
interesting part being just this:
macro_rules! define_length_type {
{$name:ident, $orient:ty} => {
pub struct $name(Length);
impl LengthTrait for $name {
type Orientation = $orient;
}
}
}
define_length_type! { LengthHorizontal, Horizontal }
define_length_type! { LengthVertical, Vertical }
define_length_type! { LengthBoth, Both }
We moved from having three newtypes of length-with-LengthDir to three newtypes with dir-as-associated-type.
Removing the newtypes and the macro
After that temporary refactoring, we had the Orientation trait and
the three zero-sized types Horizontal, Vertical, Both.
I figured out that one can use PhantomData as a way to carry
around the type that Length needs to normalize itself, instead of
using an associated type in an extra LengthTrait. Behold!
pub struct Length<O: Orientation> {
pub length: f64,
pub unit: LengthUnit,
orientation: PhantomData<O>,
}
impl<O: Orientation> Length<O> {
pub fn normalize(&self, values: &ComputedValues, params: &ViewParams) -> f64 {
match self.unit {
LengthUnit::Px => self.length,
LengthUnit::Percent => {
self.length
* <O as Orientation>::scaling_factor(params.view_box_width, params.view_box_height)
}
...
}
}
}
Now the incantation is <O as Orientation>::scaling_factor() to call
the method on the generic type; it is no longer an associated type in
a trait.
With that, users of lengths look like this; here, our <circle>
element from before:
pub struct Circle {
cx: Length<Horizontal>,
cy: Length<Vertical>,
r: Length<Both>,
}
I'm very happy with the readability of all the code now. I used to
think of PhantomData as a way to deal with wrapping pointers from
C, but it turns out that it is also useful to keep a generic
type around should one need it.
The final Length struct is this:
pub struct Length<O: Orientation> {
pub length: f64,
pub unit: LengthUnit,
orientation: PhantomData<O>,
}
And it only takes up as much space as its length and unit fields;
PhantomData is zero-sized after all.
(Later, we renamed Orientation to Normalize, but the code's
structure remained the same.)
Summary
Over a couple of years, librsvg's type that represents CSS lengths went from a C representation along the lines of "all data in the world is an int", to a Rust representation that uses some interesting type trickery:
-
C struct with
charfor units. -
C struct with a
LengthUnitsenum. -
C struct without an embodied direction; each place that needs to normalize needs to get the orientation right.
-
C struct with a built-in direction as an extra field, done at initialization time.
-
Same struct but in Rust.
-
An ugly but workable
Parsetrait so that the direction can be set at parse/initialization time. -
Three newtypes
LengthHorizontal,LengthVertical,LengthBothwith a common core. A cleaned-upParsetrait. A macro to generate those newtypes. -
Replace the
LengthDirenum with anOrientationtrait, and three zero-sized typesHorizontal/Vertical/Boththat implement the trait. -
Replace most of the macro with a helper trait
LengthTraitthat has anOrientationassociated type. -
Replace the helper trait with a single
Length<T: Orientation>type, which puts the orientation as a generic parameter. The macro disappears and there is a single implementation for everything.
Refactoring never ends!
Things are Looking Pod-tastic | Fall Time Blathering
Private and Air-Gap registry for openSUSE Kubic
Intro
Sometimes there are occasions where direct internet access is not possible (proxy/offline/airgapped). Even in this setups it is possible to deploy and use Kubernetes with openSUSE Kubic and a local private registry.
In this blog I will explain how to setup a local server which acts as private registry providing all the container images needed to deploy Kubernetes with openSUSE Kubic.
Installation
OS
As OS for our host serving the required services
openSUSE MicroOS is used, which needs
to be installed with the MicroOS Container Host system role.
The following additional packages are required and need to be installed:
- container-registry-systemd
- mirror-registry
- skopeo with the latest sync patches as provided in openSUSE Tumbleweed
- reg (optional, for testing and debugging)
# transactional-update pkg install mirror-registry skopeo container-registry-systemd reg
# systemctl reboot
After the reboot the tools are available and we can start with the setup of the registry.
Registry Setup
There is a script to setup a local private registry running in containers. By default, everybody can list and pull images from it, but only the admin is allowed to push new images.
# setup-container-registry
There is no default password for the admin account, which means it is necessary to set one to be able to sync the containers to this registries. The password must be a bcrypt (blowfish) hash:
# htpasswd -nB admin
Modify the password entry for the admin account and set the new hash:
# nano /etc/registry/auth_config.yml
Additional, the ACL rules can be adjusted.
Start the containers for the registry:
# systemctl start container-registry
# systemctl start registry-auth_server
Now your registry is running and should work. This can be verified with:
# reg ls localhost
Repositories for localhost
REPO TAGS
Certificates
By default setup-container-registry will create self signed
certificates, which are valid for 1,5 year and stored in
/etc/registry/certs. They can be replaced with official certificates.
If the self signed certificates are used, the public CA certificate needs
to be installed on all machines, which should access the registry. Copy
/etc/registry/certs/ContainerRegistryCA.crt to /etc/pki/trust/anchors/
and run update-ca-certificates on every machine.
List of container images
The tool mirror-registry will be used to create a list of containers
which needs to be mirrored. It will analyse a remote registry and
create a yaml file with all containers and tags matching a regex to
sync with skopeo to a private registry.
This tool can run on any architecture, the target platform can be specified as argument. In contrast to this, skopeo needs to run on the target architecture. Else it will fail or sync the wrong container image if there are multi-architecture container images in the source repository.
All container images below registry.opensuse.org/kubic/ are required.
Additional, it could be from help to mirror all official openSUSE
container images from registry.opensuse.org/opensuse/, too.
Else utilities like toolbox will not work.
The commnd to create a list of container images is:
# mirror-registry --out kubic-full.yaml registry.opensuse.org "(^kubic|^opensuse/([^/]+)$)"
This command will mirror all builds of all versions of the required container images. But most of the time it is enough to mirror the latest build of every version:
# mirror-registry --out kubic-small.yaml --minimize registry.opensuse.org "(^kubic|^opensuse/([^/]+)$)"
kubic-small.yaml can now be further tuned. If e.g. weave is used for the POD
network, flannel and the cilium containers don’t need to be mirrored
and can be deleted from the list.
Private registry
From a host, which has access to the source registry and the internal private registry, the container images can now be synced with skopeo:
# skopeo sync --scoped --src yaml --dest docker --dest-creds admin:password kubic-small.yaml registry.local
The --scoped option will prefix the internal registry name to the
current external repository name, so the repository will be
registry.opensuse.org/... and the pull command
podman pull registry.local/registry.opensuse.org/...
to avoid clashes with the namespace of the internal registry or by
mirroring additional registries.
The result can be verified with:
# reg ls registry.local
skopeo needs recent enough patches adding the sync option. This patches
are already part of openSUSE Tumbleweed, openSUSE MicroOS and openSUSE Kubic,
but not of SUSE Linux Enterprise Server 15.
Airgapped/offline registry
If there is no host which can access both registries, the external and the internal one, the container images needs to be copied to a disk, which then needs to be transfered into the internal network.
Copy the container images to an external disk:
# skopeo sync --scoped --src yaml --dest dir kubic-small.yaml /media/external-disk
Take the external disk to the internal registry machine and import the container images:
# skopeo sync --scoped --src dir --dest docker --dest-creds admin:password /media/external-disk localhost
Verify the result:
# reg ls registry.local
Node configuration
After the private registry is running and contains all container images, cri-o and podman need to be aware of this registry and pull the images from it.
Both tools share the same configuration file for this: /etc/containers/registries.conf.
The default configuration file is using the v1 syntax, but v2 is needed. The documentation can be found here. A working configuration template, which needs to be deployed on every kubernetes node, could look like:
[[registry]]
prefix = "registry.opensuse.org/kubic"
location = "registry.local/registry.opensuse.org/kubic"
insecure = false
blocked = false
[[registry]]
prefix = "registry.opensuse.org/opensuse"
location = "registry.local/registry.opensuse.org/opensuse"
insecure = false
blocked = false
- insecure:
trueorfalse. By default TLS is required when retrieving images from a registry. If insecure is set to true, unencrypted HTTP as well as TLS connections with untrusted certificates are allowed. - blocked :
trueorfalse. If true, pulling images with matching names is forbidden.
After updating the configuration file, cri-o needs to be restarted on that node if it is running:
# systemctl status crio
* crio.service - Open Container Initiative Daemon
Loaded: loaded (/usr/lib/systemd/system/crio.service; disabled; vendor preset: disabled)
Active: active (running) since Sat 2019-11-02 18:09:23 UTC; 8min ago
[...]
# systemctl stop crio
# systemctl start crio
Now you can deploy kubernetes!
Configuration files
Following configuration files define the behavior of the registry and the authentication server:
- /etc/registry/config.yml - configuration file for registry
- /etc/registry/auth_config.yml - configuration file for auth server
- /etc/sysconfig/container-registry - configuration file for the two containers
Laravel homestead box get corrupted!
I keep getting this same error message when I tried to run vagrant up on my machine
$ vagrant up
Bringing machine 'homestead' up with 'virtualbox' provider...
==> homestead: Importing base box 'laravel/homestead'...
Progress: 90%
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.
Command: ["import", "/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/box.ovf", "--vsys", "0", "--vmname", "ubuntu-18.04-amd64_1573673848596_76505", "--vsys", "0", "--unit", "11", "--disk", "/home/rnm/VirtualBox VMs/ubuntu-18.04-amd64_1573673848596_76505/ubuntu-18.04-amd64-disk001.vmdk"]
Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting /home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/box.ovf...
OK.
0%...
Progress state: NS_ERROR_INVALID_ARG
VBoxManage: error: Appliance import failed
VBoxManage: error: Code NS_ERROR_INVALID_ARG (0x80070057) - Invalid argument value (extended info not available)
VBoxManage: error: Context: "RTEXITCODE handleImportAppliance(HandlerArg*)" at line 957 of file VBoxManageAppliance.cpp
Arghh..this is quite anoyying.. I tried executing same command again and again but no luck. I check my disk space and there is plenty.
I have no choice, I will delete ~/.vagrant.d and execute vagrant command again
$ rm -rf ~/.vagrant.d
removed '/home/rnm/.vagrant.d/insecure_private_key'
removed '/home/rnm/.vagrant.d/rgloader/loader.rb'
removed directory '/home/rnm/.vagrant.d/rgloader'
removed directory '/home/rnm/.vagrant.d/tmp'
removed directory '/home/rnm/.vagrant.d/gems/2.4.9'
removed directory '/home/rnm/.vagrant.d/gems'
removed '/home/rnm/.vagrant.d/setup_version'
removed '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/ubuntu-18.04-amd64-disk001.vmdk'
removed '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/Vagrantfile'
removed '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/box.ovf'
removed '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox/metadata.json'
removed directory '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1/virtualbox'
removed directory '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/8.2.1'
removed '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead/metadata_url'
removed directory '/home/rnm/.vagrant.d/boxes/laravel-VAGRANTSLASH-homestead'
removed directory '/home/rnm/.vagrant.d/boxes'
removed '/home/rnm/.vagrant.d/data/machine-index/index.lock'
removed directory '/home/rnm/.vagrant.d/data/machine-index'
removed '/home/rnm/.vagrant.d/data/lock.dotlock.lock'
removed '/home/rnm/.vagrant.d/data/checkpoint_signature'
removed '/home/rnm/.vagrant.d/data/checkpoint_cache'
removed '/home/rnm/.vagrant.d/data/lock.machine-action-13105a2995d25174a20c9b1d206a4f8a.lock'
removed directory '/home/rnm/.vagrant.d/data'
removed directory '/home/rnm/.vagrant.d'
and when I execute vagrant, re-downloading the box then suddenly it worked!
$ vagrant up
Bringing machine 'homestead' up with 'virtualbox' provider...
==> homestead: Box 'laravel/homestead' could not be found. Attempting to find and install...
homestead: Box Provider: virtualbox
homestead: Box Version: >= 8.2.0
==> homestead: Loading metadata for box 'laravel/homestead'
homestead: URL: https://vagrantcloud.com/laravel/homestead
==> homestead: Adding box 'laravel/homestead' (v8.2.1) for provider: virtualbox
homestead: Downloading: https://vagrantcloud.com/laravel/boxes/homestead/versions/8.2.1/providers/virtualbox.box
homestead: Download redirected to host: vagrantcloud-files-production.s3.amazonaws.com
==> homestead: Successfully added box 'laravel/homestead' (v8.2.1) for 'virtualbox'!
==> homestead: Importing base box 'laravel/homestead'...
==> homestead: Matching MAC address for NAT networking...
==> homestead: Checking if box 'laravel/homestead' version '8.2.1' is up to date...
==> homestead: Setting the name of the VM: homestead
Vagrant is currently configured to create VirtualBox synced folders with
the `SharedFoldersEnableSymlinksCreate` option enabled. If the Vagrant
guest is not trusted, you may want to disable this option. For more
information on this option, please refer to the VirtualBox manual:
https://www.virtualbox.org/manual/ch04.html#sharedfolders
This option can be disabled globally with an environment variable:
VAGRANT_DISABLE_VBOXSYMLINKCREATE=1
or on a per folder basis within the Vagrantfile:
config.vm.synced_folder '/host/path', '/guest/path', SharedFoldersEnableSymlinksCreate: false
==> homestead: Clearing any previously set network interfaces...
==> homestead: Preparing network interfaces based on configuration...
homestead: Adapter 1: nat
homestead: Adapter 2: hostonly
==> homestead: Forwarding ports...
homestead: 80 (guest) => 8000 (host) (adapter 1)
homestead: 443 (guest) => 44300 (host) (adapter 1)
homestead: 3306 (guest) => 33060 (host) (adapter 1)
homestead: 4040 (guest) => 4040 (host) (adapter 1)
homestead: 5432 (guest) => 54320 (host) (adapter 1)
homestead: 8025 (guest) => 8025 (host) (adapter 1)
homestead: 9600 (guest) => 9600 (host) (adapter 1)
homestead: 27017 (guest) => 27017 (host) (adapter 1)
homestead: 22 (guest) => 2222 (host) (adapter 1)
==> homestead: Running 'pre-boot' VM customizations...
==> homestead: Booting VM...
==> homestead: Waiting for machine to boot. This may take a few minutes...
homestead: SSH address: 127.0.0.1:2222
homestead: SSH username: vagrant
homestead: SSH auth method: private key
homestead: Warning: Connection reset. Retrying...
homestead: Warning: Remote connection disconnect. Retrying...
homestead: Warning: Remote connection disconnect. Retrying...
homestead: Warning: Remote connection disconnect. Retrying...
homestead: Warning: Connection reset. Retrying...
homestead: Warning: Remote connection disconnect. Retrying...
homestead:
homestead: Vagrant insecure key detected. Vagrant will automatically replace
homestead: this with a newly generated keypair for better security.
homestead:
homestead: Inserting generated public key within guest...
homestead: Removing insecure key from the guest if it's present...
homestead: Key inserted! Disconnecting and reconnecting using new SSH key...
==> homestead: Machine booted and ready!
==> homestead: Checking for guest additions in VM...
==> homestead: Setting hostname...
==> homestead: Configuring and enabling network interfaces...
==> homestead: Mounting shared folders...
homestead: /vagrant => /home/rnm/Homestead
homestead: /home/vagrant/project1 => /home/rnm/workplace/laravel/project1
==> homestead: Running provisioner: file...
homestead: /home/rnm/Homestead/aliases => /tmp/bash_aliases
==> homestead: Running provisioner: shell...
homestead: Running: inline script
==> homestead: Running provisioner: shell...
homestead: Running: inline script
homestead:
homestead: ssh-rsa
==> homestead: Running provisioner: shell...
homestead: Running: inline script
==> homestead: Running provisioner: shell...
homestead: Running: inline script
==> homestead: Running provisioner: shell...
homestead: Running: inline script
==> homestead: Running provisioner: shell...
homestead: Running: inline script
homestead: Ignoring feature: mariadb because it is set to false
==> homestead: Running provisioner: shell...
homestead: Running: inline script
homestead: Ignoring feature: ohmyzsh because it is set to false
==> homestead: Running provisioner: shell...
homestead: Running: inline script
homestead: Ignoring feature: webdriver because it is set to false
==> homestead: Running provisioner: shell...
homestead: Running: /tmp/vagrant-shell20191114-1822-10idfmj.sh
==> homestead: Running provisioner: shell...
homestead: Running: /tmp/vagrant-shell20191114-1822-1pewpva.sh
==> homestead: Running provisioner: shell...
homestead: Running: script: Creating Certificate: project1.test
==> homestead: Running provisioner: shell...
homestead: Running: script: Creating Site: project1.test
==> homestead: Running provisioner: shell...
homestead: Running: inline script
==> homestead: Running provisioner: shell...
homestead: Running: /tmp/vagrant-shell20191114-1822-yobfu3.sh
==> homestead: Running provisioner: shell...
homestead: Running: script: Checking for old Schedule
==> homestead: Running provisioner: shell...
homestead: Running: script: Clear Variables
==> homestead: Running provisioner: shell...
homestead: Running: script: Restarting Cron
==> homestead: Running provisioner: shell...
homestead: Running: script: Restarting Nginx
==> homestead: Running provisioner: shell...
homestead: Running: script: Creating MySQL Database: project1
==> homestead: Running provisioner: shell...
homestead: Running: script: Creating Postgres Database: project1
==> homestead: Running provisioner: shell...
homestead: Running: script: Update Composer
homestead: Updating to version 1.9.1 (stable channel).
homestead:
homestead: Use composer self-update --rollback to return to version 1.9.0
==> homestead: Running provisioner: shell...
homestead: Running: /tmp/vagrant-shell20191114-1822-o4c7za.sh
==> homestead: Running provisioner: shell...
homestead: Running: script: Update motd
==> homestead: Running provisioner: shell...
homestead: Running: /tmp/vagrant-shell20191114-1822-13ajt5r.sh
nice!
$ vagrant global-status
id name provider state directory
--------------------------------------------------------------------------
dd946c7 homestead virtualbox running /home/rnm/Homestead
The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date (use "vagrant global-status --prune" to prune invalid
entries). To interact with any of the machines, you can go to that
directory and run Vagrant, or you can use the ID directly with
Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"
$ vagrant ssh
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-64-generic x86_64)
Thanks for using
_ _ _
| | | | | |
| |__ ___ _ __ ___ ___ ___| |_ ___ __ _ __| |
| '_ \ / _ \| '_ ` _ \ / _ \/ __| __/ _ \/ _` |/ _` |
| | | | (_) | | | | | | __/\__ \ || __/ (_| | (_| |
|_| |_|\___/|_| |_| |_|\___||___/\__\___|\__,_|\__,_|
* Homestead v9.2.2 released
* Settler v8.2.0 released
0 packages can be updated.
0 updates are security updates.
vagrant@homestead:~$
Seems, my vagrant box was corrupted and re-download the box is the solution.
Debugging on MicroOS made easier with toolbox
Until now, it was not easy to run debugging tools on openSUSE MicroOS or Kubic for the simple reason that installing any package using the transactional-update in package_name command would require a reboot for the program to be available on the system. Why so? MicroOS & Kubic use transactional updates and thus, the root filesystem is mounted as read-only. Packages that are installed are available in a "snapshot" which is selected at the next boot. openSUSE MicroOS & Kubic fall in the category of immutable operating systems.
One couldn't just reboot the system every time a simple program was needed - e.g using Vi to edit a YAML config or using database client utils for a rapid inspection.
But now we have a toolbox script available in MicroOS & Kubic that allows us to launch a container and use debugging utilities that can interact with the host. The script is written in Bash and it is based on CoreOS/toolbox.
Thorsten Kukuk, SUSE Engineer & Architect (SLES / MicroOS), announced the availability of the toolbox container on the openSUSE Kubic mailing list in October.
The toolbox script resides in the system $PATH and thus can be called directly. On its first execution, the script will pull the toolbox container from the openSUSE Container Registry.
microos:~ # toolbox
Trying to pull registry.opensuse.org/opensuse/toolbox...
Getting image source signatures
Copying blob 92a022f7c0d8 [>-------------------------------------] 2.8MiB / 122.9MiB
Copying blob f0c8107beec9 [==>-----------------------------------] 3.0MiB / 36.4MiB
The host root filesystem can be accessed at /media/root/ and since the toolbox container is spawned as the root user, the files will be modifiable.
toolbox:/ # ls /media/root/
bin dev home lib64 opt root sbin srv tmp var
boot etc lib mnt proc run selinux sys usr
Non-root toolbox containers will have read-only access to the host files, unless the --root flag is provided.
ish@microos:~> toolbox --root
Next, use zypper to install the required utilities and interact with the host and other containers.
Exiting toolbox will stop and the container and next time the script is called, the container is started again. Therefore, previously installed utilities remain available.
Each toolbox container is named by appending the username calling it, e.g toolbox-ish.
microos:~ # podman ps --format "{{.ID}} {{.Status}} {{.Names}}"
7f61fba95487 Up 10 minutes ago toolbox-ish
00ada606ec4f Up 33 minutes ago toolbox-root
CSS in librsvg is now in Rust, courtesy of Mozilla Servo
Summary: after an epic amount of refactoring, librsvg now does all CSS parsing and matching in Rust, without using libcroco. In addition, the CSS engine comes from Mozilla Servo, so it should be able to handle much more complex CSS than librsvg ever could before.
This is the story of CSS support in librsvg.
Introduction
The first commit to introduce CSS parsing in librsvg dates from 2002. It was as minimal as possible, written to support a small subset of what was then CSS2.
Librsvg handled CSS stylesheets more "piecing them apart" than
"parsing them". You know, when g_strsplit() is your best friend.
The basic parsing algorithm was to turn a stylesheet like this:
rect { fill: blue; }
.classname {
fill: green;
stroke-width: 4;
}
Into a hash table whose keys are strings like rect and .classname,
and whose values are everything inside curly braces.
The selector matching phase was equally simple. The code only handled a few possible match types as follows. If it wanted to match a certain kind of CSS selector, it would say, "what would this selector look like in CSS syntax", it would make up a string with that syntax, and compare it to the key strings it had stored in the hash table from above.
So, to match an element name selector, it would sprintf("%s",
element->name), obtain something like rect and see if the hash
table had such a key.
To match a class selector, it would sprintf(".%s",
element->class), obtain something like .classname, and look it up
in the hash table.
This scheme supported only a few combinations. It handled tag,
.class, tag.class, and a few combinations with #id in them.
This was enough to support very simple stylesheets.
The value corresponding to each key in the hash table was the stuff
between curly braces in the stylesheet, so the second rule from the
example above would contain fill: green; stroke-width: 4;. Once
librsvg decided that an SVG element matched that CSS rule, it would
re-parse the string with the CSS properties and apply them to the
element's style.
I'm amazed that so little code was enough to deal with a good number of SVG files with stylesheets. I suspect that this was due to a few things:
-
While people were using complex CSS in HTML all the time, it was less common for SVG...
-
... because CSS2 was somewhat new, and the SVG spec was still being written...
-
... and SVGs created with illustration programs don't really use stylesheets; they include the full style information inside each element instead of symbolically referencing it from a stylesheet.
From the kinds of bugs that librsvg has gotten around "CSS support is too limited", it feels like SVGs which use CSS features are either hand-written, or machine-generated from custom programs like data plotting software. Illustration programs tend to list all style properties explicitly in each SVG element, and don't use CSS.
Libcroco appears
The first commit to introduce libcroco was to do CSS parsing, from March 2003.
At the same time, libcroco was introducing code to do CSS matching. However, this code never got used in librsvg; it still kept its simple string-based matcher. Maybe libcroco's API was not ready?
Libcroco fell out of maintainership around the first half of 2005, and volunteers have kept fixing it since then.
Problems with librsvg's string matcher for CSS
The C implementation of CSS matching in librsvg remained basically untouched until 2018, when Paolo Borelli and I started porting the surrounding code to Rust.
I had a lot of trouble figuring out the concepts from the code. I didn't know all the terminology of CSS implementations, and librsvg didn't use it, either.
I think that librsvg's code suffered from what the refactoring
literature calls primitive
obsession.
Instead of having a parsed representation of CSS selectors, librsvg
just stored a stringified version of them. So, a selector like
rect#classname really was stored with a string like that, instead of
an actual decomposition into structs.
Moreover, things were misnamed. This is the field that stored stylesheet data inside an RsvgHandle:
GHashTable *css_props;
From just looking at the field declaration, this doesn't tell me anything about what kind of data is stored there. One has to grep the source code for where that field is used:
static void
rsvg_css_define_style (RsvgHandle * ctx,
const gchar * selector,
const gchar * style_name,
const gchar * style_value,
gboolean important)
{
GHashTable *styles;
styles = g_hash_table_lookup (ctx->priv->css_props, selector);
Okay, it looks up a selector by name in the css_props, and it
gives back... another hash table styles? What's in there?
g_hash_table_insert (styles,
g_strdup (style_name),
style_value_data_new (style_value, important));
Another string key called style_name, whose key is a
StyleValueData; what's in it?
typedef struct _StyleValueData {
gchar *value;
gboolean important;
} StyleValueData;
The value is another string. Strings all the way!
At the time, I didn't really figure out what each level of nested hash
tables was supposed to mean. I didn't understand why we handled style
properties in a completely different part of the code, and yet this
part had a css_props field that didn't seem to store properties at all.
It took a while to realize that css_props was misnamed. It wasn't
storing a mapping of selector names to properties; it was storing a
mapping of selector names to declaration lists, which are lists of
property/value pairs.
So, when I started porting the CSS parsing code to Rust, I started to create real types with for each concept.
// Maps property_name -> Declaration
type DeclarationList = HashMap<String, Declaration>;
pub struct CssStyles {
selectors_to_declarations: HashMap<String, DeclarationList>,
}
Even though the keys of those HashMaps are still strings, because librsvg didn't have a better way to represent their corresponding concepts, at least those declarations let one see what the hell is being stored without grepping the rest of the code. This is a part of the code that I didn't really touch very much, so it was nice to have that reminder.
The first port of the CSS matching code to
Rust
kept the same algorithm as the C code, the one that created strings
with element.class and compared them to the stored selector names.
Ugly, but it still worked in the same limited fashion.
Rustifying the CSS parsers
It turns out that CSS parsing is divided in two parts. One can have a
style attribute inside an element, for example
<rect x="0" y="0" width="100" height="100"
style="fill: green; stroke: magenta; stroke-width: 4;"/>
This is a plain declaration list which is not associated to any selectors, and which is applied directly to just the element in which it appears.
Then, there is the <style> element itself, with a normal-looking CSS stylesheet
<style type="text/css">
rect {
fill: green;
stroke: magenta;
stroke-width: 4;
}
</style>
This means that all <rect> elements will get that style applied.
I started to look for existing Rust crates to parse and handle CSS data. The cssparser and selectors crates come from Mozilla, so I thought they should do a pretty good job of things.
And they do! Except that they are not a drop-in replacement for anything. They are what gets used in Mozilla's Servo browser engine, so they are optimized to hell, and the code can be pretty intimidating.
Out of the box, cssparser provides a CSS tokenizer, but it does not know how to handle any properties/values in particular. One must use the tokenizer to implement a parser for each kind of CSS property one wants to support — Servo has mountains of code for all of HTML's style properties, and librsvg had to provide a smaller mountain of code for SVG style properties.
Thus started the big task of porting librsvg's string-based parsers
for CSS properties into ones based on cssparser tokens. Cssparser
provides a Parser struct, which extracts tokens out of a CSS
stream. Out of this, librsvg defines a Parse trait for parsable
things:
use cssparser::Parser;
pub trait Parse: Sized {
type Err;
fn parse(parser: &mut Parser<'_, '_>) -> Result<Self, Self::Err>;
}
What's with those two default lifetimes in Parser<'_, '_>?
Cssparser tries very hard to be a zero-copy tokenizer. One of the
lifetimes refers to the input string which is wrapped in a
Tokenizer, which is wrapped in a ParserInput. The other lifetime
is for the ParserInput itself.
In the actual implementation of that trait, the Err type also uses
the lifetime that refers to the input string. For example, there is a
BasicParseErrorKind::UnexpectedToken(Token<'i>), which one returns
when there is an unexpected token. And to avoid copying the substring
into the error, one returns a slice reference into the original
string, thus the lifetime.
I was more of a Rust newbie back then, and it was very hard to make sense of how cssparser was meant to be used.
The process was more or less this:
-
Port the C parsers to Rust; implement types for each CSS property.
-
Port the
&str-based parsers into ones that usecssparser. -
Fix the error handling scheme to match what cssparser's high-level traits expect.
This last point was... hard. Again, I wasn't comfortable enough with Rust lifetimes and nested generics; in the end it was all right.
Moving declaration lists to Rust
With the individual parsers for CSS properties done, and with them already using a different type for each property, the next thing was to implement cssparser's traits to parse declaration lists.
Again, a declaration list looks like this:
fill: blue;
stroke-width: 4;
It's essentially a key/value list.
The trait that cssparser wants us to implement is this:
pub trait DeclarationParser<'i> {
type Declaration;
type Error: 'i;
fn parse_value<'t>(
&mut self,
name: CowRcStr<'i>,
input: &mut Parser<'i, 't>,
) -> Result<Self::Declaration, ParseError<'i, Self::Error>>;
}
That is, define a type for a Declaration, and implement a
parse_value() method that takes a name and a Parser, and outputs
a Declaration or an error.
What this really means is that the type you implement for
Declaration needs to be able to represent all the CSS property types
that you care about. Thus, a struct plus a big enum like this:
pub struct Declaration {
pub prop_name: String,
pub property: ParsedProperty,
pub important: bool,
}
pub enum ParsedProperty {
BaselineShift(SpecifiedValue<BaselineShift>),
ClipPath(SpecifiedValue<ClipPath>),
ClipRule(SpecifiedValue<ClipRule>),
Color(SpecifiedValue<Color>),
ColorInterpolationFilters(SpecifiedValue<ColorInterpolationFilters>),
Direction(SpecifiedValue<Direction>),
...
}
This gives us declaration lists (the stuff inside curly braces in a CSS stylesheet), but it doesn't give us qualified rules, which are composed of selector names plus a declaration list.
Refactoring towards real CSS concepts
Paolo Borelli has been steadily refactoring librsvg and fixing things like the primitive obsession I mentioned above. We now have real concepts like a Document, Stylesheet, QualifiedRule, Rule, AtRule.
This refactoring took a long time, because it involved redoing the XML loading code and its interaction with the CSS parser a few times.
Implementing traits from the selectors crate
The selectors crate contains Servo's code for parsing CSS selectors and doing matching. However, it is extremely generic. Using it involves implementing a good number of concepts.
For example, this SelectorImpl trait has no methods, and is just a
collection of types that refer to your implementation of an element
tree. How do you represent an attribute/value? How do you represent
an identifier? How do you represent a namespace and a local name?
pub trait SelectorImpl {
type ExtraMatchingData: ...;
type AttrValue: ...;
type Identifier: ...;
type ClassName: ...;
type PartName: ...;
type LocalName: ...;
type NamespaceUrl: ...;
type NamespacePrefix: ...;
type BorrowedNamespaceUrl: ...;
type BorrowedLocalName: ...;
type NonTSPseudoClass: ...;
type PseudoElement: ...;
}
A lot of those can be String, but Servo has smarter things in store.
I ended up using the markup5ever crate, which provides a string
interning framework for markup and XML concepts like a LocalName, a
Namespace, etc. This reduces memory consumption, because instead of
storing string copies of element names everywhere, one just stores
tokens for interned strings.
(In the meantime I had to implement support for XML namespaces, which the selectors code really wants, but which librsvg never supported.)
Then, the selectors crate wants you to say how your code implements an
element tree. It has a monster trait Element:
pub trait Element {
type Impl: SelectorImpl;
fn opaque(&self) -> OpaqueElement;
fn parent_element(&self) -> Option<Self>;
fn parent_node_is_shadow_root(&self) -> bool;
...
fn prev_sibling_element(&self) -> Option<Self>;
fn next_sibling_element(&self) -> Option<Self>;
fn has_local_name(
&self,
local_name: &<Self::Impl as SelectorImpl>::BorrowedLocalName
) -> bool;
fn has_id(
&self,
id: &<Self::Impl as SelectorImpl>::Identifier,
case_sensitivity: CaseSensitivity,
) -> bool;
...
}
That is, when you provide an implementation of Element and
SelectorImpl, the selectors crate will know how to navigate your
element tree and ask it questions like, "does this element have the id
#foo?"; "does this element have the name rect?". It makes perfect
sense in the end, but it is quite intimidating when you are not 100%
comfortable with webs of traits and associated types and generics with
a bunch of trait bounds!
I tried implementing that trait twice in the last year, and failed. It turns out that its API needed a key fix that landed last June, but I didn't notice until a couple of weeks ago.
So?
Two days ago, Paolo and I committed the last code to be able to completely replace libcroco.
And, after implementing CSS specificity (which was easy now that we have real CSS concepts and a good pipeline for the CSS cascade), a bunch of very old bugs started falling down (1 2 3 4 5 6).
Now it is going to be easy to implement things like letting the application specify a user stylesheet. In particular, this should let GTK remove the rather egregious hack it has to recolor SVG icons while using librsvg indirectly.
Conclusion
This will appear in librsvg 2.47.1 — that version will no longer require libcroco.
As far as I know, the only module that still depends on libcroco (in GNOME or otherwise) is gnome-shell. It uses libcroco to parse CSS and get the basic structure of selectors so it can implement matching by hand.
Gnome-shell has some code which looks awfully similar to what librsvg had when it was written in C:
-
StTheme has the high-level CSS stylesheet parser and the selector matching code.
-
StThemeNode has the low-level CSS property parsers.
... and it turns out that those files come all the way from HippoCanvas, the CSS-aware canvas that Mugshot used! Mugshot was a circa-2006 pre-Facebook aggregator for social media data like blogs, Flickr pictures, etc. HippoCanvas also got used in Sugar, the GUI for One Laptop Per Child. Yes, our code is that old.
Libcroco is unmaintained, and has outstanding CVEs. I would be very happy to assist someone in porting gnome-shell's CSS code to Rust :)
Clang precompiled headers and improving C++ compile times, take #2
It turns out, the problems I mentioned last time had already been more or less solved in Clang. But only for C++ modules, not for precompiled headers. *sigh* I had really mixed feelings when I finally realized that. First of all, not knowing Clang internals that well, it took me quite a long time to get to this point figuring it all out, probably longer than it could have. Second, I've been using C++ modules when building Clang itself and while it's usable, I don't consider it ready (for example, sometimes it actually makes the build slower), not to mention that it's non-trivial to setup, not standardized yet and other compilers (AFAIK) do not yet support C++ modules. And finally, WTH has nobody else yet noticed and done the work for precompiled headers too? After all the trouble with finding out how the relevant Clang parts work, the necessary patches mostly border on being trivial. Which, on the other hand, is at least the good news.
And so I'm switching for LibreOffice building to my patched build of Clang. For the motivation, maybe let's start with an updated picture from the last time:
This is again column2.cxx, a larger C++ file from Calc. The first row is again compilation without any PCH involved. The second row is unpatched Clang with --enable-pch=full, showing again that way too large PCHs do not really pay off (here it does, because the source file is large, but for small ones such as bcaslots.cxx shown last time it makes things slower). In case you notice the orange 'DebugType' in the third row that looks like it should be in the second row too, it should be there, but that's one of these patches of mine that the openSUSE package does not have.
The third row is with one patch that does the PerformPendingInstantiations phase also already while building the PCH. The patch is pretty much a one-liner when not counting handling fallout from some Clang tests failing because of stuff getting slightly reordered because of this. Even by now I still don't understand why PCH generation had to delay this until every single compilation using the PCH. The commit introducing this had a commit message that didn't make much sense to me, the test it added works perfectly fine now. Presumably it's been fixed by the C++ modules work. Well, who cares, it apparently works.
The last row adds also Clang options -fmodules-codegen -fmodules-debuginfo. They do pretty much what I was trying to achieve with my original patch, they just approach the problem from a different side (and they also do not have the technical problems that made me abandon my approach ...). They normally work only for C++ modules, so that needed another patch, plus a patch fixing some problems. Since this makes Clang emit all kinds of stuff from the PCH into one specific object file in the hopes that all the compilations using the PCH will need that too but will be able to reuse the shared code instead, LibreOffice now also needs to link with --gc-sections, which throws away all the possibly problematic parts where Clang guessed wrong. But hey, it works. Even with ccache and Icecream (if you have the latest Icecream, that is, and don't mind that it "implements" PCHs for remote compilations by simply throwing the PCH away ... it still pays off).
So, that it's for a single compilation. How much does it help with building in practice? Time for more pretty colorful pictures:
Note that GCC(v9) and MSVC(v2017) are there more as a reference than a fair comparison. MSVC runs on a different OS and the build may be possibly slightly handicaped by some things taking longer in Cygwin/Windows. GCC comes from its openSUSE package, which AFAICT is built without LTO (unlike the Clang package, where it makes a noticeable difference).
And in case the graphs don't seem impressive enough, here's one for Library_sc, which with its 598 source files is too big for me to bother measuring it in all cases. This is the difference PCHs can make. That's 11:37 to 4:34, almost down to one third:
As for building entire LO from scratch, it can be like in the picture below (or even better). The effect there is smaller, because the build consists of other things than just building libraries, and some of the code built doesn't use PCHs. And it's even smaller than it could be, because I used --enable-pch=base, as that's what I've been using up to now (although now I'll switch to a higher level). That's about 1h42m without PCHs to 1h14m with unpatched Clang (27% percent saved), and 1h06m with patches (and the 8minutes difference is still 11% of the unpatched time). Not bad, given that this is the entire LO build. Those 6 minutes for ccache are there to show the maximum possible improvement (or rather nowhere near possible, since the compiler normally still has to do the work of actually compiling the code somehow).
In case you'd want to use this too, that's not up to me now. The patches are now sitting and waiting in the LLVM Phabricator. Hopefully somebody there still cares about PCHs too.
Hotels and Operating Systems
macOS is like a hotel. Everything is polished, designed and cared for. But it’s also sterile and you can’t bring your own furniture or cook your own food. On Linux, you need to do the dishes and take out the trash yourself. But it’s yours. No one forces their agenda on you. It simply feels like home.
Going from macOS to Ubuntu | kvz.io
Noodlings 8 | DLN Xtend, Universal Packages and CAD
Understand networking in Podman
I received a message on Twitter on 17 October from a fellow who attended the openSUSE Asia Summit 2019. Strangely, I didn't get any notification about it and it's only today that I read the message. He also attended my workshop on openSUSE MicroOS and had some questions regarding inter-Pod communication.
As a quick response, I explained him very breifly about "container networking" and pointed him to the Kubernetes documentation on IP allocation. I do realize though, that most of the times, documenation can be lengthy and {boring}, and that you would just want a simple article or blog post that clears your doubts.
Tell me about Podman networks
Rootless containers (i.e containers started using Podman as a regular user) do not obtain an IP address. Podman uses slirp4netns to allow Internet connectivity inside the container.
Communication with a rootless container is achieved by mapping the container ports to the host, e.g using -p 8080:80 to map a webserver port 80 to the host on port port 8080.
$ podman run -dt --name webserver -p 8080:80 nginx
$ curl http://localhost:8080Therefore, two rootless containers can communicate over their published ports on the host. Let's experiment this by starting an openSUSE Leap container and installing the telnet package.
$ podman run -dt --name leap leap
$ podman exec -it leap bash
4a0f95e011b9:/ # zypper in telnetWe run ip a s on the host to find its IP address. Say the IP address is 192.168.100.8. Now, from within Leap container let's telnet port 8080 over the host IP.
4a0f95e011b9:/ # telnet 192.168.100.10 8080
Trying 192.168.100.10...
Connected to 192.168.100.10.
Escape character is '^]'.The connection went through successfully, meaning from the Leap container we've been able to access the Nginx container through it's mapped port on the host.
This same experiment can be repeated using two different pods, say you have a pod that contains your web services and another pod that contains your databases.
$ podman pod create --name webservice -p 8080:80
$ podman run -dt --name webserver --pod webservice nginx
$ podman pod create --name db -p 3306:3306
$ podman run -dt --name mariadb --pod db -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" mariadbThe Nginx container will be able to reach the MariaDB database over 192.168.100.10:3306 as the same port is mapped on the host.
Ideally, these two containers could have been created in the same pod and therefore share the same network space. Then, the Nginx container would reach the database over localhost:3306 easily.
I used the above Nginx/MariaDB example to explain rootless inter-Pod communication, which was the question that was asked to me initially.
What about rootfull containers?
Rootfull containers are those that are created using Podman with root privileges, either by the root user itself or using sudo privilege.
Containers created using Podman with root privileges obtain an IP address. Podman then uses the Container Network Interfec (CNI) instead of slirp4netns for networking provisioning.
Details about the network subnet is found in the CNI config file.
$ cat /etc/cni/net.d/87-podman-bridge.conflist
{
"cniVersion": "0.3.0",
"name": "podman",
"plugins": [
{
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"subnet": "10.88.0.0/16",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}So, let's start a container with root privileges and see.
$ sudo podman run -dt --name db postgres
$ sudo podman inspect -f "{{.NetworkSettings.IPAddress}}" db
10.88.0.30The podman inspect ... command returns the container's IP address and the same is ranged within the subnet specified in the CNI config. The PostgreSQL database can be accessed over 10.88.0.30:5432 from the host or from within any other container started using root privileges.
$ telnet 10.88.0.30 5432
Trying 10.88.0.30...
Connected to 10.88.0.30.
Escape character is '^]'.



