Advanced dehydration
Back when i started using Let’s Encrypt I wanted a lightweight client. Danimo recommended dehydrated. Well it still had a different name back then. But details. So for years I had been a very happy camper with it. Then I learned that many daemons now support that you can run ECDSA and RSA certs at the same time and the server picks the right cert depending on the client. So we will solve this first.
What I learned from Russian students: logging is important
When I published my blog about openSUSE a couple of weeks ago, most questions I received in private were about the Russian students I mentioned. In that blog I quickly described how my interest in information security started, about 25 years ago. This blog gives you a bit of historical background and a few more details.
Historical background
It was 1995. I was studying at a university, but I was already running one of the servers of the faculty. It was a Linux box, and I also helped to run a FreeBSD server hosting the faculty web server. It was just three years after the Soviet army finally left Hungary. Our university had many students from Russia. While Hungarian students could attend the university for free, Russian students had to pay for their studies. As they were paying a lot, they could do anything, nobody punished their activities. And they did a lot of things, as they felt that they can still do anything in the ‘colonies’.
It was 1995, there was no Internet yet in the student dormitories. There was no Gmail, or any similar provider yet. Not even teachers received e-mail addresses automatically. Even if some people had computers at home, there was no Internet access yet from homes. Students could access servers from computer labs at the university. The Russian students had their own computer lab, where nobody else was allowed to enter.
It is 1995, the fifth consecutive year that funding was taken away from higher education. Which meant that faculties started to ask money from other faculties for their services. Russian students belonged to another faculty, so they could not get a user name on our servers.
Infosec is overrated
By that time even if I was running a couple of servers, I was just the same as the vast majority users even today. I mean, I thought that information security is overrated, ease of use, comfort are a way lot more important. It did not help either, that most of the commonly used protocols were not encrypted, like telnet, ftp, rsh and others. Even these protocols were often difficult to use from Windows machines. I was learning Linux and FreeBSD, and I was enabling all kinds of services. Using rsh between the two faculty UNIX servers was fun.
Logging is important
I checked the logfiles of the servers I managed occasionally, but mostly only to check if the hard drives were showing any signs of failure. While browsing the logs for hard drive errors, I came across some suspicious login messages. Logins from previously not seen unknown IP addresses. I knew that the addresses were from campus, so I asked around. It turned out, that they belonged to the Russian students laboratory. And talking to the user it turned out, that he was unaware that his account was used also by someone else.
The exact order of events is a kind of blurry, it was a quarter of a century ago. I started to check log messages not just for hard drive problems but also for security related events. I could see more and more logins from the Russian students laboratory. It was a kind of cat and mouse game, I was trying to keep unauthorized users out of the system. They kept coming back and started to do nasty things. Along the way I learned a lot about security:
- Network sniffing: most of the university had a BNC network and was using hubs instead of switches. Combine these with non-encrypted protocols…
- Keyboard loggers
- Black market. Access for students of our faculty was free, they just had to ask for it. Sometimes minutes after they received access, there was a login from the Russian lab. Accounts on my servers had a good price…
- Denial of Service: they tried all kinds of DoS attacks, like fork bombs, too many logins, etc.
- Stepping stone for further attacks, so I got some not so kind e-mails asking for explanation
Turning on a firewall could have been an easy way out, but seeing the IP addresses of the Russian lab in the system logs was the perfect indicator of compromise for an account. The account got quickly disabled, either for life (see black market) or until a password change. In the second case I tried to investigate, how the password was stolen. And of course gave a quick education on security awareness. Showing my log messages I tried to ask for some help to stop the Russian students, but as I was just a first year student and Russian students were paying: nobody cared.
Next steps
After so many years I do not recall any more how I got the hint, but I was suggested that I visit the Russian students computer lab. I was not supposed to enter there, but as they were messing with my servers, I did not care. The door was open, I walked in and looked around. The /etc/passwd file of my Linux box was printed on the wall. Even if encrypted, but it contained the passwords. As also described in my openSUSE blog, this was a final push towards information security.
FreeBSD already had passwords separate from the user readable passwd file, so I knew the concept. I looked around and found that the Linux distribution called Jurix had shadow passwords. It was a brand new thing in the Linux world at that time. I quickly migrated my Linux server to Jurix and did all kinds of hardening along the way. I removed all non-essential services, like rsh. Even if most users kept using telnet and other insecure services, I started to use SSH, which was just released.
When Russian students realized that they cannot get into my servers easily any more, they even tried to bribe me for access – with a counterfeit gaming CD for Windows :-)
Epilogue
As you can see, I ended up on the defender side. I did lots of security hardening and built systems that ran securely even years after I abandoned them. Logging still takes an important role in my life: I work with syslog-ng. Russian students were a major PITA at that time, but I learned a lot about security while I was trying to keep them out of the servers I managed.
opensource.com: What was your first programming language?
A couple of weeks ago editors of https://opensource.com/ sent a question to contributors: What was your first programming language? Thinking about the question brought back some nice memories about the beginnings. You can read my answer below:
What was your first programming language?
My first ever programming language was BASIC in the early eighties. One of my relatives bought a C64 for their kids to get started with learning computers. They only used it for gaming, and I was also invited. But they also had a book about BASIC, and I was curious and gave it a try. I wrote some short code, I did not even know how to save it, but it was exciting to see that the computer does what I say to it. This means that I was not paid to learn it, and it was not my choice. It was the language available to me. Obviously, when I got my first computer a few years later, an XT compatible box, I first wrote some code in GW-BASIC, the dialect of BASIC available with DOS.
What happened next?
The first time I really choose a programming language was Pascal. I asked around, checked some books, and it seemed to be a good compromise between features and difficulty. First, it was Turbo Pascal, and I coded all kinds of simple games and graphics in it. I loved Pascal, so in my university years, I even used it (well, FreePascal and Lazarus) for measurement automation and modeling how pollution spreads in groundwater.
You can read the rest of the answers at https://opensource.com/article/21/8/first-programming-language
The syslog-ng insider 2021-07: Alerting; CentOS alternatives; MongoDB;
Better late than never I just put online the July syslog-ng newsletter. Topics include:
- Sending alerts to Discord and others from syslog-ng using Apprise: blocks and Python templates
- Rocky Linux, AlmaLinux, CentOS & syslog-ng
- MongoDB support improved in syslog-ng 3.32
It is available at https://www.syslog-ng.com/community/b/blog/posts/insider-2021-07-alerting-centos-alternatives-mongodb
Null Amusement
Been old-school-pixel-pushin’ recently, for a project I’ll hopefully reveal soon. But because I’ve also been really keen on music recently, a little side diversion happened that I’d like to share.
It started with generating some samples using the good old SXFR. It’s a tiny sythesizer for generating cute oldschool chiptune/game sound effects. Sadly it doesn’t exist as an app, which is a shame.

Then it was off to Polyend Tracker, even if in this case I was sitting in front of my computer so the benefit of having a physical machine to take anywhere was sort of diminished.

In any case the included instrument editor with all its effects and filters came handy. I exported the full song as well as a single pattern to use on the Instagram loop. This time I did no mastering at all, just plain uploaded the track to soundcloud and called it done.
As for the visuals, I’ve used way many more tools than you’d expect. The icon assets were mainly done in Pixaki, a fairly polished pixel editor for iPad. I have numerous beef with it for it being premium priced, mainly in the way it does layered animation, but it absolutely delivers on the immediacy and contrasts with filesystem diving of Aseprite which otherwise beats it bar none. Usually I convert GIFs exported from Aseprite or Pixaki using ffmpeg, but this time I needed to sync the animation to the sound, so I loaded up the GIF and the exported pattern from Tracker into Blender and with the sound wave preview that somehow isn’t on by default, it was a quick job in the VSE.

Previously, Previously, Previously, Previously, Previously, Previously.
Alibaba Summer of Code 2020 – Arthas Final Report
Project information
Project name
Provide a separate tutorial for each Arthas command
Project task description
There are two parts in the old online tutorial: basic and advanced
- https://arthas.aliyun.com/doc/arthas-tutorials.html?language=en&id=arthas-basics
- https://arthas.aliyun.com/doc/arthas-tutorials.html?language=en&id=arthas-advanced
Research about the online tutorial: #742
- Each little tutorial is a little longer, and adding new content will cause the tutorial to get longer and longer.
- Users tend to lose patience.
- It’s hard to find what you need.
- More users want to see the usage of a command directly
So consider:
- Provide a separate tutorial for each command, such as a separate tutorial for the
watchcommand, so that many tips can be written in. - One tutorial per case to make it easier for users to learn to use Arthas.
What we need to:
- Move tutorial git Repository https://github.com/hengyunabc/katacoda-scenarios to Arthas itself.
- Seperate the tutorial, then send the PR merge to https://github.com/alibaba/arthas/tree/master/tutorials/katacoda
- Update the web page: https://github.com/alibaba/arthas/blob/master/site/src/site/sphinx/_include_html/arthas-tutorials.html
- Add links to the online tutorials for the official documents: https://github.com/alibaba/arthas/tree/master/site/src/site/sphinx/*.md
Implementation plan
- Move tutorial git Repository https://github.com/hengyunabc/katacoda-scenarios to Arthas itself:
After reading katacoda’s official documentation, I added the katacoda.yaml at root in Arthas repository where I specified the katacoda tutorial is located. Then all tutorials https://github.com/hengyunabc/katacoda-scenarios have been migrated to /tutorials/katacoda/ directory.
- Seperate the tutorial, provide a separate tutorial for each Arthas command
Refer to https://arthas.aliyun.com/doc/commands.html . At the same time, the command and user cases in the original tutorial are integrated and classified, and the new version of Arthas tutorial is made. The structure of each command and case tutorial is as follows:
Start demo ->Start arthas-boot ->…[details on how to use each command]…
At the same time, the user case related to the command is integrated into the command tutorial, and it is placed in a separate case tutorial for users to query.
In addition, I have integrated some related special usages. Referring to https://github.com/alibaba/arthas/issues?q=label%3Auser-case , some additional cases have been produced.
- Update the web page
- The new online tutorials include “Arthas Basics” and “Arthas Advanced” from the old online tutorials, which are placed in the “Tutorials” menu bar.
- Each related command tutorial is placed in the menu by it’s classification. I also place typical user cases online tutorials in the “User Cases” menu bar.
New version menu design:

- Add links to the online tutorials for the official documents

- Usages
Katacoda is an interactive learning and training platform for software engineers. It can use real-world environment to learn and test new technologies in the browser to help developers learn and master best practices.
Katacoda can quickly provide a temporary environment and recycle it after use. Users can operate a complete set of environment through the browser terminal interface according to the designed guidance steps, and learn and practice step by step.
The new Arthas online tutorials provide a very convenient way to learn. You just need to open the corresponding courses, and you can follow the course instructions and complete the learning step by step according to the designed steps.
- First visit the online tutorial: https://arthas.aliyun.com/doc/arthas-tutorials.html?language=en , select the course you want to study from the menu:
- The course introduction page will indicate the difficulty of the course and the time required to help you understand the basic information of the course. click
START SCENARIOto start learning.
- Enter the course, the left side is the description of this step, and the right side is a ready terminal, which can be used directly. Click the black blocks on the left to execute commands in the right:
- Click the tab on the right to switch between terminals. Then follow the step-by-step instructions to complete the learning step by step:
Milestone review
Half of the commands and user tutorials were merged into the upstream before July 31, and all commands and user tutorials were merged into the upstream on August 14, and then it was online for testing.
Due to the large scale of this project, the completion of each command online tutorial can be regarded as a milestone node.
During the project, more than 80 PRs were sent to Arthas project, with a total increase or decrease of more than 26k lines.

During the test, the number of visits to the tutorial increased dramatically, and was well received by all the users!
- Well received by users:

- Statistics of the visit and engagement duration of the tutorials. It can be seen that after the new version of the tutorial test was launched on August 14, users and engagement duration have increased significantly:

Project summary
Project deliverables
Move Current Tutorial
Adding Tutorial
User Cases
- [X] Web Console
- [X] Chinese
- [X] English
- [X] PR #1332
- [X] HTTP API
- [X] Chinese
- [X] English
- [X] PR #1423
- [X] Arthas boot supported options
- [X] Chinese
- [X] English
- [X] PR #1344
- [X] Troubleshooting method invoke exception
- [X] Chinese
- [X] English
- [X] PR #1335
- [X] Hotswap code
- [X] Chinese
- [X] English
- [X] PR #1339
- [X] Change Logger Level
- [X] Chinese
- [X] English
- [X] PR #1337
- [X] Troubleshoot logger conflicts
- [X] Chinese
- [X] English
- [X] PR #1338
- [X] Get Spring Context
- [X] Chinese
- [X] English
- [X] PR #1342
- [X] Troubleshooting HTTP request returns 401
- [X] Chinese
- [X] English
- [X] PR #1341
- [X] Troubleshooting HTTP request returns 404
- [X] Chinese
- [X] English
- [X] PR #1340
- [X] The ClassLoaders in Spring Boot application
- [X] Chinese
- [X] English
- [X] PR #1343
- [X] Find CPU usage Top N threads
- [X] Chinese
- [X] English
- [X] PR #1336
- [X] Log the output
- [X] Chinese
- [X] English
- [X] PR #1334
- [X] Async in Background
- [X] Chinese
- [X] English
- [X] PR #1354
Command
- [X] cls
- [X] Chinese
- [X] English
- [X] PR #1312
- [X] session
- [X] Chinese
- [X] English
- [X] PR #1313
- [X] reset
- [X] Chinese
- [X] English
- [X] PR #1314
- [X] version
- [X] Chinese
- [X] English
- [X] PR #1315
- [X] history
- [X] Chinese
- [X] English
- [X] PR #1316
- [X] quit-stop
- [X] Chinese
- [X] English
- [X] PR #1317
- [X] keymap
- [X] Chinese
- [X] English
- [X] PR #1318
- [X] cat
- [X] Chinese
- [X] English
- [X] PR #1319
- [X] echo
- [X] Chinese
- [X] English
- [X] PR #1320
- [X] grep
- [X] Chinese
- [X] English
- [X] PR #1321
- [X] tee
- [X] Chinese
- [X] English
- [X] PR #1322
- [X] pwd
- [X] Chinese
- [X] English
- [X] PR #1323
- [X] wc
- [X] Chinese
- [X] English
- [X] PR #1384
- [X] plaintext
- [X] Chinese
- [X] English
- [X] PR #1383
- [x] dashboard
- [x] Chinese
- [x] English
- [X] PR #1326
- [x] thread
- [x] Chinese
- [x] English
- [X] PR #1328
- [x] jvm
- [x] Chinese
- [x] English
- [X] PR #1329
- [X] sysprop
- [X] Chinese
- [X] English
- [X] PR #1331
- [X] sysenv
- [X] Chinese
- [X] English
- [X] PR #1330
- [X] vmoption
- [X] Chinese
- [X] English
- [X] PR #1347
- [X] perfcounter
- [X] Chinese
- [X] English
- [X] PR #1352
- [X] logger
- [X] Chinese
- [X] English
- [X] PR #1365
- [X] mbean
- [X] Chinese
- [X] English
- [X] PR #1357
- [X] getstatic
- [X] Chinese
- [X] English
- [X] PR #1368
- [X] ognl
- [X] Chinese
- [X] English
- [X] PR #1327
- [X] sc
- [X] Chinese
- [X] English
- [X] PR #1370
- [X] sm
- [X] Chinese
- [X] English
- [X] PR #1371
- [X] dump
- [X] Chinese
- [X] English
- [X] PR #1372
- [X] heapdump
- [X] Chinese
- [X] English
- [X] PR #1373
- [X] jad
- [X] Chinese
- [X] English
- [X] PR #1374
- [X] classloader
- [X] Chinese
- [X] English
- [X] PR #1375
- [X] mc-redefine
- [X] Chinese
- [X] English
- [X] PR #1376
- [X] monitor
- [X] Chinese
- [X] English
- [X] PR #1377
- [X] watch
- [X] Chinese
- [X] English
- [X] PR #1378
- [X] trace
- [X] Chinese
- [X] English
- [X] PR #1379
- [X] stack
- [X] Chinese
- [X] English
- [X] PR #1380
- [X] tt
- [X] Chinese
- [X] English
- [X] PR #1381
- [X] options
- [X] Chinese
- [X] English
- [X] PR #1382
- [X] profiler
- [X] Chinese
- [X] English
- [X] PR #1421
Web Page Updating
Add Links
This part is included in the Adding Tutorial part’s PR.
Add Guides
Add guides for using and contributing (#1455)
Bug Fix
Delete all the arthas-boot.md ‘–target-ip 0.0.0.0’ boot parameter (#1359)
Fix classloaderhash Problems (#1417)
Update sc/sm Online Tutorial for classLoaderClass (#1443)
Fix ID doesn’t exist in T4 (#1451)
Fix links, typos, ‘intro.md’, ‘finish.md’ for Online Tutorials (#1453)
Add Press Q or Ctrl+C to abort for enhanced commands (#1454)
Classloader using –classLoaderClass in Advanced tutorial (#1456)
Fix typos and accessibility #847 (#1460)
Add –classLoaderClass Param PRs
Classloader support matching classloader by class name (#1428)
Optimize –classLoaderClass #1428 (#1431)
Add –classLoaderClass for sc/sm (#1433)
Add –classLoaderClass for logger (#1445)
Add –classLoaderClass for dump/getstatic/jad/mc/redifine (#1447)
Other Unmerged PRs
Split arthas-advanced to arthas-command (#2)
Arthas Online course arrangement and compilation supplement (#5)
Arthas Online course arrangement and compilation supplement (#1239)
Arthas Online course arrangement and compilation supplement —— Update web pages (#1241)
Arthas Basic Command Tutorial (#1303)
Project highlights
- Straightforward and convenient menu design. Commands and user cases are classified to facilitate users to retrieve certain problems or specific usage methods.
- Introduce the online tutorial in the official documents, which is convenient for users to click the link directly.
- Online tutorial can use real environment to learn and test new technology in browser, and help developers learn and master best practices.
- Online tutorial can quickly provide a complete set of temporary environment and recycle it after use. Users can operate the environment through the browser terminal interface according to the designed guidance steps, and learn and practice step by step.
- Solve the user’s trouble of too troublesome to set up own environment, not having supported environment to try Arthas and the poor operability of the documents and the difficulty of hands-on practice.
- The git repository of the tutorial is placed in Arthas itself so that Arthas contributors can easily find the source code location of the online tutorial and continue to contribute to it.
- Actively Add –classloaderclas parameter for classloader/sc/sm/logger/dump/getstatic/jad/mc/redifine, through which you can directly specify a classloader which has no duplicate name, and do not need to use c parameter to specify dynamically changing hash value of classloader, which is convenient for online tutorial writing.
Experience
Alibaba Summer of Code 2020 has greatly trained my team cooperation and communication skills, broadened my vision, and I gained a lot.
The two mentors also gave me a lot of guiding opinions during ASoC 2020, and I would like to thank them for their hard guidance
!
Therefore, students, if you have enough enthusiasm for open source, you are recommended to participate in the future Alibaba Summer of Code activities!
KDE Gear, Plasma, systemd Update in Tumbleweed
There was one openSUSE Tumbleweed snapshot this week out of five that brought an enormous amount of package updates for those using the rolling release.
Snapshot 20210904 brought updates for systemd, GTK4, Mesa, KDE’s Plasma and Gear and many other packages.
The most recent snapshot to be released was 20210908; it updated fuse3 3.10.5 and made various improvements to unit tests more robust for the Filesystem in Userspace package. The mpg123 1.29.0 update added an--enable-runtime-tables. An update of yast2 4.4.17 provided some maintenance for the systemd package that arrived earlier in the week. A few other packages like glslang 11.6.0, libstorage-ng 4.4.36 and pinentry 1.2.0 were also updated in the snapshot.
Snapshot 20210907 updated seven packages. The package manager zypper 1.14.49 made a change to avoid calling su as it can be too restrictive for sudo user umask. The package manager library libzypp also had an update to version 17.28.3, which had a policy modification for avoid the breaking of a single rpm transaction. The AV1 decoder package dav1d 0.9.2 had some Streaming SIMD Extensions 3 and SSE4 optimizations for x86_64. Other packages updated in the snapshot were geoclue2 2.5.7, mozilla-nss 3.69.1, supermin 5.2.1 and an update to plymouth.
The 20210906 snapshot had two package updates. The --nodst option for Docker’s local network driver MacVLAN was added in the iproute2 5.14 update. The package also added support for wireless wide area network devices. There were about four months worth of updates in the 20210901.1550 permissions update, which cleaned up some paths and updated the kdesud and ksgrd_network_helper paths.
The GTK4 4.4.0 update in snapshot 20210904 added support for the gnome-shell title bar gesture protocol and made a change to use harfbuzz for color-font information. An update to systemd 249.4 arrived in the snapshot; the new systemd version was extensively tested and and the dependencies needed for the update were added. Mesa 21.2.1 was the first bug-fix release in the mesa 21.2 series, which focused on fixes for fossilize_db. Both Mozilla Firefox 91.0.2 and Mozilla Thunderbird 91.0.3 were updated in the snapshot. The browser no longer clears authentication data when purging trackers; this was done to avoid repeatedly prompting for a password. The email client fixed some user experience setup issues and also pushed a fix that sometimes sent an unnecessary “SMTPUTF8”, which caused some servers to reject the email. KDE’s Plasma 5.22.5 fixed the handling of IPV6 addresses for ksystemstats, and it had multiple fixes for the Plasma Desktop that included a fix for the emoji picker and a fix for some inconsistent behavior involving hovering over the tooltip. KDE Gear 21.08.1 fixed a brightness-effect corruption in Kdenlive that was related to the GNU Compiler Collection. Gear also had a Konsole fix involving the closing of the split view. The Ethernet station-activity-monitor package arpwatch had a major version update to 3.1, which provided a notable upstream change of adding Python 2 compatibility to massagevendor. The Linux Kernel update to version 5.14.0, which updated and enabled configurations for ARMv6 and ARMv7. Other packages to update in the snapshot were Flatpak 1.11.3, hwdata 0.351, libvirt 7.7.0, php7 7.4.23, qemu 6.1.0 and several more.
The snapshot that began the week was 20210902. The Linux Bluetooth protocol bluez 5.61 provided a couple Advanced Audio Distribution Profile fixes and fixed an issue with storing discoverable settings. The disk encryption package cryptsetup updated to version 2.4.0 and provided backend support for OpenSSL3. Support for new distributions was added in the hplip 3.21.6 printing package and transactional-update 3.5.2 fixed overlay syncing errors with SELinux.
GSoC report: syslog-ng MacOS support
For the past couple of months, Yash Mathne has been working on testing syslog-ng on MacOS as a GSoC (Google Summer of Code) student. He worked both on x86 and on the freshly released ARM hardware. And we have some good news here to share: while there is still room for improvement, most of syslog-ng works perfectly well on MacOS.
Read my blog for some historical background and the GSoC report: https://www.syslog-ng.com/community/b/blog/posts/gsoc-report-syslog-ng-macos-support
What is coming in sudo 1.9.8?
Sudo development is at version 1.9.8 beta 3. There are two major new features: sudo can intercept sub-commands and log sub-commands. In this quick teaser I introduce you to log_subcmds. I hope it is interesting enough for you to test it out and provide feedback.
So, what is log_subcmds good for? There are many UNIX tools that can spawn external applications. You only see vi in the logs, but can you be sure without session recording that your admin only edits what he is supposed to? With log_subcmds you can see all the commands started from an application run through sudo. Or you can see all the commands started from a shell, even without session recording.
You can read the rest of my blog at https://blog.sudo.ws/posts/2021/08/what-is-coming-in-sudo-1.9.8/
Kubic with Kubernetes 1.22.1 released
Announcement
The Kubic Project is proud to announce that snapshot 20210901 has been released containing Kubernetes 1.22.1.
Release Notes are avaialble HERE.
Upgrade Steps
All newly deployed Kubic clusters will automatically be Kubernetes 1.22.1 from this point.
For existing clusters, please follow our new documentation our wiki HERE
Thanks and have a lot of fun!
The Kubic Team
