Planet openSUSE Planet openSUSE /images/icon.svg /images/logo.svg 2021-01-20T15:05:12+00:00 https://planet.opensuse.org/en Pluto 3bb8ab65-05eb-463c-ad10-a3830cca5502:66429cd7-a6e1-4d19-81df-4ccfba99f0dd Peter Czanik https://www.syslog-ng.com/community/b/blog 2021-01-19T12:31:00+00:00 2021-01-19T12:31:00+00:00 Consuming logs from a Kafka topic using syslog-ng <p>There is no official Kafka source in syslog-ng, but because this question comes up often enough, I created one. It is just a temporary workaround using the program() source, but it works. It involves Java and installing Kafka manually, but it was fast and reliabe in my tests: ingesting 50,000–100,000 messages a second on my laptop in a resource-constrained virtual machine.</p> <p>Of course, I also tried a more resource-friendly solution, using kafkacat to consume log messages from a Kafka topic. While it worked perfectly on the command line, I could not get it to work with the program() source in syslog-ng.</p> <p>If you read my blog last week about using templates in the topic() parameter of the Kafka destination, the test environment will look familiar. The only notable difference is that the tool used to consume logs from Kafka is now called within syslog-ng from a program() source.</p> <h2 class="western">Before you begin</h2> <p>You do not need the most recent syslog-ng version to use the program() source. Still, I recommend you use recent packages, because they contain many useful bug fixes. You can learn more about where 3<sup>rd</sup> party syslong-ng packages for major Linux distributions are available at <a href="/3rd-party-binaries">https://www.syslog-ng.com/3rd-party-binaries</a></p> <p>Kafka might be available for your Linux distribution of choice, but it was not available in the distributions I checked. For simplicity’s sake, I use the binary distribution from the Kafka website. At the time of writing, the latest available version is kafka_2.13-2.6.0.tgz and it should work equally well on any Linux host with a recent enough (that is, 1.8+) Java. If you use a local Kafka installation, you might need to modify some of the example command lines.</p> <h2 class="western">Downloading and starting Kafka</h2> <p>A proper Kafka installation is outside of the scope of my blog. Here, you will follow relevant parts of the <a href="https://kafka.apache.org/quickstart">Kafka Quickstart documentation</a>. You will download the archive containing Kafka, extract it, and start its components. You will need network access and four terminal windows.</p> <ol> <li> <p>First, download the latest Kafka release and extract it. The exact version might bedifferent:</p> </li> </ol> <pre class="western">wget <a href="https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgz">https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgz</a> tar xvf kafka_2.13-2.6.0.tgz</pre> <p>At the end of this process, you will see a new directory named kafka_2.13-2.6.0.</p> <ol start="2"> <li> <p>From now on, you will need the 3 extra terminal windows, because first you will start two separate daemons in the foreground to see their messages, and two more windows are required to send messages to Kafka and to receive them.</p> </li> <li> <p>First, start zookeeper in one of the terminal windows. Change to the new Kafka directory and start the application:</p> </li> </ol> <pre class="western">cd kafka_2.13-2.6.0/ bin/zookeeper-server-start.sh config/zookeeper.properties</pre> <ol start="4"> <li> <p>Now you can start the Kafka server in a different terminal window:</p> </li> </ol> <pre class="western">cd kafka_2.13-2.6.0/ bin/kafka-server-start.sh config/server.properties</pre> <p>Both applications print lots of data on screen. Normally, the flood of debug information stops after a few seconds and the applications are ready to be used. If there is a problem, you will get back the command line. In this case, you will have to browse through the debug messages and resolve the problem.</p> <p>Now you can do some minimal functional testing, without syslog-ng involved yet. This way you can make sure that access to Kafka is not blocked by a firewall or other software.</p> <ol start="5"> <li> <p>Open yet another terminal window, change to the Kafka directory and start a script to collect messages from a Kafka topic. You can safely ignore the warning message, it appears because the topic does not exist yet.</p> </li> </ol> <pre class="western">cd kafka_2.13-2.6.0/ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest [2020-12-15 14:41:09,172] WARN [Consumer clientId=consumer-console-consumer-31493-1, groupId=console-consumer-31493] Error while fetching mblog_kafka_source_hack_review.docxetadata with correlation id 2 : &#123;mytest=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)</pre> <ol start="6"> <li> <p>Now you can start a fourth terminal window to send some test messages. Just enter something after the “&gt;” character and press Enter. Moments later, you should see what you have just entered in the third terminal window:</p> </li> </ol> <pre class="western">bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mytest &gt;blah &gt;blahblah &gt;blahblahblah &gt;</pre> <ol start="7"> <li> <p>You can exit with ^D.</p> </li> </ol> <h2 class="western">Configuring syslog-ng</h2> <p>Now that you have checked that you can send messages to Kafka and pull those messages with another application, it is time to configure syslog-ng. If syslog-ng on your system is configured to include .conf files from the /etc/syslog-ng/conf.d/ directory, create a new configuration file there. Otherwise, append the configuration below to syslog-ng.conf:</p> <pre class="western">source s_kafka &#123; program("/root/kafka_2.13-2.6.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest"); }; destination d_fromkafka &#123; file("/var/log/fromkafka"); }; log &#123; source(s_kafka); destination(d_fromkafka); };</pre> <p>The above configuration snippet consumes log messages from Kafka and writes them to a file under the /var/log/ directory. Make sure that settings in the Kafka source match your environment. Here the Kafka archive is extracted under the /root/ directory and the topic name is the same as in the initial tests: “mytest”.</p> <h2 class="western">Testing</h2> <p>Once you have reloaded syslog-ng, you are ready for testing.</p> <ol> <li> <p>Staying in the Kafka directory you can start the producer to send messages to Kafka:</p> </li> </ol> <pre class="western">leap152:~/kafka_2.13-2.6.0 # bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mytest &gt;blah &gt;blahblah &gt;</pre> <ol start="2"> <li> <p>You can now check whether messages successfully arrived to syslog-ng by tailing the log file:</p> </li> </ol> <p>leap152:/etc/syslog-ng/conf.d # tail -f /var/log/fromkafka</p> <p>Jan 15 13:03:25 leap152 blah</p> <p>Jan 15 13:03:29 leap152 blahblah</p> <ol start="3"> <li> <p>As usual, you can exit from the producer using ^D.</p> </li> </ol> <h2 class="western">What is next?</h2> <p>This blog is enough to get you started and learn the basic concepts of Kafka. On the other hand, this environment is far from anything production-ready. For that, you will need a proper Kafka installation and most likely the Kafka consumer in the syslog-ng configuration also requires additional settings. This setup was fast and reliable in my test environment, but that is not a guarantee that it also works well in a production environment. Let me know your experiences!</p> <p></p> <p>If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at <a href="https://github.com/syslog-ng/syslog-ng#community">https://github.com/syslog-ng/syslog-ng</a>. On Twitter, I am available as <a href="https://twitter.com/PCzanik">@Pczanik</a>.</p> <div style="clear:both;"></div> /2021/01/19/OAK-available-for-openSUSE/ openSUSE News https://news.opensuse.org 2021-01-19T00:00:00+00:00 2021-01-19T00:00:00+00:00 OAK compatibility with all openSUSE <p>While focused on the <a href="https://en.opensuse.org/openSUSE:INNOVATORS">openSUSE Innovator initiative</a> as an openSUSE member and official <a href="https://software.intel.com/content/www/us/en/develop/tools/oneapi.html">Intel oneAPI</a> innovator, I tested the OAK AI Kit device on openSUSE <a href="https://software.opensuse.org/distributions/leap">Leap 15.1, 15.2</a> and <a href="https://software.opensuse.org/distributions/tumbleweed">Tumbleweed</a>. With all the work, we made available in the SDB an article on how to install this device on the openSUSE platform. More information can be found at <a href="https://en.opensuse.org/SDB:Install_OAK_AI_Kit">https://en.opensuse.org/SDB:Install_OAK_AI_Kit</a>.</p> <p>The <a href="https://software.opensuse.org/package/opencv">OpenCV</a> AI Kit, that is, OAK, is a tiny, low-end hardware computing module based on the integrated <a href="https://www.intel.com/content/www/us/en/products/processors/movidius-vpu/movidius-myriad-x.html">Intel Movidius Myriad-X AI chip</a>. In comparison to other GPU, CPU, FPGA or TPU-based AI acceleration solutions, Movidius is a VPU architecture with 4.0 TOPS computing capacity. And it is 80 times faster for CV and AI tasks than the well-known OpenMV project, which has only 0.05 TOPS based on the ARM Cortex M7 microcontroller.</p> <p>The OAK has the same AI chip as the Intel Neural Compute Stick 2 (NCS2) but has more powerful hardware features. OAK shipped with one 1/2.3″ Sony 12MP IMX378 capable of 4K@30fps H.265 video streaming, video AI pipelined processing, and two optional 1MP monochrome global shutters OV9282 cameras for depth sensing, with all 3 cameras it turns the OAK into an RGB+D camera.</p> <p>For more information, visit <a href="https://opencv.org/introducing-oak-spatial-ai-powered-by-opencv/">https://opencv.org/introducing-oak-spatial-ai-powered-by-opencv/</a>.</p> /2021/01/18/meetup-will-discuss-survey-results-project-improvements/ openSUSE News https://news.opensuse.org 2021-01-18T00:00:00+00:00 2021-01-18T00:00:00+00:00 Meetup Will Discuss Survey Results, Project Improvements <p>The <a href="https://www.opensuse.org/">openSUSE</a> Project welcomes our followers to participate in two planned meetups to discuss results from the <a href="https://en.opensuse.org/End-of-year-surveys/2020/Data">End of the Year Community Survey</a> on Jan. 23 and Jan. 30.</p> <p>Both sessions will start at 13:00 UTC on openSUSE’s Jitsi instance and go for 1:30 hours.</p> <p>Members of the <a href="https://news.opensuse.org/2020/10/12/join-our-team-and-help-us-imporove-the-openSUSE-learning-experience/">“let’s improve the openSUSE learning experience”</a> initiative will share <a href="https://en.opensuse.org/End-of-year-surveys/2020/Data">results</a> and analysis from the survey.</p> <p>The meetup is designed to discuss ways the community can improve upon areas identified in the results as either a weakness or needing improvement.</p> <p>Topics that will be discussed in the Jan. 23 session include:</p> <ul> <li>Address pain points</li> <li>Knowledge transfer</li> <li>Promotion (How are they learning about projects)</li> </ul> <p>Topics to be discussed in the Jan. 30 session include:</p> <ul> <li>Tools driving switchers to openSUSE (Where are users coming from)</li> <li>Discuss flagship project/s</li> <li>Expanding global users</li> <li>Increasing diversity</li> <li>Increase usage with people under 34</li> </ul> <p>The meetup will take place at <a href="https://meet.opensuse.org/EOY2020">https://meet.opensuse.org/EOY2020</a>.</p> <p>More details about the <a href="https://news.opensuse.org/2020/12/19/introducing-the-opensuse-2020-end-of-year-survey/">End of the Year Community Survey</a> results can be found on the openSUSE <a href="https://en.opensuse.org/End-of-year-surveys/2020/Data">Wiki</a>.</p> https://hpjansson.org/blag/?p=615 Hans Petter Jansson https://hpjansson.org/blag/ 2021-01-17T23:50:04+00:00 2021-01-17T23:50:04+00:00 Chafa 1.6.0: Wider <p>Here’s another one from the <a href="https://hpjansson.org/chafa/">terminal graphics extravaganza</a> dept: <a href="https://github.com/hpjansson/chafa/releases/tag/1.6.0">Chafa 1.6.0</a> brings <a href="https://en.wikipedia.org/wiki/Halfwidth_and_fullwidth_forms">fullwidth</a> character support, so in addition to the usual block elements and ASCII art, you now get some mean <a href="https://en.wikipedia.org/wiki/CJK_Unified_Ideographs">CJK</a> art too. Or grab as many fonts as you can and combine <em>all of the Unicode</em> into one big glorious mess. Chafa can efficiently distinguish between thousands of symbols, so it also runs fast enough for animations — up to a point.</p> <p>Since some users want this in environments where it’s not practical to build from source or even to have nice things like GLib, I’ve started adding <a href="https://hpjansson.org/chafa/releases/static/">statically linked builds</a>. These are pretty bare-bones (fewer image loaders, no <a href="https://hpjansson.org/chafa/man/">man page</a>), so <a href="https://repology.org/project/chafa/versions">look to your steadfast distribution</a> first.</p> <p>Speaking of distributions, a big thank you to the packagers. Special thanks go to Florian Viehweger for getting in touch re. adding it to OpenBSD ports, and <a href="https://people.debian.org/~lumin/">Mo Zhou</a> (Debian), <a href="https://iodoru.org/">Michael Vetter</a> (openSUSE), <a href="https://github.com/herbygillot">Herby Gillot</a> (MacPorts), <a href="https://github.com/chenrui333">@chenrui</a> and <a href="https://github.com/carlocab">Carlo Cabrera</a> (Homebrew) for getting 1.6 out there before I could even finish this post.</p> <h2>So what’s it look like?</h2> <p>Obviously if you just want as faithful a reproduction as possible, stick with the default block elements or <a href="https://hpjansson.org/blag/2020/04/01/chafa-1-4-0-now-with-sixels/">sixels</a>. That said, fullwidth characters open up some new artistic possibilities.</p> <figure class="wp-block-image size-large"><a href="https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide.png"><img loading="lazy" width="1024" height="1280" src="//hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide-1024x1280.png" alt="Chafa rendering of Dog's Head" class="wp-image-744" srcset="https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide-1024x1280.png 1024w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide-240x300.png 240w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide-768x960.png 768w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-dogs-head-wide.png 1114w" sizes="(max-width: 1024px) 100vw, 1024px"></a></figure> <p>Above, a rendering of <a href="https://archive.org/details/dogheaddegraag" rel="nofollow">Dog’s Head</a> (1920) by Julie de Graag, <a href="https://www.rawpixel.com/image/466876/free-illustration-image-dog-julie-graag-animal">digitally enhanced by Rawpixel</a>. It was generated with the following command line:</p> <pre class="wp-block-code"><code lang="bash" class="language-bash">chafa --glyph-file /usr/share/fonts/truetype/SourceHanSansCN-Normal.otf \ --glyph-file /usr/share/fonts/truetype/SourceHanSansJP-Normal.otf \ --glyph-file /usr/share/fonts/truetype/DroidSansThai.ttf \ --glyph-file /usr/share/fonts/truetype/SourceCodePro-Regular.ttf \ --symbols 0..fffff-block-border-stipple-dot-geometric \ -c none -w 9 dog.png</code></pre> <p>Although I’d like to include a moderately large built-in selection of fullwidth symbols in a future release, for now you must load fonts with <code>--glyph-file</code> in order to achieve this effect. You also need to enable the Unicode ranges you want and curtail the use of block and border elements with <code>--symbols</code>. The latter is necessary because block elements produce more accurate results and will otherwise pretty much always come out on top during error minimization.</p> <figure class="wp-block-image size-large"><a href="https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color.png"><img loading="lazy" width="1658" height="1280" src="//hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color-1658x1280.png" alt="Chafa rendering of Shinjuku Skyscrapers" class="wp-image-748" srcset="https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color-1658x1280.png 1658w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color-300x232.png 300w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color-768x593.png 768w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color-1536x1186.png 1536w, https://hpjansson.org/blag/wp-content/uploads/2020/12/chafa-shinjuku-wide-color.png 1804w" sizes="(max-width: 1658px) 100vw, 1658px"></a></figure> <p>This is a rendering of <a href="https://www.flickr.com/photos/wilhelmja/328176297/" rel="nofollow">Shinjuku Skyscrapers</a>, <a href="https://creativecommons.org/licenses/by-sa/2.0/" rel="nofollow">CC-BY-SA</a> Wilhelm Joys Andersen. I used the same set of options to produce it, but left out <code>-c none</code>, resulting in 24-bit color — the default under VTE.</p> <p>A side effect of allowing lots of color variation is fewer wide characters. This makes sense considering that they force a pair of cells to have the same color, which is often less accurate than two narrow characters with different colors.</p> <h2>彡 (._.) ( l: ) (.-.) ( :l )</h2> <p>Like many subjects that look simple at first, terminal graphics makes for a surprisingly deep rabbit hole to be tumbling into. Chafa now spans the gamut from the most basic monochrome ASCII art to fullwidth Unicode, 24-bit color and sixels, and there’s still a lot that can be done to improve it. I will be doing so… slowly.</p> <p>If you want to help, feel free to <a href="https://github.com/hpjansson/chafa">send pull requests</a> or <a href="https://github.com/hpjansson/chafa/issues">file any issues you find</a>. I think it’s also at the point where you can achieve various surprising effects, so if you manage to get something particularly cool/sick/downright disgusting out of it, just lob it in my general direction and maybe I’ll include it in a future gallery.</p> http://cubiclenate.com/?p=8178 Nathan Wolf https://cubiclenate.com 2021-01-17T00:57:49+00:00 2021-01-17T00:57:49+00:00 openSUSE Smiles <p>As a rule, the openSUSE logo makes me happy, just seeing it. I did, in fact, add stickers to my newly acquired <a href="https://cubiclenate.com/2020/12/20/hp-elitebook-840-g7-running-opensuse-tumbleweed/">EliteBook</a> to add a bit of <a href="https://cubiclenate.com/2021/01/02/opensuse-stickers-to-enhance-your-tech/">personalized happiness</a> to it. On the openSUSE mailing list, discussing Blender, I clicked on this link to YouTube for a short video that I thought was not only really cool but cute and funny. </p> <figure class="wp-block-embed is-type-rich is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper"> <div class="jetpack-video-wrapper"><iframe class="youtube-player" width="1100" height="619" src="//www.youtube.com/embed/2r_XtGNwVbE?version=3&amp;rel=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;fs=1&amp;hl=en&amp;autohide=2&amp;wmode=transparent" allowfullscreen="true" style="border:0;" sandbox="allow-scripts allow-same-origin allow-popups allow-presentation"></iframe></div> </div></figure> <p>There is no end to my amazement of the openSUSE community. They do such a fantastic job of making a wonderful distribution with all the tools that keep me productive. I am very thankful for the reliability I enjoy using openSUSE. The community members also do a great job of helping me through a jam, should I drive myself into one.</p> <p>Now I look forward to seeing the release of Blender with this customized piece of happiness!</p> <h2>References</h2> <p><a rel="noreferrer noopener" href="https://www.youtube.com/watch?v=2r_XtGNwVbE" target="_blank">Link to YouTube animation of Geeko animation for Blender</a><br><a href="https://opensuse.org" target="_blank" rel="noreferrer noopener">openSUSE.org</a><br><a href="https://cubiclenate.com/2021/01/02/opensuse-stickers-to-enhance-your-tech/">HP EliteBook Stickers</a><br><a href="https://cubiclenate.com/2020/12/20/hp-elitebook-840-g7-running-opensuse-tumbleweed/">HP EliteBook 840 G7 running openSUSE Tumbleweed</a></p> http://dominique.leuenberger.net/blog/?p=1048 Dominique Leuenberger http://dominique.leuenberger.net/blog/ 2021-01-15T16:02:46+00:00 2021-01-15T16:02:46+00:00 openSUSE Tumbleweed – Review of the week 2021/02 <p>Dear Tumbleweed users and hackers,</p> <p>Somewhere, I read, 2021 will be the year of the Linux desktop. Do you agree? Let’s make it the year of Tumbleweed on the desktop. In any case, Tumbleweed has been steadily rolling with 5 snapshots published during this week (0107, 0108, 0110, 0111, and 0113).</p> <p>The major changes included:</p> <ul> <li>Plasma 5.20.5</li> <li>KDE Frameworks 5.78.0</li> <li>KDE Applications 20.12.1</li> <li>IceWM 2.0.0</li> <li>Xfce 4.16.0</li> <li>Mozilla Firefox 84.0.2</li> <li>Linux kernel 5.10.5</li> <li>RPM 4.16.0</li> <li>brp-check-suse: a bug fix in how it detected dangling symlinks; some packages might no fail to build, but they had dangling symlinks before which we just did not detect.</li> <li>permissions package: preparation for a full /usr merge</li> </ul> <p>So, quite a list that accumulated, in just a week. The stagings are getting lighter, but are far from done. Currently, these are the major changes being prepared:</p> <ul> <li>Multiple versions of python 3 parallel installable. Adding to python 3.8, version 3.6 will be reintroduced. Python modules will be built for both versions – Should become part of Snapshot 0115</li> <li>Tcl/tk 8.6.11</li> <li>icu 68.1: breaks a few things like PostgreSQL. Staging:I</li> <li>Bash 5.1</li> <li>Rust 1.49: breaks librsvg</li> <li>Autoconf 2.70: breaks quite a few packages. The list of failures has been noted on the current SR; no active staging left for it (no progress in the last days/weeks on it)</li> <li>openssl 3: no progress, Staging:O still showing a lot of errors.</li> </ul> /2021/01/15/xfce-kde-packages-flood-this-weeks-tw-snapshots/ openSUSE News https://news.opensuse.org 2021-01-15T00:00:00+00:00 2021-01-15T00:00:00+00:00 Xfce, KDE Packages Flood This Week’s Tumbleweed Snapshots <p>A large quantity of packages from both <a href="https://www.xfce.org/">Xfce</a> and <a href="https://kde.org">KDE</a> projects flowed into <a href="https://software.opensuse.org/distributions/tumbleweed">openSUSE Tumbleweed</a> snapshots this week.</p> <p>Hundreds of packages updated in the rolling release and KDE’s <a href="https://develop.kde.org/products/frameworks//">Frameworks</a>, <a href="https://apps.kde.org/">Applications</a> and <a href="https://kde.org/plasma-desktop/">Plasma</a> packages were the most prevalent of software package updated throughout the week.</p> <p><a href="https://kde.org/announcements/kde-frameworks-5.78.0/">KDE Frameworks 5.78.0</a> arrived in the latest <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/47YGKIJW565AQDCDRPFYY63FNIC5BDFB/">20210113</a> snapshot. Frameworks added a new compass action icon in the <a href="https://api.kde.org/frameworks/breeze-icons/html/index.html">Breeze Icons </a> and <a href="https://api.kde.org/frameworks/kconfig/html/">KConfig</a> fixed windows being inappropriately maximized on launch. User Interface framework <a href="https://github.com/KDE/kirigami">Kirigami</a> fixed some visual bugs for avatar controls and KDE’s data accessing package <a href="https://api.kde.org/frameworks/kio/html/index.html">KIO</a> fixed a shortcut reset button and the middle-click handling with the url navigation menu. Frameworks packages weren’t the only packages to update in the snapshot; the update to the 1.12.3 version of ibus-table provides a new setup tool that allows keybindings to be configured with a <a href="https://en.wikipedia.org/wiki/Graphical_user_interface">GUI</a>. Fingerprint reader package <a href="https://fprint.freedesktop.org/">libfprint</a> fixed issues that caused problem on non-x86 machines in its 1.90.6 version update. The last package to be included in the update was the parser library <a href="https://github.com/michaelrsweet/mxml">mxml</a> 3.2, which fixed handling of elements that start with a <a href="https://en.wikipedia.org/wiki/Unicode">Unicode</a> character and fixed the handling of unquoted attribute values that start with Unicode.</p> <p>The <a href="https://www.xfce.org/">Xfce</a> 4.16 desktop and many of its complementary packages arrived in snapshot <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/ES3UZI3H2YJOLICHNNH2IT2FDMDG7T2D/">20210111</a> and people are excited about this release. There are new icons and color palettes for unifying a style and look that will make people ask what desktop environment is that. The panel received some updates with a new status tray plugin and the darkmode looks sleek. The <a href="https://en.wikipedia.org/wiki/Thunar">thunar file manager</a> can easily pause during a file transfer and the release looks topnotch. <a href="https://imagemagick.org/index.php">ImageMagick</a> jumped some minor versions to 7.0.10.55 and reverted changes to the default max width/height of an image. <a href="https://www.mozilla.org/en-US/firefox/new/">Mozilla Firefox</a> fixed a <a href="https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures">Common Vulnerabilities and Exposure</a> involving a <a href="https://en.wikipedia.org/wiki/SCTP_packet_structure#COOKIE_ECHO_chunk">COOKIE-ECHO</a>. The compiler plugin for <a href="https://clang.llvm.org/">clang</a> to understand <a href="https://www.qt.io/">Qt</a> semantics, <a href="https://github.com/KDE/clazy">clazy</a> 1.9 is now back to being 4x faster. <a href="https://flatpak.org/">Flatpak</a> 1.8.4 fixed support for <a href="https://en.wikipedia.org/wiki/Ppc64">PowerPC</a>. Several other packages were updated in the snapshot to include <a href="https://openvpn.net/">openvpn</a> 2.4.10, <a href="https://github.com/bluesabre/mugshot">mugshot</a> 0.4.3, <a href="https://github.com/qpdf/qpdf/releases">qpdf</a> 10.1.0 and Xfce’s <a href="https://docs.xfce.org/apps/mousepad/start">mousepad</a> 0.5.1.</p> <p><a href="https://kde.org">KDE</a>’s <a href="https://kde.org/announcements/fulllog_releases-20.12.1/">Applications 20.12.1</a> arrived in snapshots <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/4HRXE6XCDWTYRTRWMMXBEKEJXVK23EOI/">20210110</a> and <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/37X5ULZ373446JQTVSDTPBS5PBITCYB3/">20210108</a>. The update fixed a crash when a device with a capacitybar is dragged in the file manager <a href="https://apps.kde.org/en/dolphin">Dolphin</a>. KDE’s CD and DVD application <a href="https://apps.kde.org/en/k3b">K3b</a> fixed an <a href="https://en.wikipedia.org/wiki/Infinite_loop">infinite loop</a> when clearing a DVD Video project. Video editor <a href="https://kdenlive.org/en/">Kdenlive</a> had multiple fixes to include some regressions in the keyframe move option while editing.</p> <p>Snapshot snapshots <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/4HRXE6XCDWTYRTRWMMXBEKEJXVK23EOI/">20210110</a> had an update of <a href="https://rpm.org/wiki/Releases/4.16.0">rpm 4.16.0</a>, which provided multiple documentation updates and added new version parsing and comparison <a href="https://en.wikipedia.org/wiki/API">API</a> in <a href="https://en.wikipedia.org/wiki/C_(programming_language)">C</a> and <a href="https://www.python.org/">Python</a>. With the update of <a href="https://www.smartmontools.org/wiki/Download">smartmontools</a> 7.2, <code class="language-plaintext highlighter-rouge">smartd</code> now resolves symlinks before device names are checked for duplicates. Chinese <a href="https://en.wikipedia.org/wiki/Pinyin">pinyin</a> package <a href="https://github.com/libpinyin/libpinyin">libpinyin</a> updated to version 2.6.0 and the low-level crypto library <a href="http://www.linuxfromscratch.org/blfs/view/svn/postlfs/nettle.html">libnettle</a> 3.7 added the password-hashing function <a href="https://en.wikipedia.org/wiki/Bcrypt">bcrypt</a>.</p> <p>Besides the <a href="https://kde.org/announcements/fulllog_releases-20.12.1/">20.12.1 Applications</a> update in snapshot <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/37X5ULZ373446JQTVSDTPBS5PBITCYB3/">20210108</a>, there were several more packages to land in this snapshot. The new major version of <a href="https://ice-wm.org/">icewm</a> 2.0.0 fixed horizontal scrolling in the icehelp. The windows manager also removed an unwanted separator in the taskbar and added support for the <a href="https://docs.enlightenment.org/api/imlib2/html/">Imlib2</a> image rendering engine as an alternative for the <a href="https://gitlab.gnome.org/Archive/gdk-pixbuf-xlib">gdk-pixbuf-xlib</a> rendering engine. Email client <a href="mutt.org">mutt</a> 2.0.4 dropped a patch. Web server content retriever <a href="https://www.gnu.org/software/wget/manual/wget.html">wget</a> 1.21 fixed buffer overflows in the progress bar code in some locales. Scripting language <a href="https://www.php.net/">php7</a> updated to version 7.4.14 and fixed one CVE; it also had a fix for <a href="https://www.php.net/manual/en/features.dtrace.dtrace.php">dtrace scripts</a> that caused php to crash. About 20 more packages were updated in the snapshot.</p> <p>Snapshot <a href="https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/2RPVS7JDL5WFBHCNCRSGFU6ULMBHKZLG/">20210107</a> began the week and, with the exception of three <a href="https://www.xfce.org/">Xfce</a> plugin packages, the snapshot was all <a href="https://kde.org/announcements/plasma/5/20.5/">Plasma 5.20.5</a>. Just two days after being released upstream, <a href="https://kde.org/plasma-desktop/">plasma-desktop</a> fixed the order of the actions of the emojier and had adjustment fixes for the panel height on the top and left. The Bluetooth devices integrator for Plasma <a href="https://userbase.kde.org/Bluedevil">bluedevil5</a> now shows only paired devices in KConfig Modules (KCM) and applet. Plasma’s network manager paused the scanning of wifi when appropriate to avoid password entry jumping to different used networks.</p> 3bb8ab65-05eb-463c-ad10-a3830cca5502:18bbb16d-c9e5-4d04-8aef-a90c32de3949 Peter Czanik https://www.syslog-ng.com/community/b/blog 2021-01-12T11:46:00+00:00 2021-01-12T11:46:00+00:00 Kafka destination improved with template support in syslog-ng <p>The C implementation of the Kafka destination in syslog-ng has been improved in version 3.30. Support for templates in topic names was added as a result of a Google Summer of Code (GSoC) project. The advantage of the new template support feature is that you no longer have to use a static topic name. For example, you can include the name of your host or the application sending the log in the topic name.</p> <p>From this blog you can learn about a minimal Kafka setup, configuring syslog-ng and testing syslog-ng with Kafka.</p> <h2 class="western">Before you begin</h2> <p>Template support in the C implementation of the Kafka destination first appeared in syslog-ng version 3.30. This version is not yet available in Linux most distributions, and even where it is available, Kafka support is not enabled. You can either compile syslog-ng 3.30 or later yourself, or use 3rd party package repositories to install it. You can learn more about them at <a href="/3rd-party-binaries">https://www.syslog-ng.com/3rd-party-binaries</a> In most cases, Kafka support is not part of the base syslog-ng package, but available as a sub package. For example, in (open)SUSE and Fedora/RHEL packages it is available in the syslog-ng-kafka package.</p> <p>Kafka might be available for your Linux distribution of choice, but for simplicity’s sake, I use the binary distribution from the Kafka website. At the time of writing, the latest available version is kafka_2.13-2.6.0.tgz and it should work equally well on any Linux host with a recent enough (that is, 1.8+) Java. If you use a local Kafka installation, you might need to modify some of the example command lines.</p> <h2 class="western">Downloading and starting Kafka</h2> <p>A proper Kafka installation is outside of the scope of my blog. Here we follow relevant parts of the <a href="https://kafka.apache.org/quickstart">Kafka Quickstart documentation</a>. We download the archive containing Kafka, extract it, and start its components. You will need network access and four terminal windows.</p> <p>First, download the latest Kafka release and extract it. The exact version might be already different:</p> <pre class="western">wget <a href="https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgz">https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgz</a> tar xvf kafka_2.13-2.6.0.tgz</pre> <p>At the end, you will see a new directory: kafka_2.13-2.6.0</p> <p>From now on, you will need the 3 extra terminal windows, as first we start two separate daemons in the foreground to see their messages, and two more windows are required to send messages to Kafka and to receive them.</p> <p>First, start zookeeper in one of the terminal windows. Change to the freshly created directory and start the application:</p> <pre class="western">cd kafka_2.13-2.6.0/ bin/zookeeper-server-start.sh config/zookeeper.properties</pre> <p>Now you can start the Kafka server in a different terminal window:</p> <pre class="western">cd kafka_2.13-2.6.0/ bin/kafka-server-start.sh config/server.properties</pre> <p>Both applications print lots of data on screen. Normally, the flood of debug information stops after a few seconds and the applications are ready to be used. If there is a problem, you will get back the command line. In this case, you have to browse through the debug messages and resolve the problem.</p> <p>Now you can do some minimal functional testing, without syslog-ng involved yet. This way you can make sure that access to Kafka is not blocked by a firewall or other software.</p> <p>Open yet another terminal window, change to the Kafka directory and start a script to collect messages from a Kafka topic. You can safely ignore the warning message, as it appears because the topic does not exist yet.</p> <pre class="western">cd kafka_2.13-2.6.0/ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest [2020-12-15 14:41:09,172] WARN [Consumer clientId=consumer-console-consumer-31493-1, groupId=console-consumer-31493] Error while fetching metadata with correlation id 2 : &#123;mytest=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)</pre> <p>Now you can start a fourth terminal window to send some test messages. Just enter something after the “&gt;” sign and hit Enter. Moments later, you should see what you just entered in the third terminal window:</p> <pre class="western">bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mytest &gt;bla &gt;blabla &gt;blablabla &gt;</pre> <p>You can exit with ^D.</p> <h2 class="western">Configuring syslog-ng</h2> <p>Now that we checked that we can send messages to Kafka and pull those messages with another application, it is time to configure syslog-ng. As a first step, we just check sending logs to Kafka and check the results.</p> <p>If syslog-ng on your system is configured to include .conf files from the /etc/syslog-ng/conf.d/ directory, create a new configuration file there. Otherwise, append the configuration below to syslog-ng.conf.</p> <pre class="western">destination d_kafka &#123; kafka-c(config(metadata.broker.list("localhost:9092") queue.buffering.max.ms("1000")) topic("mytest") message("$(format-json --scope rfc5424 --scope nv-pairs)")); }; log &#123; source(src); destination(d_kafka); };</pre> <p>Note that the source name for local logs might be different in your syslog-ng.conf, so check before reloading the configuration. The name “src” is used on openSUSE/SLES. As a safety measure, check your configuration with:</p> <pre class="western">syslog-ng -s</pre> <p>While it cannot check if you spelled the source name correctly, a quick syntax check will ensure that all necessary syslog-ng modules are installed. If you see a message about JSON or Kafka, install the missing modules.</p> <p><a name="_GoBack"></a> Once you reloaded syslog-ng, you should see logs arriving on your third terminal window in JSON format, similar to these:</p> <pre class="western">blablabla &#123;"SOURCE":"src","PROGRAM":"syslog-ng","PRIORITY":"notice","PID":"5841","MESSAGE":"syslog-ng starting up; version='3.30.1.25.g793d7e4'","HOST_FROM":"leap152","HOST":"leap152","FACILITY":"syslog","DATE":"Dec 15 15:04:58"} &#123;"SOURCE":"src","PROGRAM":"systemd","PRIORITY":"info","PID":"1","MESSAGE":"Stopping System Logging Service...","HOST_FROM":"leap152","HOST":"leap152","FACILITY":"daemon","DATE":"Dec 15 15:04:57"}</pre> <h2 class="western">Using a template in the topic name</h2> <p>To use a template in the topic name, the syslog-ng configuration needs two modifications. First of all, you need to modify the topic(). But you also need to provide an additional parameter: fallback-topic(). Note that topic names can only contain numbers and letters from the English alphabet. Special characters or letters with accent marks are rejected. This is why you need a fallback-topic: if a topic name cannot be used, the related message is saved to the topic named in the fallback-topic(). You can find the modified configuration below:</p> <pre class="western">destination d_kafka &#123; kafka-c(config(metadata.broker.list("localhost:9092") queue.buffering.max.ms("1000")) topic("mytest_$PROGRAM") fallback-topic("mytest") message("$(format-json --scope rfc5424 --scope nv-pairs)")); }; log &#123; source(src); destination(d_kafka); };</pre> <p>Using this configuration, the name of the application sending the log is also included in the topic name. Once you reload syslog-ng, you will receive a lot less logs on the “mytest” topic. But, for example, postfix logs will still arrive there, as they include a slash in the application name. Alternatively, you can send a log with accent marks yourself. Being Hungarian is an advantage here, but German also has its own share of accent marked characters. For example, you can use “logger” to send logs and the “-t” option can be used to set the application name:</p> <pre class="western">logger -t öt_szép_szűzleány_őrült_írót_nyúz bla</pre> <p>You will see the related message in the “mytest” topic:</p> <pre class="western">&#123;"SOURCE":"src","PROGRAM":"öt_szép_szűzleány_őrült_írót_nyúz","PRIORITY":"notice","PID":"6177","MESSAGE":"bla","HOST_FROM":"leap152","HOST":"leap152","FACILITY":"user","DATE":"Dec 15 16:21:01"}</pre> <p>By now, you should have logs from a couple of applications. Stop the application pulling logs from Kafka on the third terminal window, and list the available topics. You should see something similar:</p> <pre class="western">bin/kafka-topics.sh --bootstrap-server localhost:9092 --list __consumer_offsets mytest mytest_syslog-ng mytest_systemd</pre> <p>When you start the collector script again with mytest_systemd as a topic, you will most likely not see any input for several minutes. The reason is that by default, the script is only collecting any new messages. Check the built-in help how you can check earlier messages.</p> <h2 class="western">What is next?</h2> <p>This blog is enough to get you started and learn the basic concepts of Kafka and the syslog-ng Kafka destination. On the other hand, it is far from anything production-ready. For that, you need a proper Kafka installation and most likely the syslog-ng configuration also needs additional settings.</p> <p></p> <p>If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at <a href="https://github.com/syslog-ng/syslog-ng#community">https://github.com/syslog-ng/syslog-ng</a>. On Twitter, I am available as <a href="https://twitter.com/PCzanik">@Pczanik</a>.</p> <div style="clear:both;"></div> 5ffa8798de4b961b33f24a96 Ish Sookun https://sysadmin-journal.com/ 2021-01-10T06:26:57+00:00 2021-01-10T06:26:57+00:00 SOGo calendar synchronization breaks due to emoji in the event title <img src="//sysadmin-journal.com/content/images/2021/01/bugfix-freepik.com.jpg" alt="SOGo calendar synchronization breaks due to emoji in the event title"><p>An emoji can break a calendar. 😳</p><p>I am using the <a href="https://sogo.nu/about.html">SOGo Groupware</a>. I noticed that certain emojis in the event title would prevent calendar apps from synchronizing using the CalDAV protocol. I checked the logs but could not find much. I had my doubts about what could be causing it. Then, this <a href="https://sogo.nu/bugs/view.php?id=3616">bug report</a> confirmed that I should investigate on the UTF-8 encoding support.</p><p>I checked the database character set.</p><pre><code>MariaDB [sogo]&gt; select @@character_set_database; +--------------------------+ | @@character_set_database | +--------------------------+ | utf8 | +--------------------------+ 1 row in set (0.001 sec)</code></pre><p>The database name is <code>sogo</code> and we are using MariaDB.</p><p>I found the character set to be <code>utf8</code>, to my surprise. I had to dig a little further to understand what was wrong with it. </p><p>It turned out that the MariaDB <code>utf8</code> character set supports a maximum of three bytes per character. Therefore, emojis being four bytes long weren't being inserted into the database. Consequently, that breaks the calendar synchronization. The solution was to use the <code>utf8mb4</code> character set which supports four bytes per multi-byte character.</p><p>I altered the database character set and collation.</p><pre><code>MariaDB [sogo]&gt; ALTER DATABASE sogo CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;</code></pre><p>I also applied it to every table in the database, e.g:</p><pre><code>MariaDB [sogo]&gt; ALTER TABLE sogo_store CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;</code></pre><p>Afterwards, I could create events using an emoji in the title. The event would synchronize across my calendar apps but the emoji would not show. It would appear as four questions marks (????) instead.</p><figure class="kg-card kg-image-card kg-width-wide kg-card-hascaption"><img src="//sysadmin-journal.com/content/images/2021/01/sogo-calendar-emoji-issue.png" class="kg-image" alt="SOGo calendar synchronization breaks due to emoji in the event title"><figcaption>SOGo calendar emoji issue</figcaption></figure><p>A little bit of further digging and I found that SOGo needs to be made aware of the full unicode support. It should be specified in the <code>/etc/sogo/sogo.conf</code> configuration file.</p><pre><code>MySQL4Encoding = "utf8mb4";</code></pre><p>Restart the SOGo service. Emojis should be then accepted in the event titles.</p><figure class="kg-card kg-image-card kg-width-full"><img src="//sysadmin-journal.com/content/images/2021/01/sogo-calendar-emoji-issue-fixed.png" class="kg-image" alt="SOGo calendar synchronization breaks due to emoji in the event title"></figure><p>I can now put my recurrent coffee breaks in the calendar. ☕</p><hr><p>Credits:<br><a href="https://www.freepik.com/vectors/web">Web vector created by stories - www.freepik.com</a></p> http://dominique.leuenberger.net/blog/?p=1046 Dominique Leuenberger http://dominique.leuenberger.net/blog/ 2021-01-08T14:35:13+00:00 2021-01-08T14:35:13+00:00 openSUSE Tumbleweed – Review of the week 2021/01 <p>Dear Tumbleweed users and hackers,</p> <p>A new year is already upon us, the first week of it is already. We humans might have to get used to writing ‘2021’ instead of ‘2020’, for Tumbleweed, this seems not to matter at all. The week has kicked off strong with 6 snapshots (0101, 0102, 0103, 0104, 0105, and 0106). The number of incoming submissions is also increasing again, showing that contributors are returning from their well-deserved holiday.</p> <p>The most interesting changes in snapshots published this week were:</p> <ul> <li>Linux kernel 5.10.4; this kernel does cause some issues on Intel hardware, make sure to see <a href="https://bugzilla.opensuse.org/show_bug.cgi?id=1180543">https://bugzilla.opensuse.org/show_bug.cgi?id=1180543</a> and the workaround documented in comment#3</li> <li>Mozilla Firefox 84.0.1</li> <li>Mesa 20.3.2</li> </ul> <p>The staging projects mostly contain the same major changes in planning, but work to get them ready is in progress:</p> <ul> <li>Plasma 5.20.5</li> <li>KDE Applications 20.12.1</li> <li>Linux kernel 5.10.5</li> <li>Xfce 4.16.0</li> <li>icu 68.1: breaks a few things like postgresql. Staging:I</li> <li>Multiple versions of python 3 parallel installable. Adding to python 3.8, version 3.6 will be reintroduced. Python modules will be built for both versions.</li> <li>RPM 4.16: all build issues in Staging:A have been fixed, the earlier reported crash on upgrades is also fixed. Final QA run in progress</li> <li>brp-check-suse: a bug fix in how it detected dangling symlinks (it detected them, but did not fail as it was supposed to)</li> <li>permissions package: prepares for easier listing, while supporting a full /usr merge</li> <li>Autoconf 2.70: breaks quite a few packages, Staging:C. NOTE: in some cases, it now implicitly starts gtkdocize; but at least for distro bootstrap packages (ring0) we cannot BuildRequire gtk-doc. So we must trick autoreconf with GTKDOCIZE=true</li> <li>Rpmlint 2.0: experiments ongoing in Staging:M</li> <li>openssl 3: not much progress, Staging:O still showing a lot of errors.</li> </ul> <p></p>