Category: Linux

Putting an image on a Raspberry Pi

  1. Download a .raw.xz file
    1. In this case, it was https://download.opensuse.org/distribution/leap/15.4/appliances/openSUSE-Leap-15.4-ARM-KDE-raspberrypi.aarch64.raw.xz
    2. Yes, I also downloaded the .sha256 file and ran sha256sum against the downloaded image to make sure the image file was not damaged during transfer.
  2. Open a terminal session and become root
  3. Determine which device the SD card is
    1. In this case, it was /dev/sdc
  4. Copy the image file to the SD card
 xzcat /home/david/Downloads/openSUSE-Leap-15.4-ARM-KDE-raspberrypi.aarch64.raw.xz | dd bs=4M of=/dev/sdc iflag=fullblock oflag=direct status=progress; sync

LibreOffice is working better now (somewhat), and I don’t know why

Okay, so I had a theme that wasn’t installed correctly, which made automatic updates for all of OpenSUSE complain they wouldn’t work. I did a re-install of everything KDE, and the missing theme dependency no longer prevents automatic updating from working seamlessly. Cool.

But, I saw the full re-install of KDE “upgraded” LibreOffice. Sure enough, I’m back to running version 7.1

Well, at least I have version 6.4 on disk, and could pretty easily downgrade if needed. Might as well try it out and see.

At first blush, I didn’t appear to have the problem. But, I’d seen that before. I moved the window to the secondary monitor, and the problem returned. Not cool.

But this time, if I move the LibreOffice window back to my primary monitor, the problem vanishes; the window returns to normal. That used to never happen: once the window layout was corrupted, no matter where I placed it, it remained corrupted.

So I can use the current version LibreOffice – I just need to use it on my secondary monitor. I’m okay with that. If that’s the worst thing I have to put up with this month, I’m a lucky guy.

Actually, I just started doing some SharePoint work. Already, I have way worse going on, and it has nothing to do with LibreOffice or KDE or OpenSUSE. 😉

LibreOffice broken – it’s version 7 that is broken, and the problem is multiple monitors

What confused me before is that sometimes it appeared to work, but then it would break again. I finally figured out that it was sliding the window from one monitor to the other that invoked the broken behavior.

I went back to the earliest version of LibreOffice that is “official”, and it was still broken. I went and got the latest version of LibreOffice 6, and it is no longer broken.

LibreOffice has been painfully broken on OpenSUSE KDE for a couple months now

And the solution was to install libreoffice-gtk3

I’m rather surprised that it remained so painful for so long. If I were a new user, I’d still be broken.

I don’t know what it takes to figure out that this package should be in the “required” list. Perhaps I installed something else that added GTK3, and now that it’s there, the LibreOffice UI became significantly damaged? Perhaps LibreOffice sees some GTK3 component and then runs as if libreoffice-gtk3 is present?

The result is that every LibreOffice document would open with a split in the middle of the document, and, the main menu was gone.

I had found a document that said that perhaps KDE dark themes were to blame, or were out of date. Man I fouled some things up chasing that wild goose. I also lost some documents I had built, as tried to delete bad configuration files that seemed they might be the problem.

Anyway, if your LibreOffice looks like this:

And you want it to look like this (with File menu and everything):

install the libreoffice-gtk3 module.

Raspberry Pi – sed for customization

I like automation, and have a project to migrate a friend’s web site to WordPress. There are a lot of MP3 files to move, so I’d like to script it.

I prefer to not test in production (in other words I’m not insane when it comes to computer operations) but that means having a consistent development environment. The Raspberry Pi is a very capable piece of equipment for a very reasonable price. So I think I want to automate building a development environment there.

I’m going to go with the Raspberry Pi OS with desktop

The short form of installing this is:

  1. Download the correct image
    1. I went with Raspbian instead of OpenSUSE because I want sound to work. Apparently the sound driver for Raspberry Pi has not been merged into Linux officially. As soon as it is, then OpenSUSE will have the sound driver baked in. But until then, I want to go with a Linux distribution that adds in the drivers for the Raspberry Pi sound hardware.
  2. Copy the image to the Micro SD card
    1. Open a terminal prompt
    2. unzip -p 2021-05-07-raspios-buster-armhf.zip | sudo dd bs=4M of=/dev/sdc iflag=fullblock oflag=direct status=progress; sync
      1. This assumes that current working directory is the directory where the 2021-05-07-raspios-buster-armhf.zip file is.
      2. This comes from https://scribles.net/writing-raspbian-os-image-to-sd-card-on-linux/ – Thank you Max
      3. Note that the “out file” part of the DD command has a device specifier, and if you put in the wrong one, you can completely destroy your machine. Before you plug the Micro SD card in, do df -h and note the list of devices. None of the listed devices must be in your of=/dev/sdx option of the dd command.
      4. For example: I do df -h and I see a bunch of /dev/sda and a few /dev/sdb listed. This tells me that I must not use either of those for my outfile device. In my case, it happens to be /dev/sdc for my outfile device.
      5. My machine does not have a built in Micro SD slot. I bought an external USB dongle that lets me plug in memory cards. When I plug the dongle in, then /dev/sdc becomes an available device. This is correct – I don’t want dd overwriting any of my permanent devices.

After the image is copied to the Micro SD card, I can (software) eject the removable device, plug it into the Raspberry Pi and power on the Pi. After first boot, there are security updates to install; but after that I’ve got a virgin machine with bog standard software on it. Development is ready to begin!

First, create a couple of aliases

echo "..='cd ..'" > ~/.bash_aliases

echo "ll='ls -l'" >> ~/.bash_aliases

Finally, sed to edit /etc/inputrc

sudo sed --in-place=.original -e '/^# "\\e\[5/ s/# //' -e '/^# "\\e[\6/ s/# //' /etc/inputrc

Okay, so what does this do?

sed is the stream editor that let’s one search and replace inside files.

sudo is the command that grants permissions in Raspbian to edit files in /etc

--in-place=.original is the sed option to copy the file to a new name, first, before editing the file. The file will named the same as it was before, but with .original appended to the file name. I could just have easily used .backup or .bak

-e is the sed short form of --expression

It tells sed that what is between the single quotes is an expression to process. Note that the above command line has two expressions.

The two expressions are almost identical. Let’s break the first one down:

/^# "\\e\[5/ s/# //

First is a search, designated by the first /…/ pair. Second is a substitution, designated by the /…/…/ triplet (but the reason it is a substitution is because of the s in front of the triplet).

In this case, what we are searching for is:

^# "\\e\[5

The line in /etc/inputrc literally has this:

# "\e[5~": history-search-backward

You can see that the sed search expression ^# "\\e\[5 differs from the actual line by the leading carat character ^ and a couple extra backslashes.

That leading ^ (carat) is the instruction to sed that the match only works at the beginning of a line. First find the beginning of a line, then proceed.

The next few characters are just an exact match: # "

The line in inputrc has \e following the double quote mark; but, because this is going to processed on a command line (or by a shell script), we have to warn the command line interpreter that there is an actual backslash in the expression. Backslashes are normally an “escape” character on the command line, warning that “whatever you would normally do with the next character that you see, don’t do it. Even though the character is in the list of special characters that get special handling, just ignore that and deal with it as if it were not a special character”. Which is a long way of saying that to deal with \ in the data, we need to specify \\ in the script or command line.

Same thing applies with the [ character; it is normally a special character. The inputrc file has e[5 in the data, so we need e\[5 in the expression.

So the search part within the first two slashes matches when there is a beginning of a new line, a pound sign character, a space character, a (double) quote character, a backslash character, the letter e, an (opening) square bracket character, and the numeral 5.

The substitution part of the expression matches on the pound sign character and the space character, and replaces them with nothing. That’s what s/# // does: “s” in front of the slash triplet says we are going to do substitution, what to substitute are the quote and space, and what to substitute them with is the empty set: between the second and third slashes is nothing.

The second expression differs from the first by 6 instead of 5. This is from Changing the history search keystrokes

Easy button:

update_system_defaults.sh

(One should always look at the contents of the file instead of just blindly running it)

Firefox / Mozilla – on Windows – decided I did not need a Bookmarks button. WTH?

Two environments: work and home. Two platforms: Linux and Windows. Symptom is only on Windows, but spanned both work and home.

I do use the Firefox Sync feature, but of course two different email accounts for work versus home. I realize I am kind of stretching things; but, I use the Firefox sync thing across both Windows and Linux for both work and home.

Once I finally figured out how to get the Bookmarks button back, I tried to synchronize now to get the good/repaired installation of Firefox on Windows to propagate over to the broken versions. That did not work.

So: Firefox on Linux (both work and home) did not lose the Bookmarks button. Firefox on Windows (all machines) lost the bookmarks button. If it were a Firefox Sync thing, I would have thought that the repaired Windows installation would have propagated over to the other Windows machines.

Also, if it were a Firefox Sync thing, I don’t think it would have happened at the same time on two different accounts (work and home). Sure, I could have hit a bad keystroke and removed this button I wanted in one browser, and then Sync propagated the bad config. But that it happened at both work and home at the same time makes me think that it was a new build of Firefox that just clobbered the thing.

I am disappointed that 1) Mozilla didn’t have quality checks in place to catch this first, and 2) that Firefox Sync does not propagate the repair to the synchronized machines.

So, on every Windows machine, I now need to do:

(hamburger menu) –> More Tools –> Customize Toolbar

Then on the new screen, find the Bookmarks Menu button, and drag it to the toolbar. Repeat for all the Windows machines you have.

Need to print + OpenSUSE 15.3 upgrade – What could go wrong?

I needed to go to a new doctor yesterday. The day before, they had called and left a message that I would also need to bring along a list of my current pharmaceutical prescriptions. I got the bright idea to log in to the online pharmacy web page and print my current list. This is about 40 minutes before I need to step in to the home office to report to work.

It went poorly.

Still, if this is the worst thing to happen to me this month, I am a fortunate man. I’m a fortunate man who cannot print from Linux, but I’m still a fortunate man.

Certainly part of the problem is my fault; I had upgraded from OpenSUSE 15.2 to 15.3. 15.3 was released two weeks ago; I upgraded about ten days ago. This was not enough bake time. I should have listened to my own advice: do less yeet and more tootle. But yeeted I had, so the story unfolds ….

Okay, so I logged in to the pharmacy web page, and used the browser to print. Got no printer noises, but no error about anything, either.

The print driver I’m using is from OpenPrinting.org and it did previously work. I did print something three weeks ago. But today, nada.

Go into the printer manager in OpenSUSE 15.3 and do a test print. No printer noise, but no error alert either. It asks if the print worked; (no) so says do journalctl to see what went wrong.

I don’t like journalctl. It spits at me about permissions, and I used to just be able to just grep a log file – any log file – and search for terms like “error” or “warn” or “cups”. I just want to print, man.

Okay, dig in and find that there is an error with the driver. Reinstall the driver. The driver will not install.

The driver is dependent on LSB. LSB = Linux Standard Base, which was the idea that the various packagers of Linux would all agree on what should be in a base install of Linux (that supports LSB). Software vendors could count on the base packages being there, or worst case say, “this software needs LSB, please install it”.

I had previously installed LSB (to get the printer to work), but now it’s missing. That must have happened during the 15.2 to 15.3 upgrade.

Okay, no big deal: zypper in lsb

Problem: nothing provides ‘/usr/bin/pidof’ needed by the to be installed lsb-foo

Well that’s darling. It’s a bug, and it is fixed in OpenSUSE Factory. I just want to print, man, and it’s now 20 minutes before work.

Okay, go to the fallback position: print to PDF, copy the file to a Winders box, and print from there.

I have Nextcloud client installed and running on most of my machines. Copy the file to my Nextcloud folder. Go to a Windows machine – there are no new files in the Nextcloud folder. Machine is acting wonky anyway, so I reboot (yeet!)

  1. Microsoft decided I needed a Weather widget in my taskbar, so they inserted one without asking. I need to lose some time praying to remove the murderous rage I have toward Microsoft for being so un-invitingly forcefully helpful.
  2. Nextcloud client has an update, would I like to install? Yes, please. What was that about less yeet and more tootle?
  3. Nextcloud client version 3.2.2 is no longer compatible with your older Nextcloud server. Have a nice day!
  4. It is now 10 minutes before work. I just want to print, man; my travel time to the doctor did not pad with lead time for print fixing, and as the famous mage once said: “Outlook not so good”.

Okay, what about the web page version of the Nextcloud server? Right, dang, I forgot I was going to need my physical 2nd Factor authentication key. Back to the living room to get it.

Logged in on the Windows box. The file is not there.

Dang it! The Nextcloud client on my Linux box has the same version problem. Back to the living room, open up the Nextcloud files web portal, do the physical physical 2nd Factor authentication thing and copy the file up. Back to the home office, open the PDF in the Nextcloud files web portal in Firefox and hit the print button. Finally, noise from the printer in the living room.

Put on a shirt and my shorts, get an energy drink out of the refrigerator, and I’ve got 30 seconds to spare.

“One does not simply press print”

Next week, I’m going to install a firewall router!

New toy: Raspberry Pi 4

So of course, the first thing I want to do (after installing Raspbian and applying updates) was to add some aliases and switch the editor and history search commands.

Changing the editor to vim

Changing the editor to vim was easier, once I knew how.

  • Install vim
  • update-alternatives

Vim does not come installed by default on a Raspbian. But adding it is easy enough:

sudo apt-get install vim

I had found someone that said (when thing weren’t working the way I wanted) to sudo apt-get install vim-full ← this does not work! There is no package “vim-full” from which to install. At least not on my fresh-out-of-the-box Raspbian machine.

sudo update-alternatives --config editor

This opened up a list of available editors, numbered for selection, with an asterisk next to the default. It was set to nano, but I wanted vim.

Except that there were two vim choices: vim.basic and vim.tiny

Which one to choose? Neither looks like vim.full to me. 😉

Turns out I wanted vim.basic

Now, I can be in the less utility, and if I want to edit the file (and I already have permission to do so) I can hit the v key and be editing in vim.

Adding some aliases

This was super easy. I created a file, ~/.bash_aliases and added the alias commands to it.

alias ..='cd ..'
alias ll='ls -l'

Changing the history search keystrokes

This one was the closest to what is described in my How to make Ubuntu have a nice bash shell like OpenSuSE post.

sudo vim /etc/inputrc

Find the commented out commands, and uncomment them:

# alternate mappings for "page up" and "page down" to search the history
# "\e[5~": history-search-backward
# "\e[6~": history-search-forward

All I have to do is to delete the “#” character that declares \e[5~ and \e[6~ to be a comment. With vim, this is the x key

rsync is wonderful, but ….

rsync /datastore/61/E4 /newserver/61/E4

is wrong and will mess you up!

Imagine if you will, that you have a whole bunch of data stored on an old server, and you need to copy it to a new server. The rsync utility would be an obvious way to go. There are things about the job and rsync that you might want to tweak, though, and that’s where things get ugly. Part of this is bash’ fault.

Imagine if you will, that your data store is 120 million small files (emails) stored in 256**3 directories. 256 cubed is 16,777,216 sub directories.

The programmer that created the data store to hold all these files needed subdirectories to put the files in. Linux doesn’t really like 20,000+ files in one directory. It would be better to have more subdirectories, with less files per subdirectory. So the programmer started with a loop:

for 00 .. FF mkdir

Then the programmer did a change directory into each of those directories he just made, and did the exact same thing. 

cd 00;for 00 .. FF mkdir;cd ..
cd 01;for 00 .. FF mkdir;cd ..
cd 02;for 00 .. FF mkdir;cd ..
...
cd FF;for 00 .. FF mkdir;cd ..

That gets you to 256 squared, which is 65,536

And then the programmer did a change directory into each of those directories he just made, and did the exact same thing. All 65,536 second level subdirectories got a third level of another 256 subdirectories. That gets you to 16,777,216 which is 256 cubed.

So your file server directory structure might contain this:

/datastore/61/E4/7D

Inside good old 61/E4/7D there might be twenty to thirty files, each one holding the content of an email, or a metadata file about the email. The programmer was pretty good about filling all of the datastore subdirectories to nineteen files each, then twenty files each, then twenty one files each. No Linux system is going to have a problem with twenty one files in a subdirectory.

The only real problem here is if you need to traverse everything in /datastore – this takes forever

Back to the problem of copying everything from /datastore to /newserver. Let’s assume that /newserver in on a different machine, and we are using remote file system mount command to make the remote machine appear to be a local disk (mount point).

You might think the rsync command ought to look like this:

rsync --archive /datastore /newserver

There are two things that make this sub-optimal. First, it is single-threaded. Second, there is no progress feedback.

The single threaded part isn’t so bad; it just means that we are losing speed due to rsync overhead. The server has twelve cores, the network is 10 Gbps Fibre Channel, the /datastore disk has multiple spindles, but rsync was designed for slow networks way back when in the bad old days.

At this point, you might ask “why not do a straight cp -r” (copy command, recursive)? It’s not a terrible idea; but, what if there were a network glitch? The entire cp -r would have to be started over, and every bit already copied would be copied again. This is where rsync shines: if the files in the destination are the same as the source, the copy is skipped. cp -r also suffers from the same lack of progress feedback.

Did I mention that the 120 million files are also 9.3 terabytes of files? I really don’t want to get to 98% done and then have a network glitch cause me to copy another 9.3 TB over, which would be the case with cp -r

The tests I’ve done indicate that four rsync commands, running simultaneously, copied the most data in the shortest period of time in my environment*. More than four rsync commands at once, and I started to saturate the disk channel. Less than four rsync commands, and something is waiting around, twiddling it’s thumbs, waiting for rsync to get busy with the copying again, which it will do, as soon as it finishes up with the overhead it’s working on.

The other problem is a lack of progress feedback. The copy is going to take multiple days. It would be nice to know if we are at 8% complete or 41% complete or 93% complete. It would be nice to be able to compute what the percentage complete is.

Well, how about 64K rsync commands, each with a print statement of the directory it is processing? And if we could run four of them in parallel, we could get the multiple jobs speedup too.

You might think the rsync commands ought to look like this:

rsync --archive /datastore/00/00 /newserver/00/00
rsync --archive /datastore/00/01 /newserver/00/01
rsync --archive /datastore/00/02 /newserver/00/02
rsync --archive /datastore/00/03 /newserver/00/03
rsync --archive /datastore/00/04 /newserver/00/04
...
rsync --archive /datastore/FF/FF /newserver/FF/FF

but WOW would you ever be wrong!

Remember old /datastore/61/E4/7D up there? This format for rsync would put E4 in the source under E4 in the destination! In other words, although the source looks like this: /datastore/61/E4/7D the destination would look like this: /newserver/61/E4/E4/7D

To be done right, the command needs to look like this:

rsync --archive /datastore/00/00/* /newserver/00/00/
rsync --archive /datastore/00/01/* /newserver/00/01/
rsync --archive /datastore/00/02/* /newserver/00/02/
rsync --archive /datastore/00/03/* /newserver/00/03/
rsync --archive /datastore/00/04/* /newserver/00/04/
...
rsync --archive /datastore/FF/FF/* /newserver/FF/FF/

The source needs a trailing slash and asterisk to tell rsync to copy the stuff underneath the source (not the source itself) to the destination (which is finished with a slash).

Enter the problem where bash is a pain in the ass.

Well, before I go there, let me mention that it wasn’t too bad to write a Perl script to write this bash script, and do three things per source and destination pair:

echo "rsync --archive /datastore/00/00/* /newserver/00/00/"
rsync --archive /datastore/00/00/* /newserver/00/00/
echo "/newserver/00/00/" > /tmp/tracking_report_file

The first line prints the current status to the screen. The second line launches the rsync. The third line overwrites a file, tracking_report_file, with the last rsync finished.

So, crank up screen first, launch the bash script, and some number of days from now, the copying will be done.

That /tmp/tracking_report_file gives me a pair of hexadecimal pairs, which I can then use to compute percentage complete. For example, when /newserver/7F/FF updates to /newserver/80/00, then we are going to be just over 50% done.

Heck, I can detach from screen, and I don’t even have to watch the rsyncs happen. I mean that I do need to, but I don’t have to. Better yet, I can take the same routine that converts the pair of hexadecimal pairs into percentage complete and wrap that inside a cron job that sends an email. Progress status tracking accomplished!

But this does not solve the single-threaded rsync problem.

And ultimately, I could not get it done.

What looked to be an okay solution was using the find command, to feed into xargs which could do shell stuff in parallel. I even got as far as getting bash shell variables to create the rsync --archive /datastore/00/00/00 /newserver/00/00/00 part.

Okay, that would be 16 million smaller rsyncs instead of 64 thousand larger ones, but I might even be able to bump up the parallelism to six or eight or nine.

But the serious problem the rsync –archive /datastore/00/00/00 /newserver/00/00/00 command has, is the naive problem: the missing trailing slash and asterisk are going to put the source underneath a destination. I need to put the trailing slash and asterisk on there.

And bash says “that’s a nope”

Trailing slashes and asterisks are automatically culled from output, because (reasons).

Oh well. The find command also spits out the directories it finds in rather random order. My bash script with sequential rsyncs by sorted order means that the last one complete really is some-percentage-of-the-total done. But if find chooses to spit out /datastore/b3/8e/76 instead of /datastore/00/00/00 then my status tracking doesn’t actually work. I would be forced to traverse all of /newserver/ and count which of the 17 million are complete; which would take freaking forever.

Yes, I said 17 million. Did you notice that the programmer that created subdirectories did some of them in lowercase hexadecimal? That happened when we brought in another email system (Exchange). Lovely.

*the last time I did this migration, although it was on a four core box, then.