WordPress migration notes

I have a production WordPress site on Amazon Lightsail that I need to migrate away from. These are notes on how to migrate over only the stuff I want to keep.

Backstory: Amazon Lightsail was very inexpensive, at under $5 per month for hosting on their smallest machine, and it did fine. Two things became problems, however:

  • Bitnami WordPress is super easy to spin up, and everything just works. But upgrading to a newer version of something (say PHP or MySQL or something) is a non-starter. The only way to upgrade is to spin up a new machine and do a migration to a new machine.
  • Amazon recently did a price increase. Now, I can get a Linode machine with double the RAM for only $2 more, and that will include backups.

Okay, so I need to migrate, but over the years, I’ve tried different plugins, and even though many of them were uninstalled, the installation routine left crap in the database. How to migrate to a new server, but leave behind the crap? This will be the topic of this post.

First, I installed WP-CLI, instructions can be found here.

Then, on the new machine, I installed only those Plugins which I know I need.

I took a snapshot backup at this point, simply because it seems prudent.

On the new machine, I logged in with ssh and ran this:

wp --path='/var/www/html/wordpress' db query "SHOW TABLES" --skip-column-names --allow-root

This gives me a list of the tables in the new machine that I want from the old machine.

+-----------------------+
| wp_commentmeta        |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_termmeta           |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+

This is a pretty minimal list; the old machine has a list 362 tables long! Matomo was a particularly egregious offender here.

With this information, I can use a script written by Mike Andreasen over on the WP Bullet website to dump the databases on the old machine:

# set WP-CLI flags
WPFLAGS="--allow-root"

# define path to the database dumps without trailing slash
DBSTORE="/tmp"
# get the name of the database
DBNAME=$(wp config get DB_NAME ${WPFLAGS})

# list all of the tables regardless of database prefix
TABLELIST=(wp_posts wp_postmeta)

# create the temporary directory for storing the dumps
mkdir -p ${DBSTORE}/${DBNAME}

# loop through tables and export, log details to /tmp/mysqlexport-<database>.txt
for TABLE in ${TABLELIST[@]}
do
    # export the table
    wp db export ${DBSTORE}/${DBNAME}/${TABLE}.sql --tables=${TABLE} ${WPFLAGS} | tee /dev/stderr
done > /tmp/mysqlexport-${DBNAME}.txt

With this done, I scp the files from the old machine to my local machine. Then I scp them up to the new machine. The next script assumes they are in the sql directory in the wordpress folder.

I tried it, but I should have taken a snapshot, first. 😉

I need to search-and-replace all instances of the old domain name in the MySQL dump files, and put in the new domain name. Technically, once the actual switch happens, the new machine will be found at the old name, so this shouldn’t be necessary. But, the whole reason for migrating to a development machine is to test out this migration process. And the new machine does have a different domain name.

The script to upload the MySQL dumps looks like this:

# define WordPress path
WPPATH="/var/www/html/wordpress"

# loop through all of the 
for DUMP in /var/www/html/wordpress/sql/*.sql;
do
    wp db import ${DUMP} --allow-root --path=${WPPATH}
done

But, until the data is cleaned up, the new WordPress website gets the dreaded white-screen-of-death.

New Debian install; ssh and sudo changes

Similar to what I wrote in New OpenSuSE Tumbleweed cannot ssh in but this time with Debian. This has to be done from a physical console login on the machine (or if it was a VM, from the hosting company’s console login desktop service). I’m logged in as root.

apt-get install vim

Debian is pretty bare-metal, man. This is probably good from a security and stability point-of-view.

cd /etc/ssh/
cd /etc/ssh/

Find PermitRootLogin and uncomment it, and change it to yes

Find #PubkeyAuthentication yes and uncomment it.

Find #AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2 and uncomment it and remove the second file authorized_keys2

Find PasswordAuthentication no and uncomment it and change it to yesnote that this is temporary!

Save and exit the sshd_config file. I’m not sure which service(s) would need to be restarted here, so I issue the reboot now command and watch the machine reboot. Today’s hardware is amazingly fast, compared to what we lived with a decade ago.

Now, from my remote machine, I ssh in as root. I get asked about accepting the private key, and get prompted for the password. Once I get in, I know I’m good to proceed to the next step.

ssh-copy-id root@host.domain

I get asked to put in my password again, and now public key logins are enabled, instead of password-based logins.

I log in as root again, but this time without a password. At this point, I do some customizations per How to make Ubuntu have a nice bash shell like OpenSuSE (although this is Debian). One nice thing is that ~/.bashrc already had aliases ready for ll being an alias for ls -l

Something I don’t understand is why I cannot copy / paste from the Debian ssh session. My guess is that is has something to do with LS_OPTIONS in the bashrc file. Anyway….

I still needed to add alias ..='cd ..' though.

I log out.

I log in as a non-root user, with a password.

ssh-copy-id user@host.domain

I log in as the non-root user, without a password. Same thing: I add the customizations I like, where I can edit with vim, from doing a less on a file, the .. alias for changing directory up one, and using PageUp to search history. I log out.

I log in as root again. Now, I need to give my non-root user sudo rights.

adduser whatever-the-non-root-user-is sudo

Back to editing /etc/ssh/sshd_config

Find PermitRootLogin and uncomment it, and change it to no

Find PasswordAuthentication yes and uncomment it and change it to no

And then I save and exit the file and reboot the box.

Now I can ssh as the non-root user, and I cannot log in via ssh as root. Also, no-one can attempt to log in with just a password. This is good.

Temporary fix for Nextcloud calendar broken sync

Nextcloud has a nice home page called the Dashboard, which has calendar items and to-do list on it. But ever since Calendar App version 4.5, it has been broken for items sourced outside of Nextcloud. In other words, if you create a calendar item on your smartphone and sync it in to Nextcloud, on the Calendar web page you can see the item, but on the Dashboard home page it will be missing. The solution is to downgrade the Calendar app to version 4.4.5

Steps to perform:

  1. In the Nextcloud admin interface, find the Calendar app and disable it
  2. ssh into your Nextcloud instance
  3. cd /var/www/html/nextcloud/apps/
  4. mv calendar calendar-old
  5. wget -q https://github.com/nextcloud-releases/calendar/releases/download/v4.4.5/calendar-v4.4.5.tar.gz
  6. tar xvf calendar-v4.4.5.tar.gz
  7. chown -R user:group calendar
  8. In the Nextcloud admin interface, select the Disabled apps section. Then Enable (but not update) the Calendar 4.4.5 app.

And now, when you go back to your Dashboard home page, your calendar will have all the items on it. 🙂

You do get to apply this fix after every update. 🙁

Technically, this post title is somewhat misleading: sync is not broken. What is broken is that items that sync in from CalDAV sources apparently have something that, when it is present, causes the Dashboard page to skip those calendar items. It just looks like sync is broken because you knew the items were on your calendar: but when you look at the Dashboard for today, they are missing. I suppose a better title would be Temporary fix for Nextcloud calendar (some) items missing from Dashboard

Ogg > MP3 (thanks, Apple) (not)

I have several CDs (Compact Discs, not Certificates of Deposit) of music that I like. When I popped them into my PC, I got several folders of files I could copy from. I chose to copy the .ogg files because I liked the idea of using an encoding format without weird licensing issues.

Apple has foiled that plan. If I try to play a playlist on an Apple device, the .ogg files get skipped because (apparently) Apple doesn’t feel like playing nice with the Open Source community. They may have more money than God, but adding another codec – that doesn’t have license issues – to their devices isn’t something they are going to spend money on.

When I work on-premises in the office, my co-workers are often noisy and annoying. I want to pop in my Airpods and play background music to drown out their inane chatter. I don’t want to carry the music files on my device; but I do have a Nextcloud server at home that can stream the audio from the Music app web page. I can log in on my iPhone and play the playlist.

But because it’s an iPhone, it auto-skips the Ogg Vorbis files. This doesn’t happen when I’m at home playing the same playlists on Linux or Windows.

So now I get to re-copy the files from the physical media to my NAS (network attached storage) which in this case is a Synology.

First, I get to delete the files with the .ogg file extension. Two steps (for example):

exiftool -p '$filename' -if '$album =~ /WOW Worship: Yellow \(disc 1\)/' *.ogg > wow_worship_ogg_file_list
This generates a file, wow_worship_ogg_file_list, which has the file names in a list.

then to delete them:

xargs -I{} rm -r "{}" < /path/wow_worship_ogg_file_list

Second, after having cleared out the disk space, I can copy from my physical CD to my NAS. That takes a while; and, after it is done, the file names aren’t wonderful. Rename music files to their title to the rescue.

Except, of course, for a duplicate file name. I have an MP3 file I bought from Amazon (published by Monstercat) with the same title as one of the files from the WOW Worship CD. I would prefer to rename the Monstercat file, but really if I’m going to be running the rename music files to their title command often, I need to change the Title inside the .mp3 file. If I don’t, the next time I run it, it will attempt to rename the file to a duplicate name that is already in use.

Exiftool doesn’t really write new Titles, apparently. I think it can, depending on the file type. I wonder if the weird license problems of MP3 are at the root of the problem. Whatever: the answer was to add the id3v2 program and use it instead.

id3v2 -t 'Title by Artist' file.mp3

Now the rename music files to their title script moves the one file to the new file name, and the other file to its simpler file name than what came off the CD.

Discord app update – hooray! :-)

In my last post I whined that it takes a while before OpenSuSE gets the updated Discord app. I checked today, and the update was there. I only have three days to catch up on: nice!

Thank you to Wojciech Kazubski for updating the app in the repository.

I think instead of creating a new post of when Discord goes broken (does an update) and when the new version shows up fixed, I’ll just make a list here.

  • 2023-12-27 Discord client update came through, and this time it matches the server version 🙂 I was down fifteen days, but I’ve been out of town for six days.
  • 2023-12-19 Discord update came through, but maybe it’s too late? Yep, still down. OpenSuSE updated to version .38-1.1 but the Discord app wants to install version .39
  • 2023-12-12 Discord stops working until an update can be applied. Shucks. Three days. I have a group of friends for whom with-alex-jones-returning-journalists-worried-theyll-no-longer-be-only-source-of-misinformation would be a fun topic; but they will have to wait.
  • 2023-12-09 Discord works again, yay! Only four days downtime.
  • 2023-12-05 Discord stops working until an update can be applied. Shucks. Four days was nice while it lasted.
  • 2023-12-01 Discord works again, yay! Only three days downtime.
  • 2023-11-28 Discord stops working until an update can be applied. Shucks. But it was a nice long run.
  • 2023-11-18 Discord works again, yay!
  • 2023-11-10 Discord stops working until an update can be applied. Shucks.

Discord App update again (sigh)

In my previous post, I explained that every time Discord publishes an app update, I’m locked out* until someone fixes it in the OpenSuSE repositories. Four days ago, it showed up in my list of updates, and hooray! I was back in Discord after eight days of being locked out.

Today, another update is published. Sigh.

*“Locked out” is a poor term: I am voluntarily opting-out. I boycott Flatpak and Snap Apps because I dislike those technologies.

OpenSuSE updates this morning: 1,671 (but Discord isn’t one of them)

Every time Discord updates their app, I don’t use Discord for many days. This is a bummer because I have a circle of friends on the Internet I’ve known for 20+ years, and Discord is where we have settled (so far). So when Discord does an update, I go through a dry spell of not being in contact with them.

The OpenSuSE folk tell me I should install the Flatpak version of Discord or the Discord Snap app. I’m not a fan of either Flatpak or Snap. Snap fouled up a machine months ago (and it’s still broken to this day) when I installed it. Flatpak seems like a (no better) replacement for my RPM repositories. Worse, it duplicates storage and doesn’t integrate unless I convert everything over; so that just creates more points for stuff to break. But at least the single developer doing the Flatpak doesn’t have to integrate, so that’s not his problem.

I can use the web version of the Discord app. It has keystroke conflicts with the web browser, though, and because I use temporary containers for everything, it treats me like a new user who didn’t see that sticky post from five years ago….

So, I wait until someone asks the Discord people to update the RPM in the OpenSuSE repository. Eventually it happens. Time before last, it was eleven days.

Home Assistant media folder – connect to SMB share

Ooof. This one kicked my ass for a really long time. The question is “How to connect the Home Assistant Media folder to an SMB share?” There’s a wizard, but what to enter for the Remote share entry is murky.

A part of this is pretty obvious, but the other part is not. Of course, I tried the wizard first, but I didn’t enter the Remote share entry correctly. I tried reading the documentation, but it wasn’t much more than “For the Remote share entry, put in the remote share.” Home Assistant would always fail to mount the share, and the error message was (essentially) “It didn’t work”. Sigh.

I had previously created an SMB share on my Synology NAS, and could map to it just fine from my main Linux desktop, from my Nextcloud instance, and from Windows machines I have here on my home network. I knew from my Nextcloud install (adding it to /etc/fstab) that the vers=3.0 option was important.

Doing a search found a Youtube video about editing the /config/configuration.yaml file and running a shell command. It mentioned that the vers=3.0 was important. Maybe this is what I need? This turned out to be a rabbit hole (but no rodent with a mean streak a mile wide at the end 1).

Since the system-launched shell command wasn’t working, I tried the next logical step: try it from an actual command line. It didn’t work. I think that is because of Docker and the impermanence of the terminal shell and sandbox for security.

I installed a terminal app in Home Assistant, but whenever I tried the same mount command that worked on Linux, it would fail on Home Assistant with “permission denied”. Not really helpful. In fact, it seems unhelpful, because if I read between the lines 2 I see “your password is wrong” – which it wasn’t. “Permission denied” is the error message you get when your password expires, and the credentials file has the old password. Of course, I knew my password was correct: but if I were someone brand new to this, I would have been mislead by my own thinking.

Here is the mount command that does work in Linux but not in Home Assistant:

mount -t cifs -o vers=3.0,credentials=/config/.smbcredentials //mysynology.domain.tld/sharename_smb/data /media/nasfiles 3 4

The problem that I was running into was that the Home Assistant documentation never tells you what it wants for “Remote share”. The dialog box says “This is the name of the share on your storage server” – but that doesn’t help, because it doesn’t specify what to put in. That’s why I’m writing this post: if you have a mount command that does work elsewhere, the pieces you need are here.

Over on the Synology, it told me the share name was smb://mysynology.domain.tld/sharename_smb/data

That does not work here in Home Assistant.

Here are the settings that do work:

So, from the mount command above, the Server entry is mysynology.domain.tld and the Remote share entry is sharename_smb/data

Phew. This was a long time in figuring out, as I tried all sorts of stuff for the Remote share entry:

  • /sharename_smb/data 5
  • mysynology.domain.tld/sharename_smb/data
  • //mysynology.domain.tld/sharename_smb/data
  • mysynology/sharename_smb/data
  • smb://mysynology.domain.tld/sharename_smb/data
  • \\mysynology.domain.tld\sharename_smb\data
  • \\sharename_smb\data
  • \\\\sharename_smb\\data
  • sharename_smb
  • data ( with the Server being mysynology.domain.tld/sharename_smb )

Hopefully, if you found this post, it helps:

Server entry is mysynology.domain.tld

Remote share entry is sharename_smb/data

  1. Thank goodness it wasn’t the Rabbit of Caerbannog ↩ī¸Ž
  2. which is a bad idea. ↩ī¸Ž
  3. host name and share name changed to protect the innocent. Not that any of this is on the public Internet, but why tempt the random bored teenager? They can be pretty clever and persistent. ↩ī¸Ž
  4. Yes, I had to create a directory named config off the root of my Linux box and copy the .smbcredentials file to it for the mount command to be an exact replica of what would have gone in the shell command in /config/configuration.yaml ↩ī¸Ž
  5. so close. ↩ī¸Ž

New Nextcloud setup with cron and transactional file locking problems (solved)

I set up Nextcloud on a new instance of Debian, and thought I had added all the pieces for memory cache and file cache, and had set up cron to run php -f /var/www/html/nextcloud/cron.php correctly. But in the Administration Overview screen I was still seeing this:

  • Last background job execution ran 2 hours ago. Something seems wrong.
  • The database is used for transactional file locking. To enhance performance, please configure memcache, if available.

But had installed Redis and APCu and configured them … so what is wrong?

I should mention that I’m using php 8.2. Apparently, with that new version of php, the APCu code now needs an additional setting that wasn’t needed before.

Find your way to /etc/php/8.2/mods-available and edit the apcu.ini file. Add this:

apc.enable_cli=1

Finally! I have the green check mark: All checks passed.

How to test if you cron job is going to run correctly:

sudo -u www-data php -f /var/www/html/nextcloud/cron.php

I had to add the sudo package to Debian, because the basic server build did not come with that. But what it does do, is let me switch user and do the command. First, I specify the same user that Apache is going to use: www-data and then I run the PHP interpreter, using the file /var/www/html/nextcloud/cron.php

Prior to the change, it erred out with a rather ugly OCP\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)

Now after the change it simply runs without reporting anything (everything ran sucessful)