Author: davidg

The Helm migration

I really liked my The Helm email appliance. But because the company running the service behind it is going to exit this business, I need to migrate stuff. Oh so much stuff…..

Of course, really, it becomes so-much-stuff because once I’m in a little, I want to pile on more. If Reddit hadn’t become so much trash, I’d have probably been living in /r/SelfHosted these past few weeks. Well, that and except that I’m cloud hosting for myself instead of keeping a box here at home.

Anyway, The Helm provided me with a SMTP server on it’s own domain name, and, NextCloud Files. (It did not include any other parts of NextCloud, though) (I think. Maybe contacts, too?). The company provided DNS services, too. And because no ISP is going to let me run an SMTP server here inside my home, it provided VPN services to AWS where boxes on the public Internet could send port 25 mail from.

I needed to move, and move quick. I’ve seen before how “oh I’ve got plenty of time” turned into “oh crap! It’s due tomorrow‽” enough times to remember the pain.

So now I have learned and am running:

  • A Dovecot and Postfix and rspamd server, with Redis
  • RoundCube attached to same
  • ISPMail attached to same (which is a web administration console for accounts in Dovecot and Postfix)
  • A caching DNS server on same
  • A Linode DNS server, so that Certbot can authorize a wildcard Let’s Encrypt SSL certificate.
  • NextCloud (full suite)
  • Duplicati for backup
  • and I haven’t ever added WordPress yet

I’m least happy with NextCloud. There is a lot of stuff that doesn’t work, and the documentation is poor, and a lot of the forum answers are “just read the documentation, newbie.”

I’m also not really happy with Duplicati. I loved it in version 1, because it was “just” a Python script. It ran on Windows, and I could very easily back up to Amazon S3. In fact, it was my introduction to learning AWS. Version 2 comes with it’s own web server so that it can be cross-platform and have a GUI; but that means adding Mono to my previously somewhat lean Linux server. By the way, accessing a web site on a “localhost” only web server? Here’s a reminder of how.

I started seeing a memory leak, and now I have to reboot the server once in a while. As Tenets of IT number 6 points out, rebooting is a band-aid. Really, I should remove the code that creates the memory leak. I think I’ll move to Restic and Backblaze.

Though I realy want to add WordPress and migrate this blog there, next.

Certbot and wildcard domains and –expand, oh my!

Nope, you cannot use –expand if you are using a wildcard helper (in my case --dns-linode)

The command that worked was

certbot certonly --dns-linode --dns-linode-credentials ~/somefolder/somefile.ini -d davidgerisch.com -d gerisch.me -d *.davidgerisch.com -d *.gerisch.me --cert-name davidgerisch.com

certbot –expand was no good because of –dns-linode. My only choice was certbot certonly.

But leaving off the original certificate name created a new certificate in a new location with -0001 tacked on to the name. No way do I want to have to wrangle the original certificate with it’s expiration date and this new certificate and it’s other expiration date. Besides, my web server is already configured for the original certificate. Reconfiguring the web server was less than ideal.

So the secret was to use the –cert-name option to specifically update the existing certificate.

2022-12-27 Update: if you go to add another domain (which happened to be this one) and you get the error “Certbot failed to authenticate some domains (authenticator: dns-linode). The Certificate Authority reported these problems:
 Domain: newdomain.tld
 Type:   unauthorized
 Detail: No TXT record found at _acme-challenge.newdomain.tld

 Domain: firstdomain.tld
 Type:   unauthorized
 Detail: No TXT record found at _acme-challenge.firstdomain.tld

Hint: The Certificate Authority failed to verify the DNS TXT records created by –dns-linode. Ensure the above domains are hosted by this DNS provider, or try increasing –dns-linode-propagation-seconds (currently 120 seconds).”

The problem may actually be a leftover file at /etc/letsencrypt/renewal

I had two files in there: firstdomain.tld.conf and firstdomain.tld-0001.conf

Certbot was trying to use the -0001.conf file instead of the real file. The real file pointed to the actual certificates being served up. The -0001.conf file was pointing to certificates with -0001 in their name, which were never served up to any of my web sites.

Amazon Echo abandonment, a month in.

I’m trying Apple HomeKit stuff instead. It is very disappointing. Amazon understands “cloud” and Apple does not. Or maybe Apple’s heart just isn’t in it. Perhaps someone there felt a need to compete with Amazon, so they started HomeKit. But, once the reality hit of how much change it would take to do a great job, they grew disheartened and gave up.

Either way, the Apple HomeKit stuff is a Yugo to Amazon’s Porsche.

Of course, the Apple stuff is as expensive as a Porsche, so it’s a bad deal.

I was watching the television show Silicon Valley and at one point they openly mocked Apple that Apple Maps was so bad. Worse was Microsoft Zune which made me LOL. Point is, Apple then decided to make Apple Maps good, and today it is. In fact I had an address here in town I needed to get to, and Google Maps completely failed it. So I tried Apple Maps and it worked. That was quite a good accomplishment in my view: Apple delivers a better app than Google.

But HomeKit today is no bueno.

It increasing looks like I need to invest some time and effort into Home-Assistant.io

(Potential) Future Modern Discourse

AMC theaters and Zoom have announced a collaboration. Big-ass Zoom meetings with a group of people in each the theater (17 cities so far).

I think the Libertarian Party should use this technology to conduct this sort of event, to nail down what they want their official party platform should be.

Once a month, every month, a new topic is tackled. Once we get all the topics defined, we wrap around and revisit each topic, to see if it needs some realignment. Perhaps new technology brings about some change that gives us reason to adjust a position.

Then, anyone running for office who wants the Libertarian Party endoorsement would need to pledge to support all the topics defined. Also, any Libertarian candidate should know what the party stands for, as declared by it’s membership.

Once a month, dinner and a movie, except instead of a movie broadcast into your brain, you get to interactively participate in building the future.

The year 2022: Late stage 2021 but with new, higher prices

h/t to one of Scott Adams Twitter followers, responding to a challenge to summarize 2022 in the snarkiest way possible.

The whole thing is a psy op run by incompetents at behest of elites inflicted upon the aimless. It came about through sixty years of indoctrination: “Buy this shit from our advertiser; that will make you happy.”

Linode base to LVM conversion

In my last post, I whined that I couldn’t find a how-to on how to convert a Linode virtual machine to an LVM setup. Well, I’ve done it, so I should write this up, no?

I didn’t want the machine to have a swap partition; so there were three things to do:

  1. swapoff while logged on, inside the machine
  2. Edit /etc/fstab to delete the line for the swap drive
  3. Outside the machine in the Linode manager, delete the disk
    • So first I had to power the machine down
    • Then in the Linode virtual machine manager, I had to switch to the Storage tab
    • Now I can click on the swap drive and delete it.
      • I don’t know why, but WordPress is being stupid with lists, which it didn’t used to prior to the most recent “upgrade”. This sublist is supposed to be numbered, damnit. And this particular list item was supposed to be indented even further.

The next thing to do was to shrink the existing disk. I do not know if I could have just done that. I see a resize option in the Linode storage manager. It may be that they have cloud-init wired in, and using the resize button would also have run stuff inside the machine to make everything nice. That’s not the way I went. 🤷

In the Linode manager (at the upper level, where you can see all your virtual machines), there is a three-horizontal-dots menu button. (I don’t know what is the good name for this button. I like the three horizontal lines, stacked, menu buttons because I can call it a hamburger button, and people get the idea of a bun with a patty in between. But I digress.)

I clicked on the three-horizontal-dots menu button, and chose the Rescue mode menu option. This powers down my virtual machine and attaches it as storage to a rescue mode virtual machine (running Fennix). Then in the Linode manager, I used Launch LISH Console to spawn a new web page which is the remote console into the Fennix machine. Although I’m inside the Fennix machine, /dev/sda is still my virtual machine’s main disk. It is not mounted at this time, which is good. So then I ran the command to shrink /dev/sda with resize2fs /dev/sda 9G

So a very real problem with me writing this up is that I don’t have a history command to verify this is what I did. That history was recorded in the Fennix virtual machine which is destroyed after reboot. I’m pretty sure the command was resize2fs /dev/sda 9G but I don’t actually know. When I look stuff up now, it looks like resize2fs applies to the partitions inside a disk device rather than the device itself. But I’m pretty sure I did this.

Then, using the Linode manager, I did shrink the disk. So the next steps were:

  1. Reboot out of rescue mode (wait for everything to boot back up)
  2. Power down the virtual machine (wait for it to shut down)
  3. In the Linode manager of my virtual machine, resize the one-and-only disk to 9 GB
    • The base machine had used about 5 GB of the 25 GB allocated. This leaves another 4 GB free disk space, even prior to moving /var off to another disk.
  4. Then, I added four disks:
    • home
    • tmp
    • var
    • var/mail

Of course, when I added these disks, I had to pick the sizes of what I wanted each to be.

The next part of the puzzle wasn’t obvious either: how does Linode map these newly added disks to the virtual machine? The answer is that by default, it does not.

That’s over in the Configuration tab of the virtual machine manager. (Earlier documentation appears to have called this the Profile tab). Doing an edit of my virtual machine, I could pick the /dev/sdX and assign it to the disk I had created for my purpose.

Okie dokie, time to power up and do the LVM stuff.

Create the physical volumes: pvcreate /dev/sdb /dev/sdc /dev/sdd /dev/sde

Create the volume groups:

vgcreate vg_mail /dev/sdb
vgcreate vg_tmp /dev/sdc
vgcreate vg_home /dev/sdd
vgcreate vg_var /dev/sde

Create the logical volume groups:

lvcreate vg_mail -l 100%FREE -n lv_mail
lvcreate vg_tmp -l 100%FREE -n lv_tmp
lvcreate vg_home -l 100%FREE -n lv_home
lvcreate vg_var -l 100%FREE -n lv_var

So at this point, we have logical volumes, inside of volume groups (which have physical devices assigned). LVM makes this storage available at /dev/mapper

Format the new storage:

mkfs.ext4 /dev/mapper/vg_mail-lv_mail
mkfs.ext4 /dev/mapper/vg_tmp-lv_tmp
mkfs.ext4 /dev/mapper/vg_home-lv_home
mkfs.ext4 /dev/mapper/vg_var-lv_var

Now comes the tougher part, moving the new storage into production.

The process is to shut down the system to Init Level 1 (so that as little as possible is currently running), mount the new storage, copy the files over, rename the old storage out of the way, and then update the /etc/fstab to reflect the new storage mount point.

Inside the running virtual machine, I gave the command init 1

Now I have to use the Linode virtual machine manager Launch LISH Console to get logged into the running machine (Init Level 1 turns off the network).

mkdir /mnt/newvar
mount /dev/mapper/vg_var-lv_var /mnt/newvar/
cp -apx /var/* /mnt/newvar
mv /var /var.old

Okay, the contents of /var are now inside the LVM logical volume. Now to configure the system to mount that logical volume at the file system mount point /var

First, use blkid to identify the universally unique identifier assigned to the LVM volume. Perhaps blkid says your LVM volume is this:

/dev/mapper/vg_var-lv_var: UUID="epstein-didnt-kill-himself-605169120" BLOCK_SIZE="4096" TYPE="ext4"

Then, edit /etc/fstab to have the UUID entry for the mount point:

UUID="epstein-didnt-kill-himself-605169120" /var ext4 defaults 0 1

Do this for the other LVM volumes and then clean up. Before rebooting, you should try mount -a just to make sure there are no errors; because if there are errors mounting things, that’s going to make the reboot suck, badly.

Cleanup was to delete /mnt/newvar and to delete /var.old (and the other LVM mount points processed the same way).

Kind of hating cloud servers right now

How in the world am I supposed to create LVM (Logical Volume Management) disk layouts on a cloud VM with a single big disk? Before I start piling in data, I want to put /var/mail on it’s own partition.

Maybe it’s just that Google is stupid, and the answer is plain as day if I could find it.

Linode is annoying, because the pages I found said (in essence) “Don’t use LVM, use our attached disks at an additional $2 per disk per month.” Well, I could add a disk and then use LVM to configure it. But that means that I’m going to have a 25 GB /boot partiition and then hardly anything else over on the new disk. What it won’t do is keep the system from going comatose if some process starts spamming a log file and fills the disk. That’s stupid. And I’d be paying $2 a month, forever, for the stupidity.

I want to install LVM so that I have the option of adding another disk later, and it would be super easy. I’ve done LVM at work for years now, and it’s great. But at work, I get to install the machine from a boot ISO, and I get to go through every step of the install. Linode creates new virtual machines from images, where the disk is pre-configured. I don’t get to say I want /home on a separate volume (for example).

Every search I’ve done about LVM has two assumptions behind it: 1) there is a newly added virgin disk, or 2) during install, choose to partition the disk the way you want.

Nothing appears to address the situation where I’ve got a 25 GB disk with 20 GB free, and I’d like to move /home and /var and /tmp to /dev/sda1 /dev/sda2 /dev/sda3

I need to do pvcreate, but it errs out because I don’t have a newly added virgin disk.

I doubt this problem is particular to Linode; I suspect Rackspace and Vultr have the same problem – the preconfigured image is what you get; go kick rocks if you want something else.

It is frustrating, becasue I cannot be the first person on the planet to have thought of this or asked this question. But if the answer is obvious, I’m not finding it with Google search.

The Helm email appliance – you were a good product

I really liked my Helm email appliance. It has done well by me.

Unfortunately, the business behind it doesn’t see it’s future getting better, so they are going to call it quits. I have until December 31, 2022 to build a replacement email server. This is turning out to be a larger project that I’d like.

I do appreciate that The Helm company gave me plenty of warning (I got the email more than two weeks ago). I hope the people at the company find something else they can do which brings more success to them. You have my many thanks for your years of solid service.

Amazon disappointment – I’ve removed my Echo (Alexa) devices

There was a recent policy change at Amazon which I hate, and as a result, I have removed the Amazon Echo (Alexa) devices and app from my life. It does mean I’ll be carrying my phone with me more.

A part of the Vision Statement for Amazon is “Our vision is to be earth’s most customer-centric company”. Well, this change in policy belies that; trying to annoy their customers for more money is the new practice. That’s the nice way to put it; predation is the stark reality.

So, what happened?

  1. Amazon Music started inserting audio advertising into my morning music play. This happened four days ago.
  2. Amazon raised their prices on Prime membership, and I opted-out at the beginning of October (about one month ago).

I’ve mentioned before that I hate bullies, and dislike advertising. I also really liked setting up my morning wakeup routine to start the day with inspirational music. This change by Amazon crossed all three lines.

So if my morning wakeup routine is spoiled anyway, what really do I need these devices for?

Other than as voice controlled light switches, they are sometimes a convenient voice controlled timer. I don’t need this – it’s a nicety at best.

The bargain was they would listen in, the app on my phone would do tracking, and Amazon would do (whatever) with that data. I assumed they were monetizing it somehow; that was fine – that was the bargain. But now that they’ve crossed the line and spoiled my morning wakeup routine, I’m out.

Really, the only power consumers have is the power of money; either the boycott or favored trade.

Part of the annoyance is that I purchased the .mp3 files outright. I made my playlists out of only these files. Yes, I wanted the artists to get paid for their work, and thought is was only fair that Amazon also got paid for doing the work to set up the deal, import the files and handle the transactions to the artists. My point is that I paid for these files. Anyone that would wrap my files inside their advertising is a bully / predator.

This morning, the advertisement specifically said “Buy Amazon Music Unlimited and you won’t get advertisements”. Or, I can just completely opt out. Spotify costs the same as Amazon Music Unlimited, they do have Joe Rogan, and they have an API I can use to create my morning playlist programatically.

In that way, Spotify is better: I can write a Python script to classify songs into lists, and pick two from the spirtitual category, one from the energetic category, build today’s list and program Spotify to play that. I could even then put the songs in a FIFO queue (perhaps with some randomization). Much better than anything I could get with Amazon Music.

It’s not lost on me that the Open Source community has a project, Mycroft, which would let me connect to my IoT devices without the data tracking which was the part of the Amazon Echo bargain. I’ve already got one Raspberry Pi. All it really needs is a microphone. Guess what I’ve got on order?

Maybe I don’t even need Spotify. Maybe I can just get Mycroft or Home Assistant to play .mp3 files on various Bluetooth connected devices.

Until I get that set up, I’ll have to use my phone apps for controlling the lights and keeping timers. This is a minor inconvenience at worst. And if eventually I hook Mycroft up to a Home Assistant and a Magic Mirror, the better.