Fun project: ROCm and local LLMs

Back in May, I got the idea that while I was still employed and had money, I should buy upgrades for my computer. I did so. Last week, I implemented the plan, and it worked.

Background

Back in 2017 (nine years ago!), I bought an ASRock motherboard. At the time, AM4 CPUs were all you could get. I did not buy the latest and greatest CPU, because the latest and greatest is always overpriced. Although Ryzen 7 was available, an AMD Ryzen 7 1800X went for $500. Instead, I bought a Ryzen 5 1500X for $150.

Later, in 2020, I bought an MSI motherboard because I was having trouble with random lockups. But I was still on the Ryzen 5 1500X and 16 GB RAM.

This rig did just fine for me for the last six years. Really, for the last nine years. I’d kept a video card from 2013, though, and it has only 3 GB of RAM. Okay, back in 2013, I splurged the heck out of my purchase there because the video card was $700 before tax and shipping. But for the games I enjoyed playing (Factorio, Tropico, Endless Space, XCOM2, World of Warcraft), it was plenty.

Somewhat, I’d like to play Baldur’s Gate III next. The 3 GB RAM video card was nowhere near good enough.

May 2026 purchases

32 GB RAM (Man, those prices hurt.)
MSI Radeon RX Vega 56 (8G RAM)
AMD Ryzen 7 5700X3D CPU

The RX Vega 56 was the video card with the largest amount of RAM that I could get without growing into a PCI 5 slot. The motherboard has only PCI 4 slots, so it didn’t make sense to me to pay more money for a card with a capability it cannot use. In the back of my mind too, I didn’t want to mix a card with PCI 5 capability being stymied by a PCI 4 motherboard. If everything matches and lines up, then there won’t be weird problems because some software expects PCI 5 access.

I did want the Ryzen 7 5700X3D processor because it is built with a 7 nm process (oh my goodness!) and consumes (only) 105 watts. The X3D part of it has a whopping 93 MB of L3 cache.

Well, when I received the parts, of course I added the RAM first. Easy peasy, but it didn’t change things much. I wasn’t doing much work that needed RAM.

I went to install the RX Vega 56, and… this video card is about 1/4″ too long for my case. Oof.

Well, way back when, I’d built a Windows 7 box and bought gear for it to be beefy. It sat in a large and quiet Antec P180 case, with an Antec 850 watt power supply.¹ So I’ve got a case large enough, and it has the power supply with the extra connections for the RX Vega 56. I’m just going to have to find the time to do the migration.

Migration

I had moved the Antec case / Windows 7 box into my old office, where I used to work from home, and then pulled all the components out.

Last week, on Wednesday, everything lined up, and the whole day was mine. I started with vacuuming and dust blowing the Antec case. Then I powered down my main machine and started the migration process. Between the cleaning and component moves, it took about five hours before I hit the power switch on the new rig.² It had the same MSI motherboard but now with the Radeon RX Vega 56 and the Ryzen 7 5700X3D.

The magic smoke did not escape! In short order, Debian was running, and I was happy. I did move the audio from electrical to S/PDIF (IEC 60958-3).

The project

I asked an AI to guide me through adding ROCm to this machine. It did so pretty easily. However, the devil is in the details, and pretty quickly, I was running into all sorts of obstacles. It helped that I had a specific goal: turn an MP3 file into a summary of a meeting.³

The AI guided me through installing llama.cpp⁴ with HIP to connect to the hardware via ROCm. There were numerous packages that HIP expected would be there that I had to find out about because they were missing. Thankfully, pasting the messages of the failed compiles would lead the AI to figuring out what was missing.

By the time I was done, I’d added all these:

rocminfo clinfo hipcc cmake build-essential git pkg-config mesa-utils libamdhip64-dev rocm-cmake clang llvm libclang-rt-dev libclang-rt-19-dev libomp-17-dev libhipblas-dev libcurl4-openssl-dev

One of the snags was that llama.cpp had a dependency on HIP 6.1, but my hardware can only do HIP 5.7.

git clone https://github.com/ggml-org/llama.cpp.git llama.cpp-rocm57

Amazingly, the AI guided me through downloading the llama.cpp-rocm57 source code, find the check-in for the code that depends on HIP version 6.1, and guided me through deleting it!

This worked!

I am an utter git newbie, but the AI has enough training to guide me through the whole process.

After getting the initial part of ROCm, HIP and llama.cpp-rocm57 installed, I asked about how to do transcription. “Please guide me through installing an LLM to do speech to text, inputting an mp3 file recording of a meeting, and outputting a transcript of what was said.”

The AI then guided me through installing Whisper (whisper.cpp). That went easy. I got about 11,000 words out of the transcription.

Then I asked how to summarize it. It generated a prompt file for me, and threw that at llama.cpp-rocm57. I did run into the problem that the 11,000 words were too much for the 8 GB video card. But I split the transcript file in two, and ran llama.cpp-rocm57 on each piece.

It worked.

I even got a nice little summary like this:

Action items

Action	Owner	Deadline	Evidence/notes
Find a cook	Unclear	Unclear	The group is looking for a cook.
Decide on menu	Unclear	Unclear	The group needs to decide on a menu.
Determine ticket price	Unclear	Unclear	The price of the event is still uncertain.

This is just so cool: from an hour-long MP3 recording of the first planning meeting to a list of action items, all on my local machine. I’m thrilled.

Foolishly, I hadn’t kept on with the task of installing a Bitcoin miner in it. I would later find in this machine a Bitcoin wallet from summer 2014. But back in 2014, the idea of running an 850 watt power supply full-time during summer really did not appeal to me. But if I had been able to mine 1 Bitcoin, that would be worth about $80,000 today. ↩︎
I’m retired now, so I don’t have to be in a rush. ↩︎
I had volunteered to be the project lead on a New Year’s Eve party. I had recorded the first meeting on my Sony ICD-UX570. In the past, I would pay Amazon to transcribe it for me, but that would cost around $3. I wasn’t terribly thrilled about uploading a recording of a meeting to the cloud, either. ↩︎
I tagged this post with “Facebook” because although I have my beef with them, they do provide their large language model, Llama, as open source and for free. Credit where credit is due, man. ↩︎

Dystopian Future

Dear FutureMe,

Just saw this new video of a dystopian future: ROSE | Short Film by Omid Pakbin

It is well done. Omid says he got the entire film done for $8,000.

I’d like to post it in the politics channel of some of the Discord servers I’m on, where my Circle-Of-Friends-On-The-Internet hang out. Well, I’d like to, but being on OpenSuSE, the Discord app is broken because the publisher has a new version out, and they don’t publish to the OpenSuSE repositories. I have to wait until a community member packages it for me. I have a running list of when Discord is working versus not working here: Discord app update – hooray! 🙂

As of this writing, Discord has been down since the 12th, and today is the 17th.

Anyway, if I’m still alive in 2028, how close did Omid nail it?

We already know that China is implementing the social credit score system today.

Seven days after Biden was elected, a guy, Douglass Mackey, was charged and found guilty of a tweet in the 2016 Presidential Race. Yes, in 2023 he was sentenced to 7 months in prison for a 2016 joke tweet. “Social Media Influencer Douglass Mackey Sentenced after Conviction for Election Interference in 2016 Presidential Race“

Recently there was a hearing on weaponizations of the government, where testimony was provided that after the 2016 election, federal officials said “something had to be done”. In their view, an outsider presidential candidate was such a threat that they needed to implement some censorship regimes: and they did so. Both Facebook and Twitter (pre- Elon) censored the Hunter Biden laptop story so that it wouldn’t spread and change the 2020 election.

There have been several hearings about weaponizations of the federal government to implement a censorship industrial complex. The censorship industrial complex was proposed in 2018, and was in force at Facebook and Twitter for the 2020 election. We wouldn’t know about Google unless a whistleblower comes forward; but their deplatforming of certain people on YouTube is obvious censorship.

Google, for weeks, was scrubbing news about the biggest denier of Pizzagate being found to be a producer of child porn. Today, they no longer say “this story is developing therefore it is too new to report on”, but they don’t link to much that makes Slade Sohmer look bad. They (mostly) only link to news that it’s all conspiracy theory.

So Google is on board with the censorship industrial complex. Facebook is too. Twitter / X under Elon appears to be the only social media company against it. Rumble will gladly host people against it, but they aren’t gaining much traction.

Bruce Schneier also points out that spying on people will only get worse with AI.

So the question is: just how close to truth is Omid Pakbin’s video going to get? It’s a great movie short: less than twelve minutes long. Very high quality, and amazing that it was pulled together which such a low budget.

“Free” apps for your smartphone

There has been a long standing piece of knowledge in the computer industry that if you are not the customer, you are the product being sold. That is to say that advertisers are the customer, and the data that the “free” services harvest from you is what is being sold to them.

There’s a reason why you get offers for a free hamburger if you order is from their app (and the like). If you use the app, you collect reward points and get discounts and such.

Oh By The Way

TANSTAAFL

Do be very selective in what apps you install. They are all pretty much data harvesting machines.

What are the fools at Facebook thinking?

Banning Doctor Ron Paul? Don’t these idiots know he is one of the few good politicians in Congress?

Doctor Paul is famous for having a sign on the edge of his desk in his Congressional office, designed to stop lobbyists in their tracks: “Don’t steal. The government hates competition”. Any time a lobbyist asks for a meeting and offers to donate to Doctor Paul’s re-election campaign in exchange for a yes vote on some piece of legislation, Ron points to the sign and says “If it was a good piece of legislation, you wouldn’t need to come in here with this bribe. Sorry for wasting your time. Goodbye.” And then he informs his secretary to never book a meeting with that lobbyist ever again.

Ron Paul is famous for voting no on every piece of legislation that wastes your and my tax money. As such, he has gotten the nickname “Doctor No”.

A signature joke by Jay Leno waits until the beginning of the congressional summer break. Then on that day, Jay says “Congress has begun their summer recess. The realm is safe, once again.” Ron Paul is working this problem from the inside.

Ron Paul quote number one: “War is not popular. It may seem popular in the short run, when there appears to be an immediate victory and everyone is gloating but war is not popular. War is not popular. People get killed, and body bags end up coming back. War is very unpopular, and it is not the politically smart thing to do.”

Sure, the congresscritters getting lobbyist money from the military industrial complex will tell you that war is great. Well, it’s great for funneling money into the pockets of the people in the military industrial machine. Quote: “War is never economically beneficial except for those in position to profit from war expenditures.”

Ron Paul also understands the basic truth of all government. Quote: “Values in a free society are accepted voluntarily, not through coercion, and certainly not by law… every time we write a law to control private behavior, we imply that somebody has to arrive with a gun, because if you desecrate the flag, you have to punish that person. So how do you do that? You send an agent of the government, perhaps an employee of the Bureau of Alcohol, Tobacco and Flags, to arrest him. This is in many ways patriotism with a gun – if your actions do not fit the official definition of a “patriot,” we will send somebody to arrest you.”

And, we use tax money to pay for that agent. Regarding intervention in Sudan, the quote was “… I have no right – no moral right or constitutional right – to come with a gun and tax the people and say: “I will take money because I want to do good”.”

Quote: “The most important element of a free society, where individual rights are held in the highest esteem, is the rejection of the initiation of violence. All initiation of force is a violation of someone else’s rights, whether initiated by an individual or the state, for the benefit of an individual or group of individuals, even if it’s supposed to be for the benefit of another individual or group of individuals. Legitimate use of violence can only be that which is required in self-defense.”

This is true for all government action. Government has been described as a bunch of do-gooders, doing good with other people’s money. You can do good with your own money. And then, you aren’t guilty of robbing me of mine.

Here’s another quote about government power: “What we give up on is a tyrannical approach to solving a social and medical problem, and we endorse the idea of voluntarism, self-responsibility, family, friends, and churches to solve problems, rather than saying that some monolithic government is going to make you take care of yourself and be a better person. It’s a preposterous notion, it never worked, it never will. The government can’t make you a better person. It can’t make you follow good habits. Why don’t they put you on a diet? You’re a little overweight, and I think you need government help!” (pointing out the absurdity of the ideas of some about government power).

Quote: “The theory of the IRS is rather repugnant to me because the assumption is made that I, the government, owns 100% of your income and I permit you to keep 5%, 10% or 20%. You’re vulnerable, you’ve sold out. The government can take 80% if they want, which they did at one time.”

I like to make a joke that in a year, working 40 hours a week for 52 weeks a year computes to 2080 hours. Is it coincidence that the IRS form is the 1040 ? (fifty percent)

Quote: “Freedom is not defined by safety. Freedom is defined by the ability of citizens to live without government interference. Government cannot create a world without risks, nor would we really wish to live in such a fictional place. Only a totalitarian society would even claim absolute safety as a worthy ideal, because it would require total state control over its citizens’ lives. Liberty has meaning only if we still believe in it when terrible things happen and a false government security blanket beckons.”

Doctor Paul voted against the PATRIOT Act because he was only given one hour to read it before the vote, and it was a 400 page bill. That shows he has integrity, even in the face of taking unpopular action.

And the fools at Facebook banned him.

I can only believe that they’ve turned off their brains, and are listening only to the pro-government-growth scare mongers and empire building lobbyists and bureaucrats.

What do they know about Ron Paul? Nothing. But their “friends” say he is a bad man. So they practice a principle which cannot fail to keep them in everlasting ignorance: that principle is contempt prior to investigation.

Shame on you Facebook.