What Is The “AMD AI Chatbot”? Does It Beat NVIDIA’s “Chat With RTX”?

AMD have started advertising their “AI Chatbot” a lot recently, but what exactly is this? Is it some groundbreaking new AI Chatbot that will ‘defeat’ Chat With RTX (from NVIDIA) or ChatGPT (from OpenAI)?

In AMD’s install instructions, they mention LM Studio – and a few different AI LLM models you can try out. I break down what this all means in this video, and also discuss why “AMD’s AI Chatbot” isn’t really AMD-specific after all!

I also discuss some alternative Large Language Models that you can run on your PC, instead of the ones that AMD recommend.

If you prefer text over video, please read on for the guide/transcript version of this video.

Video Transcript And Guide

Hey everyone, there’s always a “latest craze” in the tech industry that gets investor’s dollars flowing. A few years ago it was “bitcoin” and “blockchain”.

You MIGHT already know that NVIDIA is doing REALLY well in the AI space, to the point that gaming is now only a minor priority for NVIDIA. But what about AMD? Well they are eager to compete with NVIDIA, so they’ve recently been advertising the AMD AI chatbot. But is this anything special, or just marketing?

AMD advertising their AI support
AMD advertising their AI support

AMD AI Chatbot

Well if you look at one of AMD’s “AI blog posts” about how to run a local “AI chatbot” on your “Ryzen AI PC” – lots of buzzwords there – you’ll see that it mentions LM Studio. But here’s the thing: that’s NOT an AMD specific project. LM Studio is just a neat way of running a range of Large Language Models (basically, AI Chatbots) on your Mac, Windows or Linux PC. If you look at the system requirements, they’re wide open – both NVIDIA and AMD graphics cards are supported, and any CPU that supports AVX2 can run LM Studio.

That would all suggest that ANYONE can run “AMD’s AI Chatbot”, right? Or is there some “special AI” that AMD are doing here that I’ve overlooked so far? 

Well firstly it’s worth pointing out that AMD ARE working on improving the drivers for both their CPUs and GPUs so that they have more “AI support”. They’re also keen to do a deep tech dive on their blog that discusses how you can quantize and run some AI models from scratch – which is pretty cool.

Screenshot from the AMD blog discussing how to quantize a model
Screenshot from the AMD blog discussing how to quantize a model

So they are getting more serious about AI support. What this means is that one year ago, we might have struggled to run local AI chatbots on our PCs, and now we can – so that’s pretty cool.

LM Studio AMD Chatbot Install

We basically just need to follow the instructions: download LM Studio (or the preview release if you have a 7000 series GPU). I downloaded the standard LM Studio but as I discuss later on, I CAN still use my GPU to power the AI chatbot – it’s just that since it’s a 6000 series GPU, I can’t take full advantage of some extra AMD AI support. Once the program has downloaded, you can search for a specific AI model – either of the two they recommend should work fine for basic demo purposes.

So I went with LLAMA and I downloaded it. You will notice that LM Studio tells you what your RAM and VRAM capacity is, and that the models will show whether the models can run entirely in your graphic card’s VRAM (assuming your GPU supports running AI models, of course).

LM Studio showing the model selection and GPU Offload options
LM Studio showing the model selection and GPU Offload options

That’s good to know because being able to load the entire model into RAM or VRAM WILL make things run a lot faster. While AMD recommends the Q4 model, choosing the highest number of quants that runs smoothly on your hardware is probably best because you’ll end up with a more powerful AI model.

The model will be quite a few gigabytes so plan ahead if you have a slower internet connection. Once it’s downloaded, click the Chat tab on the left hand sidebar and select the particular model you want to use. Before chatting, select “GPU Offload” if this option is enabled because THIS will use your graphics card to speed up the processing of the chatbot. You can then start chatting. You WILL probably notice a small delay before you get each result – that’s normal enough for AI chatbots.

Some sample chats with my LLAMA LLM AI model
Some sample chats with my LLAMA LLM AI model

YES ChatGPT and Claude are a bit quicker, but this runs entirely local so it will be slower. You can see that the results still come back fairly quickly and are pretty decent results. I asked it to generate a super awesome poem which it did, and then I asked it to generate a very simple Python script which it also did – and it ran through fine.

AMD AI Chatbot Thoughts 

So that’s pretty cool, it is actually amazing to me how far AI has come in the last one or two years. Being able to run a half decent chatbot entirely on our own consumer-level hardware is pretty sweet. However it’s still worth pointing out that these local chatbots are nowhere near as powerful as the chatbots provided by OpenAI and others… which I guess makes sense, of course, because they’ve spent hundreds of millions of dollars on AI supercomputers, after all!

A ChatGPT response to one of my questions
A ChatGPT response to one of my questions

But it IS still cool that we can run a chatbot locally that costs us no money at all and doesn’t send all our data off to “the cloud” somewhere. My light testing of “AMD’s AI Chatbot” – and remember that it’s not really AMD-specific – is that a local chatbot can work fairly well especially for simple coding help, so this could be a decent stop-gap measure if you’ve exhausted your usage limits on ChatGPT or Claude, for example.

Some silly chats with LM Studio
Some silly chats with LM Studio

Alternatives To AMD LLM Chatbot

NVIDIA webpage showing one of their FE cards
NVIDIA webpage showing one of their FE cards

Having said all that, it’s worth pointing out that some users prefer kobold for running AI models. It IS more technical and harder to set up, but it’s also more powerful. And this can also run fine on supported AMD hardware.

So is “AMD’s AI Chatbot” some groundbreaking new development that will change AI forever, and make Lisa Su the world’s richest person? Well no, it won’t. NVIDIA are still waaay ahead here, to be honest. Chat With RTX is a more powerful chatbot, and NVIDIA’s graphics cards run AI models better than AMD’s cards will (in general) due to NVIDIA’s tensor cores which are pretty amazing and powerful, to be honest. 

I’d say that if you were looking to build a brand new PC and one of your main priorities was running local LLM models, you should go with an NVIDIA GPU and check out Chat With RTX – or even run through the AMD Chatbot blog guide because THAT will run just fine with NVIDIA hardware too! AMD ARE running behind the “AI curve” here, to be honest. BUT it’s still good that AMD are improving hardware support for AI models, and if you have an all-AMD build like I do then at least we can now run local chatbots through LM Studio with minimal hassle – that’s pretty cool.

cropped A picture of me Tristan
About Tristan Perry

Tristan has been interested in computer hardware and software since he was 10 years old. He has built loads of computers over the years, along with installing, modifying and writing software (he's a backend software developer 'by trade').

Tristan also has an academic background in technology (in Math and Computer Science), so he enjoys drilling into the deeper aspects of technology.

Tristan is also an avid PC gamer, with FFX and Rocket League being his favorite games.

If you have any questions, feedback or suggestions about this article, please leave a comment below. Please note that all comments go into a moderation queue (to prevent blog spam). Your comment will be manually reviewed and approved by Tristan in less than a week. Thanks!

Leave a comment