AMD have started advertising their “AI Chatbot” a lot recently, but what exactly is this? Is it some groundbreaking new AI Chatbot that will ‘defeat’ Chat With RTX (from NVIDIA) or ChatGPT (from OpenAI)?
In AMD’s install instructions, they mention LM Studio – and a few different AI LLM models you can try out. I break down what this all means in this video, and also discuss why “AMD’s AI Chatbot” isn’t really AMD-specific after all!
I also discuss some alternative Large Language Models that you can run on your PC, instead of the ones that AMD recommend.
If you prefer text over video, please read on for the guide/transcript version of this video.
Video Transcript And Guide
Hey everyone, there’s always a “latest craze” in the tech industry that gets investor’s dollars flowing. A few years ago it was “bitcoin” and “blockchain”.
You MIGHT already know that NVIDIA is doing REALLY well in the AI space, to the point that gaming is now only a minor priority for NVIDIA. But what about AMD? Well they are eager to compete with NVIDIA, so they’ve recently been advertising the AMD AI chatbot. But is this anything special, or just marketing?
AMD AI Chatbot
Well if you look at one of AMD’s “AI blog posts” about how to run a local “AI chatbot” on your “Ryzen AI PC” – lots of buzzwords there – you’ll see that it mentions LM Studio. But here’s the thing: that’s NOT an AMD specific project. LM Studio is just a neat way of running a range of Large Language Models (basically, AI Chatbots) on your Mac, Windows or Linux PC. If you look at the system requirements, they’re wide open – both NVIDIA and AMD graphics cards are supported, and any CPU that supports AVX2 can run LM Studio.
That would all suggest that ANYONE can run “AMD’s AI Chatbot”, right? Or is there some “special AI” that AMD are doing here that I’ve overlooked so far?
Well firstly it’s worth pointing out that AMD ARE working on improving the drivers for both their CPUs and GPUs so that they have more “AI support”. They’re also keen to do a deep tech dive on their blog that discusses how you can quantize and run some AI models from scratch – which is pretty cool.
So they are getting more serious about AI support. What this means is that one year ago, we might have struggled to run local AI chatbots on our PCs, and now we can – so that’s pretty cool.
LM Studio AMD Chatbot Install
We basically just need to follow the instructions: download LM Studio (or the preview release if you have a 7000 series GPU). I downloaded the standard LM Studio but as I discuss later on, I CAN still use my GPU to power the AI chatbot – it’s just that since it’s a 6000 series GPU, I can’t take full advantage of some extra AMD AI support. Once the program has downloaded, you can search for a specific AI model – either of the two they recommend should work fine for basic demo purposes.
So I went with LLAMA and I downloaded it. You will notice that LM Studio tells you what your RAM and VRAM capacity is, and that the models will show whether the models can run entirely in your graphic card’s VRAM (assuming your GPU supports running AI models, of course).
That’s good to know because being able to load the entire model into RAM or VRAM WILL make things run a lot faster. While AMD recommends the Q4 model, choosing the highest number of quants that runs smoothly on your hardware is probably best because you’ll end up with a more powerful AI model.
The model will be quite a few gigabytes so plan ahead if you have a slower internet connection. Once it’s downloaded, click the Chat tab on the left hand sidebar and select the particular model you want to use. Before chatting, select “GPU Offload” if this option is enabled because THIS will use your graphics card to speed up the processing of the chatbot. You can then start chatting. You WILL probably notice a small delay before you get each result – that’s normal enough for AI chatbots.
YES ChatGPT and Claude are a bit quicker, but this runs entirely local so it will be slower. You can see that the results still come back fairly quickly and are pretty decent results. I asked it to generate a super awesome poem which it did, and then I asked it to generate a very simple Python script which it also did – and it ran through fine.
AMD AI Chatbot Thoughts
So that’s pretty cool, it is actually amazing to me how far AI has come in the last one or two years. Being able to run a half decent chatbot entirely on our own consumer-level hardware is pretty sweet. However it’s still worth pointing out that these local chatbots are nowhere near as powerful as the chatbots provided by OpenAI and others… which I guess makes sense, of course, because they’ve spent hundreds of millions of dollars on AI supercomputers, after all!
But it IS still cool that we can run a chatbot locally that costs us no money at all and doesn’t send all our data off to “the cloud” somewhere. My light testing of “AMD’s AI Chatbot” – and remember that it’s not really AMD-specific – is that a local chatbot can work fairly well especially for simple coding help, so this could be a decent stop-gap measure if you’ve exhausted your usage limits on ChatGPT or Claude, for example.
Alternatives To AMD LLM Chatbot
Having said all that, it’s worth pointing out that some users prefer kobold for running AI models. It IS more technical and harder to set up, but it’s also more powerful. And this can also run fine on supported AMD hardware.
So is “AMD’s AI Chatbot” some groundbreaking new development that will change AI forever, and make Lisa Su the world’s richest person? Well no, it won’t. NVIDIA are still waaay ahead here, to be honest. Chat With RTX is a more powerful chatbot, and NVIDIA’s graphics cards run AI models better than AMD’s cards will (in general) due to NVIDIA’s tensor cores which are pretty amazing and powerful, to be honest.
I’d say that if you were looking to build a brand new PC and one of your main priorities was running local LLM models, you should go with an NVIDIA GPU and check out Chat With RTX – or even run through the AMD Chatbot blog guide because THAT will run just fine with NVIDIA hardware too! AMD ARE running behind the “AI curve” here, to be honest. BUT it’s still good that AMD are improving hardware support for AI models, and if you have an all-AMD build like I do then at least we can now run local chatbots through LM Studio with minimal hassle – that’s pretty cool.
If you have any questions, feedback or suggestions about this article, please leave a comment below. Please note that all comments go into a moderation queue (to prevent blog spam). Your comment will be manually reviewed and approved by Tristan in less than a week. Thanks!