Help with LLM Setup (LM Studio & Hermes Agent)

[tinfoil-hat]

Hi, I followed this tutorial:

https://www.amd.com/en/blogs/2026/run-hermes-agent-locally-on-amd-ryzen-ai-max-processors-and-radeon-gpus.html

I have a GPU with 8GB VRAM and the Model I chose is qwen3.5-9b.
I got it all connected, but I have an error message saying:

Failed to initialize agent: Model qwen/qwen3.5-9b has a context window of 45,455 tokens, which is below the minimum 64,000 required by Hermes Agent.  Choose a model with at least 64K context, or set model.context_length in config.yaml to
override.

Is that Setup even possible in terms of the model I chose and the GPU I have? I'm assuming I just can increase

context_length: 65536

in ~/.hermes/config.yaml
Or is that not possible with my 8GB VRAM?

 

[dos2unix]

8GB is very very tight for a reasonable LLM, also while I like AMD, rocm is definitely not up to same maturity as cuda.
I haven't tried this in 6 months or so, but I never could get the webUI to work correctly with rocm.

 

[tinfoil-hat]

8GB is very very tight for a reasonable LLM, also while I like AMD, rocm is definitely not up to same maturity as cuda.
I haven't tried this in 6 months or so, but I never could get the webUI to work correctly with rocm.

Thank You for the quick answer. The Model works just fine for Text Chat and Crawling the web via Searx-NG. Sadly I can't get Hermes Agent to work.

 

[AlphaObeisance]

I'm not familiar with Hermes directly, but I did google it. I think the very nature of a self learning AI model would be problematic with limited hardware due to the vary nature that as the model learns, the model will inevitably grow to accommodate the learned dataset. So while you might be able to get it to function initially, eventually it would out grow your hardware limitations.

Purely theoretical. If any wiser than I could validate or debunk this theory I'd appreciate it. Seems like logical thinking but then again, i just rolled out of bed.

 

[dos2unix]

So you got me interested again...

It's a hog, but pretty sweet once it's setup.

 

[AlphaObeisance]

So you got me interested again...

It's a hog, but pretty sweet once it's setup.

Seeing OpenWebUI's (?) got me prompted to try it myself. Birds of a feather in that regard.

 

[tinfoil-hat]

I got one step closer to solve this. When you try to connect hermes agent to LM Studio, you have to enter as API-Key: lmstudio but now I get

Failed to initialize agent: Model qwen/qwen3.5-9b has a context window of 4,096 tokens, which is below the minimum 64,000 required by Hermes Agent.  Choose a model with at least 64K context, or set model.context_length in config.yaml to
override.