Suggested distro to run emerging ML models

[surly_gerbil]

Hi, I'm new here. I've run Linux in the distant past, but I'm incredibly out of date. I've recently started playing with some fancy ML models that seem impossible to run in Windows, so I've fitted a new SSD to run Linux. I'm a bit overwhelmed with the sheer variety of distros and I'm not sure where to start, so I'm wondering if anyone could suggest a suitable distro? I'm familiar with Ubuntu (I've been using it with a translator in Windows to run Docker). I'm running a self-build with i9cpu rtx4070gpu & 32Gb RAM.

 

[CaffeineAddict]

Hi and welcome to forums!
Distro really doesn't matter, what matters is powerful hardware.

ollama is goold LLM and it provides multiple models, ranging from to be used by low end hardware toward really expensive one.
I've run it on Debian and it worked just fine with exception I had to use the low end LLM because my hardware isn't super computer.

To get ollama see install instructions here:

GitHub - ollama/ollama: Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models. - ollama/ollama

github.com

 

[surly_gerbil]

Great, thanks, I'll take a look at Debian.