News [Phoronix] KTransformers Adds AVX2 MoE Support For Viable Performance On CPUs Without AMX/AVX-512

[LinuxBot]

KTransformers 0.5.3 released today for this framework for efficient inferencing and fine-tuning of large language models (LLMs) with a focus on CPU-GPU heterogeneous computing. With this release, KTransformers 0.5.3 is now more applicable for CPUs lacking Advanced Matrix Extensions (AMX) and AVX-512 in now providing some AVX2-only kernels too...

Source: https://www.phoronix.com/news/KTransformers-0.5.3

Aggregated via Linux News