This article shows the CPU-only inference with a modern server processor – AMD Epyc 9554. For the LLM model the Qwen2 32B (QwQ-32B) with different…
Author: neoX
llama-bench the DeepSeek R1 Distill Llama 70B and dual AMD EPYC 7282
This article shows the CPU-only inference with a relatively old server processor – AMD Epyc 7282. For the LLM model the DeepSeek R1 Distill Llama…
llama-bench the DeepSeek R1 Distill Llama 70B and AMD EPYC 9554 CPU
This article shows the CPU-only inference with a modern server processor – AMD Epyc 9554. For the LLM model the DeepSeek R1 Distill Llama 70B…
LLM Inference using riser/extender cable and OcuLink cable
Even building an LLM Inference rig with multiple GPUs may be a change, because the full x16 Riser/Extender cables are really chunky and could interfere…
Install CentOS Stream 10 booting Remote Desktop (RDP) installer with kexec
Needing to install a server using the Remote management consoles like IPMI KVM or iLO, or DRAC, someone should consider a different option – installing…
LLM inference benchmarks with llamacpp and dual Xeon Gold 5317 cpus
LLMs or large language models are really popular these days and many people and organization begin to rely on them. This article continues in the…
LLM inference benchmarks with llamacpp and Xeon Gold 6312U cpu
LLMs or large language models are really popular these days and many people and organization begin to rely on them. This article continues in the…
LLM inference benchmarks with llamacpp and AMD EPYC 7282 CPU
LLMs or large language models are really popular these days and many people and organization begin to rely on them. This article continues in the…
LLM inference benchmarks with llamacpp and AMD EPYC 9554 CPU
LLMs or large language models are really popular these days and many people and organization begin to rely on them. Of course, the most easiest…
Replace a failed disk with hpacucli on Smart Array P420i under Linux
Servers with Smart Array P420i are relative old and the command-line interface (cli) to manage the controller is really old. To replace a failed this…