For the LLM model the Alibaba’s Qwen 3.6 27B with different quantization are used to show the difference in token generation per second and memory…
Tag: llama-bench
llama-bench the Qwen3.6 27B and NVIDIA Tesla V100 SXM2 32GB
For the LLM model the Alibaba’s Qwen 3.6 27B with different quantization are used to show the difference in token generation per second and memory…
llama-bench the Qwen3.6 27B and NVIDIA RTX 3090
For the LLM model the Alibaba’s Qwen 3.6 27B with different quantization are used to show the difference in token generation per second and memory…
llama-bench the LLama 4 Scout 17B 16E and AMD EPYC 9554 CPU
This article shows the CPU-only inference with a modern server processor – AMD Epyc 9554. For the LLM model the Meta’s Llama 4 Scout 17B…
llama-bench the Phi-4 14B and AMD EPYC 9554 CPU
This article shows the CPU-only inference with a modern server processor – AMD Epyc 9554. For the LLM model the Microsoft’s Phi-4 14B with different…
llama-bench the Gemma 3 27B and AMD EPYC 9554 CPU
This article shows the CPU-only inference with a modern server processor – AMD Epyc 9554. For the LLM model the Google’s Gemma 3 27B Instruct…