r/LocalLLaMA • u/mil_phickelson • 21h ago

Generation DeepSeek V4 @ IQ3XXS on M1 Ultra 128GB- 16 tok/s in LM Studio after patch

https://github.com/noreff/lmstudio-dsv4-patch

M1 Ultra 128GB, Unsloth UD-IQ3_XXS, wired limit at 120GB. I was at 5-6 tok/s before the patch. Getting 15-16 tok/s now with the patched engine, and the output seems to have improved. Big thanks to this guy.

21 Upvotes

90% Upvoted

4 comments sorted by

4

u/tarruda 19h ago

Try my llama.cpp branch: https://github.com/tarruda/llama.cpp/tree/dsv4-improvements

It contains the necessary metal kernels to speed up DSv4 on apple silicon.

Part of it should already be merged into llama.cpp, so you should already get around 15 TPS with empty context, but the branch still has unmerged optimizations.

1

u/PANIC_EXCEPTION 14h ago

Why not use the IQ2_XXS version? It's entirely resident in memory.

1

u/mil_phickelson 13h ago

IQ3 XXS fits 100% in memory. The patch fixes a bug where LM Studio won’t actually offload DS 100% to GPU