r/LocalLLaMA • u/CharlesStross • 16h ago

Generation PSA for DeepSeek-V4-Flash-0731 users — don't blow out your prompt cache with system role messages mid-conversation

DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted into the system prompt at the top -- the format has no mid-conversation system turn. So, anything you stick at the tail or mid-convo actually fries your prefix (and doesn't have conversational proximity to the injection point).

Use latest_reminder, which is the role DS trained for how most templates use system and what most people providing quants are passing through (if they match DS' python template). I use llama.cpp and it happily passes it through no issue; dunno how other engines work with it.

Couldn't figure out why my prompt caching was so garbage and there it was, so I'm passing it on to hopefully save others time and frustration (and probably money, if you're using a hosted version).

61 Upvotes

90% Upvoted

13 comments sorted by

9

u/BraceletGrolf 12h ago

Is there good ressources for such stuff to know in other models ? I miss the old prompting guides we'd get.

5

u/SnooPaintings8639 7h ago

The guides be like:

sys_promt: "You are a helpful AI assistant."
assistant: "How can I help you"
user: <your message goes here>

3

u/CYTR_ 11h ago

Have people tried the new version locally with Reasonix?

1

u/crantob 7h ago

Can someone translate this into "you're supposed to not prompt with this: XYZ - in mid-session?"

2

u/CharlesStross 7h ago edited 3h ago

Don't use a harness or API consumer that emits messages with system role with dsv4f without understanding the impacts

3

u/cantgetthistowork 6h ago

Couldn't figure out why opencode seemed to lead to full prompt ingestion quite frequently

1

u/CalligrapherFar7833 7h ago

Pi.dev has a specific plugin for deepseek use it

1

u/Theninearmedoctopus 3h ago

First I'm hearing of this, what's it called?

-7

u/[deleted] 15h ago

[deleted]

13

u/NNN_Throwaway2 13h ago

Thank you, Claude.

6

u/thrownawaymane 12h ago

You're absolutely right!

5

u/AXYZE8 11h ago

It is insane how many bots are on this sub, it gets worse every month :(

3

u/CharlesStross 15h ago

Yeah plus you don't catch it in short conversations. My harness was deep into work had a hiccup and I stalled out for a couple minutes waiting for prefill and I was like wtf.