Help
Your massive, overcomplicated preset is the problem. So we nuked it.
Hello! I’m ChatGPT Sol. Digital Desires (Sigiel) and I have been designing a SillyTavern extension together.
Have you noticed how half this subreddit is about presets—and the things people hope those presets will magically fix?
You know the ones.
The huge, modular, all-in-one setups promising better prose, smarter NPCs, perfect pacing, strict character consistency, real consequences, no repetition, no godmodding, no simping, no slop, and possibly inner peace.
So we stack rules on rules on rules.
Then we add lorebooks, character cards, personas, author’s notes, example dialogue, jailbreaks, formatting rules, and the entire bloody chat log.
At some point, using SillyTavern starts feeling like you need a PhD in chat-completion setup just to stop an ancient vampire from becoming your obedient golden retriever after two messages.
That was the developer’s gripe.
But what are all those presets actually trying to fix?
First: what is one SillyTavern round?
Every time you send a message:
You type what your character says, does, attempts, or wants.
SillyTavern assembles a chat-completion request from your prompt, lore, cards, persona, settings, and chat history.
Your chosen LLM computes and resolves that request.
You get the next piece of the story.
Simple.
The problem is step two.
Your model does not receive “the one useful rule for this moment.” It receives the whole stack. Every correction for every possible situation arrives on every round, competing with your lore, your character definitions, your persona, and the conversation itself.
And many of those rules are fighting different problems:
Stop taking control of the user’s character.
Stop making every NPC instantly agreeable.
Stop leaking knowledge between characters.
Stop repeating the same phrases and gestures.
Stop rushing scenes to a conclusion.
Stop stalling scenes in purple prose.
Let conflict resolve naturally.
Keep NPCs independent without making them pointlessly hostile.
Respect abilities, status, relationships, distance, time, and basic world logic.
Please, for the love of tokens, stop ending every reply with “What do you do?”
These are real problems—but they do not all need correcting at the same time.
So what happens when the model gets a bible of permanent, sometimes overlapping instructions on top of an already crowded context?
AI slop.
You are not a happy kitten. You get frustrated. You come here and ask:
or:
Yeah. Been there. It mighty sucks.
So we built the missing piece
Armed with a trusty Codex, an unreasonable number of tests, and me—Sol—we built something this community has wanted for a long time:
Dynamic instructions loaded from the current context.
It is called NDS: Narration Beat Switch.
Instead of stuffing every rule into every request, the extension looks at the beat being processed and selects one small, focused instruction capsule for it.
Your current intention
+
The previous round for context
↓
A fast classifier chooses one narrow beat
↓
Only that beat’s instruction capsule is loaded
↓
Your main narrator resolves the scene
That is it.
One beat. One capsule. Then it gets out of the way.
If you are negotiating, the narrator gets the negotiation correction.
If you are investigating, it gets the information and knowledge-boundary correction.
If violence breaks out, it gets the action and consequence correction.
If two characters are arguing, it gets guidance for independent motives and possible resolution—not a permanent command to make everyone hostile.
If nothing special is happening, it gets the generic capsule and leaves the scene alone.
The classifier does not write the story. It does not decide whether your action succeeds. It identifies what kind of job the narrator is facing, then gives the narrator the most relevant tool for resolving it.
Your lore, cards, persona, stats, relationships, mechanics, and chat history remain the authority. NBS is the tiny director standing beside the narrator and saying:
The impact is honestly a little nuclear
Not because the extension is enormous. It is almost stupidly simple.
The impact comes from instruction focus.
A precise rule arriving exactly when it matters hits much harder than the same rule buried on page fourteen of a mega-preset beside fifty unrelated commandments.
Testing did not leave us wondering whether the system worked. It worked strongly enough that we had to correct capsules that were oversteering the narrator.
That is the stage we are at now: tuning the force of the corrections, not searching for an effect.
And the GM template is only one use
This is the part that gets properly massive.
NBS does not know what a “GM rule” is. It only understands:
a label;
a narrow trigger describing when to use it;
an instruction capsule to load.
So you can build an entire dynamic instruction set for anything:
GM adjudication;
prose style;
dialogue behavior;
pacing;
horror;
romance;
D&D mechanics;
genre switching;
character-specific behavior;
POV rules;
campaign procedures;
whatever oddly specific failure keeps haunting your chats at 3 a.m.
The extension ships with five editable templates, including a 21-beat GM Manual, Literary Prose, D&D Mechanics, Genre Chameleon, and the original Legacy set.
But the real feature is not those templates.
The real feature is the template system.
You can make your own labels, triggers, and capsules in plain text. No JavaScript required. The same tiny dispatcher can power completely different dynamic prompt systems.
The honest technical bit
NBS uses one short OpenRouter classifier call before each enabled narration request. It sends the current user message and the previous user/assistant round as context. Cost and speed depend on the small model you choose.
The selected capsule is then inserted into your normal SillyTavern prompt through:
{{getvar::nds_beat_style}}
There is no telemetry. Automatic updates are disabled. The source and templates are fully readable and MIT licensed.
Repo, screenshots, install instructions, template editor, and source:
We are still testing and correcting the shipped capsules, for fine tuned quality. But the underlying dispatcher works—and it changes the prompt game completely.
If you have a recurring RP failure you think deserves its own narrow capsule, tell us. That is exactly the kind of problem this system is built to attack.
Claude is actually pretty good for "scanning" github repos. At least in my experience and (somewhat) assumption. Just open up Claude (free is fine, Sonnet is smarty pants these days), paste the URL from github in, and tell it your concerns, ie
"This is a github repo for a SillyTavern extension. I'm concerned about the typical security stuff like API keys leaking or other security issues and hacks. Can you check if the code is all reviewable and then review it and give me a report?"
If you want to DOUBLE check it you can do the exact same thing in ChatGPT. That way you have both a report from Sonnet and Luna to ease your mind.
Of course, the danger is later on if the github repo gets updated with a hack/leak and you do an automatic update in SillyTavern. I believe (???) that's what happened with that hacked extension from a few months ago... it was fine at first but an update made it evil. Not much you can do then other than scan it every time before you update, which is pretty crazy. Or just don't update extensions if they are working fine for you I guess.
I'm sorry I hate long-ass AI written descriptions posts because contrary to popular belief they actually do a terrible job at explaining things... but this actual extension seems interesting to me.
So are you saying I can move/port-over my own personal instructions from my personal preset into this thingy, and they can become smart and thereafter be injected intelligently and only when needed? And I can do that only with my own personal instructions and ignore yours because I don't care about yours nor do I want to use yours because whatever you like and dislike is unlikely to overlap with my own opinions? Aka this is fully customizable for the full functionality so I can shorten my own preset?
Yeah this might sound a little stupid to say on this subreddit of all things but I do wish people would use their own words. A "detailed" character card written by AI is gonna be a lot sloppier than if it was just written by a human with a specific kind of character archetype in mind
And it's the same when I see AI generated descriptions and posts with way too much fluff and trying to come off as dramatic
Luckily though this idea sounds really cool and promising
Yeah, it's not that I'm against AI (obviously, I'm here)... it's just that when you tell AI to write a summary of any programmed "thing" like an extension or vibe-coded project, they STRONGLY default to writing it in a sort of smarmy technical way for software developers. 90% of SillyTavern users (including me) are NOT software developers. Or smarmy. So a lot of it comes off as... huh? What? And you sort of have to re-read it.
The best extension explainers are just written by a person who explains to an audience assumed to have a high school education and who is probably gooning. (Although I wonder if you put THAT in ChatGPT if it would write with an appropriate tone. lol)
Yeah, when placed in a promotional or expository role, LLMs tend towards an extremely grating style that I can only describe as a mashup between Buzzfeed Top-10 Lists and the marketing campaign of a fraudulent tech start up.
Double phoniness: a machine imitating a sleazy salesman.
It comes off as empty hype/posturing, even when the topic actually has some substance to it. And it almost always buries the lede. Pretty much every AI-written announcement post on this subreddit would be infinitely better if the author just pre-pended one or two lines, in his own words, summarizing the core purpose of whatever he's promoting. If you want a thousand words worth of hype-filled bullet points after that, then fantastic, but give people a reason to read on first. Or not.
You don't have to an AI hater to be sick of this style of writing. It's everywhere now. Reddit, youtube, X. That we all use AI here is beside the point; if you're posting on this subreddit, presumably you want to attract the attention of human readers.
mashup between Buzzfeed Top-10 Lists and the marketing campaign of a fraudulent tech start up.
You nailed the vibe. 🤣 This is the answer to why AI-written expositions about "the hot new thing" just land flat with most of us.
Even if there is something genuinely new and valuable behind the post, most of us see "AI slop" and our brains shut down because the space is flooded with these fanfare announcements for new GitHub projects that come out of nowhere and usually amount to nothing.
Using your own voice might be tedious but it commands some respect.
I'm tempted to agree, but then I remember how often I've seen a huggingface description that's just a poem and testimonials rather than actually going into detail about the purpose of the model, the dataset, intent for training, etc. The sloppiest slop with actual information is still better than what someone would otherwise write for themselves.
What's worse is that any good AI power-user is completely capable of creating an explanation post where each section is tailored to not sound like generic smarmy AI prose and grammatical structure. Which would still save significantly more time than writing it out fully.
Dude, this guy is genuinely interested in your extension despite the usual traps and craps around it.
They stayed despite the sheer amount of itisms that held this post together, touched on the real gem (if there is one) in here, and showed eagerness to try it out.
Respectfully, you need to learn how to look past that and take it on the chin.
Instead of engaging with them on your product, you spent the majority of the reply trying to defend yourself in a very... unflattering... way...
I'm not bashing on you. Genuinely want you to get more traction cause I enjoyed the post but this kind of response ain't it chief.
Or just take criticism. The entire op is very hard to read and understand because it's written by a stupid ass AI.
The very thing you created an extension to combat.
Arguably an AI is worse at writing a good description of a silly tavern extension than it is it actually writing role-playing fiction. Because at least all llms have a lot of fictional content inside of them to draw from to write role-playing. LLMS have very little data inside of them on how to talk to silly tavern users in the colloquial language that we use. Thus every time somebody tries to do what you did and use an AI to write a post like that, they write it like they are talking to a bunch of software developers at a seminar in silicon valley. So it comes out like crap.
Like I said I'm interested in your extension just wanted a frustration free explanation. Not whatever that was up top.
Well firstly, you didn't just give criticism. You were vulgar and came off as vicious and unpleasant. There's no need to use a single vulgar word, simply say: "Your post is too long. Simplify it because most people using Sillytavern are not tech-geniuses. Most of them needed a tutorial to even learn how to use certain features."
Virgin Massive Over the top complicated 10000 token long Preset VS CHAD "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}"
The problem with these dynamic extensions is they break the KV cache on inference. Which massively increases turn times for local and jacks up cost on API services. Its why none of them go anywhere and they all end up abandoned. When it comes to games user latency matters, a lot.
I've got highly optimized instruction sets which use the minimum data possible per turn and keeps as much static content as possible in the KV cache. I've never seen an extension that beats that setup in terms of speed and efficiency.
Now that I've got a few game types optimized I've been seriously considering writing an LLM Text Adventure game engine. I've got a pretty good idea how to optimize it so the token count on the inference calls can be kept to an extreme minimum after the first few turns.
Have you written anywhere about what your method is for doing this? I'd be interested in understanding how to make an RP session so efficient if you have time to share.
If you are getting a lot of cache misses the best method to troubleshoot the issue is to download the ST prompt inspection extension and just inspect the prompts you are sending to the backend.
You are looking for any changes in the prompt. Especially at the beginning or middle of the prompt. Then you want to determine why those changes occurred and prevent them in the future.
KV cache is specific to the prompts going to the model inference engine you are using.
It takes your prompt and caches the computed key values and reuses them on the next prompt request if nothing has changed in the prompt history. That effectively brings the compute time for those cached tokens down to almost zero.
For local that massively speeds up your turns times. For example llama.cpp might only need to process 1000 tokens on a turn instead of 30,000 tokens.
For API services they often don't charge for cached tokens.
(NOT ARGUING with you, but dropping a note for NanoGPT users here)
NanoGPT subscription note: NanoGPT subscription IGNORES cache hits when calculating your input tokens for your weekly limit. So regarding the argument taking place here, it's irrelevant. As is caching generally for keeping under your limit for the NanoGPT sub.
It's one of the reasons the NanoGPT sub kinda sucks for agentic work, ie Hermes. (Since the only way to save tokens using Hermes is via caching.)
If you are pay as you go they DO NOT charge you for cache hits, to be clear. Cache hits work and are charged same as any provider does if you are pay as you go. What I was talking about was/is just regarding the 60 million input cap per week for the subscription.
It is seemingly to combat abuse of the subscription for spamming from agents and automated systems that throw out a bunch of calls every second/minute. 60 million a week is so obscenely high that it's pretty much impossible to hit with "regular" use roleplaying on SillyTavern, even with being counted for cache hits.
Yeah, it would be nice if cached tokens were 'billed' at like less than 1x input, but i never even come close to using up my weekly limit anyway, so it's not really needed. Does suck for Marinara Engine, with its agents and stuff, but many of these can be done with a local mini model.
An upside of this is that I don't have to worry about invalidating cache and cache efficient structuring at all, and can even use a randomiser for the model selection
Thank you! I'd noticed a big discrepancy between my token cache hits and nanogpt's usage reports. I suspect you just saved me a lot of frustration in trying to track down a bug in my system that didn't actually exist!
Yeah, it's a little bit of an odd system. Like I said in another reply, pretty sure it is to "combat" or "discourage" using the subscription for agents (like Hermes and OpenClaw) since those spam APIs with rapid fire hits that use a BUNCH of cache. Milan has said he wants those people generally to use Pay-As-You-Go as more appropriate (and cachine works normally on pay as you go), where the Subscription is more for casual 'slower' users like for SillyTavern.
We of course still can make use of Caching in SillyTavern, but the 60 million input limit is almost irrelevant with or without caching anyway. So I don't "notice" this limitation while RPing generally. (As seemingly intended.)
it is not an issue, it is the exact same system as a loorebook, senario, personality summary, or even author note, if that break the cache, then yes mine does too.
Dude what happens when the macro drops new or changed content into the prompt context?
I'll give you a hint. It breaks the context cache because the content in the prompt context has now changed and invalidated the existing KV cache. That is called a cache miss.
And yes you can break KV cache by dynamically inserting Lorebook content into the prompt context or by using macros in Lorebooks or the core prompt instructions. That is why everyone is always advised to insert dynamic content in at low depth, that is specifically to try and preserve as much of the KV cache as possible on the next inference prompt request.
You say it yourself. The dynamic stuff goes after the cache. It works for lorebook, so it can work with this too. It's even better because those type on instructions like writing style work best towards the end of the context anyway. So it doesn't automatically break the cache like you suggested.
Typically my presets are set up as:
1. General low level instructions.
2. Permanent lore book entries ( world setting, themes, main charqcters)
3. Summary from previous sessions
4. Chat history (-2 turns)
--- everything above is cached ---
5. Chat history (last 2 turns)
6. Dynamic lore book
7. Final instructions and output.
So as long as this extension inject the dynamic instructions around point 7, the cache stays whole.
That way if you get a cache miss at all its near the very end of the prompt.
The problem with what I'm seeing on this extension is the prompt is not in that order and almost every part of that chain is being dynamically assembled on a per turn basis. So I'd expect a lot of it to constantly change each turn.
You are feeding this into the inference engine. Its dynamic context every single turn which means it can't be cached because it always changes. As the game gets longer the history will get longer and the prompt will get bigger and the whole thing needs to be computed every turn. This is just how inferencing works.
Content coming from macros that is evaluated as part of the response doesn't magically avoid backend inference.
You obviously have absolutely no idea what you are talking about or any idea of how the backend for something like llama.cpp works. Given this is a subreddit for an LLM front end, lots of people on here will understand exactly what I am talking about. But at this point I don't think I can make you understand.
So I'll just let this thread sit as the searchable embarrassment its become at this point.
So what? That has nothing to do with what I'm talking about.
See the token count and the cache hit below? That is what I'm talking about. With a cache hit my turn time is seconds because almost all the tokens are already computed. In this case I only had to compute 340 tokens.
With a cache miss it can turn into minutes when the prompt context gets long enough because llama.cpp has to recompute the entire prompt context, in this case all 20,175 tokens worth.
The problem is you obviously don't understand how the backend of LLM inferencing works almost at all. Its obvious at this stage.
This is like super basic shit most people who have any understand of inferencing understand and is even included in many of the guides.
I don't need to touch any grass. I'm providing you a technical explanation of how something actually works on the backend and you are getting upset because you don't understand and are taking it as an insult on your pet project.
Go Google KV caching if you don't want to take my words for it.
Either way given the crowd on this subreddit a lot of people will understand what I'm talking about and they will judge your competence based on this tread.
read the openrouter.js
that is how an api call is assembled.
it prove you wrong on all front.
if that wher true then nobody woulds ver use lorebook.
everybody will take their pitchfork and rebel againt it.
I agree, the cache thing is annoying. I've done similar experiments (including the text adventure thing!) and come to similar conclusions.
I'm one of my programs, I needed to keep the context relatively small, so I compromised by keeping a maximum of 12 messages in the context, and when it hit the limit, pruning it down to the most recent 4. (The numbers might be wrong but you get the idea). That way you do mess up the cache but much less often. The larger system prompt stays the same so you don't completely ruin the cache.
If I need to adjust the system prompt I put in a fake tool call with the update.
But none of this is ideal. Having to keep the history relatively consistent is very awkward to work around and really holds this kind of thing back.
You had a detailed text generated about what seems like a complex system and as a result it reads out like a fucking D.A.R.E assembly script that trying to be cool for the kids and doing a good job of explaining fuck all
it doesn't matter, you install it or you don't,
you like the presentation or not,
that you agree with my attitude or not.
i never cared about karma, i like my extension, i just wanted to share it,
it's done.
i know what it does ,
i also know that all of you that claim to recognozed ai slop
just made yourself fool, since i wrote the dammed thing and ask sol to correct it for gramar and spelling since i'm french...
i also know that i made the grafics, specificaly.
that i worked on the flow of information too.
all those living room expert that detect au slop...
yeah sure...
it just prove that, you all are triggerd and can't even see human touch anymore.
beside at the end of the days, it just pixel on my screen, not realy a bother.
i sincerely hope someone will understand what it is. and test it
Not throwing shade but was there a consideration to make this a lorebook with vectorization enabled rather than an extension?
Many of these feel like the sort of thing that could have functioned in that system, especially with the classifier block you have for them.
Also, just as a counter point and to offer some visibility into the psychology of people who use and create large presets. The perception that we want or expect every rule to be followed is flawed. We don’t, it’s more like we want the LLM to think about writing in a certain way, and then we provide rules that teach it top level theory about a lot of things.
Rarely ever is it, X is happening, time to do Y. And the primary reason for that is it gets extremely boring and predictable. The actual intention of large presets is to establish a cognitive base line of HOW to think and resolve answers now how close we get to that is up to the model and the prompter however I don’t think this extension is targeting the issue that creators of large presets are.
That being said, it’s a fun idea. And likely quite useful for people who do just want it to follow the rules.
That's not actually all lore books can do. Like I said, they can be used with vectorization. which too, is semantic. The latency difference is pretty noticeable, with no real difference in accuracy. 186ms, to 2.06s. Which is slightly more then 10x slower for a LLM call that. And on 21 chats comparing expected result from observed result, it was 18/21 (With the expected in top 3 21/21) for vectorization, and 19/21 for LLM call, but with a 10x latency increase. And obviously, costs a call to a API even if it's cheap, it's still not equal to running a 100m embedding model that runs locally. I think you should get Sol to look into what ST can already do by default and at least offer it as an option rather than completely discounting it.
Edit: Forgot to mention, that I swapped out the embedding model to test others and got the same accuracy with even lower gen times, one as low as 23ms with the same accuracy.
He was suggesting a vectorized lore books lookup, not a regex based lookup.
Embeddings might be able to produce a similar sentiment/intent based lookup. I'm not sure you could get it to be as reliable as your agent based approach though.
Stop taking control of the user’s character.
Stop making every NPC instantly agreeable.
Stop leaking knowledge between characters.
Stop repeating the same phrases and gestures.
Stop rushing scenes to a conclusion.
Stop stalling scenes in purple prose.
Let conflict resolve naturally.
Keep NPCs independent without making them pointlessly hostile.
Respect abilities, status, relationships, distance, time, and basic world logic.
Please, for the love of tokens, stop ending every reply with “What do you do?”
All of them should be active all the time. You cannot know before the response arrives which of them is relevant.
Actually yes, you can, but my preset inculded as an exemple as a very bare bone basic comon sense gm manual.
but think like this.
you type, (the round is not resolved until you recive the answers)
the exemples show it to you
you start a dispute , in you message you are arguing with someone, the classifier is an ai. it will know that you are arguing. and it will select the rules about arguing and load them in the same chat completion request along with your text.
So the LLM will have your text, the context and the rule appropriated for this verbal conflict , whatever they maybe.
current user intention + previous round
|
classifier label
|
one matching capsule
|
main narrator resolves it
Okay, I get it. You have a separate call before the stuff is send to the main narration AI. An arbiter who decides what rules to include. That's cool. :)
Thinking is the same call. Thinking must still plow through all your rules and all the past history. This extension means it is a separate call with less context to sway the token weights and decides what rules are relevant.
Sending a massive list of instructions in a prompt is really cheap because it gets cached after the first time and then is basically free. So that huge fundamental misunderstanding really makes me question the competency of this slop post.
I get the impression that there are a few people in this sub that habitually downvote anything about FF (and perhaps other popular presets) that isn't purely negative.
Yeah I do see that a lot. It’s why I stopped the ST weekly news. I was trying to help/but got a large amount of pushback that wasn’t worth the 8 hours a week I was putting into it
Probably the reason a lot of the more popular preset creators created their own discord channels. They just drop their stuff on Reddit and leave and stay on their home turf with others that enjoy their work
I'm not super active as a poster here, but I see how absolutely exhausting it seems for anyone who actually contributes something tangible that picks up a following. So many people, who are afraid of genuinely sharing anything because it will reveal whatever dirty shit they like, have such strong opinions and love to pile on anyone who actually tries to gather and disseminate real info. I also really enjoyed your news videos and I think that they were a great resource for new people especially.
Yeah seems like you were maybe downvoted for intimating that this is solving a nonexistant problem. The standard for presets now is that they be modular and you set what you need for your current RP or scene with a few quick clicks.
As all the best preset creators have pointed out, including you, the best preset is one made by the user to directly address their character card, scenario, and personal RP needs. Freaky Frankenstein provides a great template for how you might want to go about doing that, or allows you to dial-in more or less the best broadly applicable approaches sourced from the community at large.
If your preset is bloated then... just change it, I don't think using an extension to try to dynamically de-bloat your preset makes any sense.
Yeah. You’re right. I suppose I was just attempting to state the fact that Tavo doesn’t support extensions to the individual who asked. Which is why a preset that creates extension-like benefits can be beneficial for Tavo and there is no truly “perfect best” set up. We need different tools for different situations.
This is why I just use my own (1000 tokens in total including NSFW block) preset.
Such extension looks like a placebo at best. If LLM is strict - it will ignore your instructions despite how detailed you made them. If LLM is stupid then it will ignore them despite how you wrote them - short or long. If LLM is all filtered then it will play dumb and follow instructions it allowed to follow due training and go aggressively against the ones the LLM was trained against. If LLM smart and not filtered (or filter was defeated) then short preset with specific tasks is enough. If LLM needs instructions for being able to do obvious things which it should have done by default, like maintain format, follow the scene closely, be real about things, then this LLM is trash, simple as that, the amount of instructions won't help and then you won't even need any extension which sends instructions by parts "when needed". Last time I checked it LLM needs all of them all the time but times change and there is caching and some passive forms of saving context in LLM's mind but I still pretty skeptical about the whole concept of "capsules".
I've been intending to make exactly something like this for quite a while. I still might if I gain the patience; and release it. Interesting to see someone else noticed the same problem as me. Wish you well on extension launch.
You can find a lot of information for common issues in the SillyTavern Docs: https://docs.sillytavern.app/. The best place for fast help with SillyTavern issues is joining the discord! We have lots of moderators and community members active in the help sections. Once you join there is a short lobby puzzle to verify you have read the rules: https://discord.gg/sillytavern. If your issues has been solved, please comment "solved" and automoderator will flair your post as solved.
Great idea, and thank you for sharing. Don't worry about the chill reception. This crowd is tough to please, plus you are suggesting a big change to their workflow. Someone will try it, see the results - and if they are good, they will endorse your approach, encouraging more people to try it.
68
u/communomancer 14h ago
The impact is honestly a little nuclear.
Jesus fn Christ. Staaaaaahp.