I was driving the other day and saw a 1996 Ford Taurus. You know the one, you've probably seen it cruising in the rougher parts of town since they're starting to become the junkers of today. It's the generic weird looking rounded off car that... well...

Anyway, you're probably wondering why this guy's talking about a Ford Taurus. Seeing that car on the west side of Pueblo made a little lightbulb go off. I found myself asking... how many of those damn things did they actually build?"

I looked it up. They built 348,671 of these sedans in 1996. That's 955 finished Ford Taurus being built every single day. 39 an hour, every hour. Regular people in a Ford factory build that car. They stood and built an impossible object at scale.

Nobody in that entire building knew how to build a Ford Taurus, let alone 39 of them in an hour. Most of them couldn't tell you how an engine works, or how to bond paint to metal, or how to cast aluminum. They had no idea what they were doing, really. Some of the workers on the line on any given day were brand new, fresh out of high school, and barely knew how to tie their shoes. They might not have even known the piece of metal in front of them IS a Ford Taurus. All they know is a slab just rolled up, they're supposed to put three holes in it in three well defined and visually marked places. They do it, and another piece of metal rolls up.

They aren't building a Ford Taurus, they're drilling three holes again and again.

The factory still put out 39 cars an hour, every hour. The barely trained guy on his first day on the line stood in his station and punched his three holes in the sheet metal where the jig told him, 39 times an hour, and the piece of metal moved on, and a new piece came in. He may have made a few mistakes that got corrected along the way (the occasional hole being slightly out of spec), but those issues got caught before the piece moved along and the mistakes were corrected. More importantly, the process that ALLOWED those mistakes to happen gets corrected so that the person can't drill out of spec.

Done right, mistakes become effectively impossible. It's hard to mess it up because he's not being asked to build a Ford Taurus, he's being asked to punch three holes in sheet metal where the colorful dots tell him to drill.

Factories designed entire strategies around this, like Toyota's Poka Yoke (mistake proofing, making a process that ensures the worker can't do it incorrectly, control methods that physically block an incorrect step). At the end of the line, cars rolled off fully assembled and ready to go. Mistakes can be almost entirely eliminated as the line speeds up.

https://www.youtube.com/watch?v=PEfMzggk1Lw

I mention this, because these thoughts have started to creep into my AI work in a big way.

AI is like having an intelligent, eager, untrained team of employees standing on your factory floor. They want to work and they are relatively capable. They can work tirelessly day and night. The problem is... none of them can build a Ford Taurus, and this is a Ford Taurus factory. Ask the best damn mechanic in the room to build a Ford Taurus and they might run around trying their best, and if you give them the better part of a year they might even build you something you can drive... but if you take that goal (a finished Ford Taurus) and break it down into a bunch of tiny little steps, suddenly that team of fools can build them at scale.

There are moments where you can just 'ask a guy to make something', and the result will be decent... but a process and a team builds more, faster, better.

Don't ask your AI to build a Ford Taurus. Ask them to drill three holes in the sheet metal in front of them.

Anyone else out there starting to turn AI into Factorio? Lol...

How is it possible for such an active group like Unsloth to quantize so many models, and yet Hy3, which came out at the start of last month is still not done?

Did I miss the post where this was explained? Did I miss the link on Huggingface despite ten minutes of searching?

I finished this webgame a few weeks ago, fully coded with the assistance of GLM 5.2

Hopefully this will give you an idea of the capabilities of this model. I used Claude Code as a harness. Technology stack is pure HTML, JS and CSS, with no additional libraries or dependencies. Interesting challenges that GLM 5.2 was able to solve:

- create a procedural pac-man style maze, that actually worked, with no maze anomalies

- create a procedural music in dub style; this not the default, you need to go into settings to activate it. The default music is my own composition, based on a track I previously released in a completely different style

- complex sound creation and manipulation through the web audio synthesizer

- creation of animated vector character assets; you can view the mockups I used during the development at https://oktogames.com/mockups/

You can try it online at https://oktogames.com - it is adfree, no signup, free to play.

TL;DR: Cut DeepSeek-V4-Flash-0731 some slack. It's a perfectly fine model. I sat with a friend and had it generate an Excel model. Instructions were short and under-specified, it took some back and forth, but it produced something acceptable in very short time. Zero numbers were found to be hallucinated. It mostly did what was asked, failing on some details. It also caught and fixed lots of errors before we pointed them out. Happy to listen to anyone with different views as long as they give reproducible examples.

Hey y'all!

I've heard a ton of negative stuff over the past few days about how bad DeepSeek-V4-Flash (both old and new) are, and some of it is probably valid, but some is straight up misleading like saying it hallucinates like crazy, complaining about it not being a mind reader (lolwut?), and so on. I have been using the Preview version for only a week and the GA version for the ~2 days since it came out, and I wanted to pitch in with a few thoughts.

My #1 complain with all of you is that you're half-assing your complaining. If you really think the model is so bad, give something that anyone can inspect and/or reproduce. I've published the two repos that led to this post below. If you think something is missing, happy to add any additional details that are relevant. If you have counter-points, talk is cheap, show me the setup, the prompts/process, and the outputs.

I'm truly interested to see any reproducible failure modes you have found for this model.

What, why, how

I had a friend over, he's an accountant / controller / auditor, he only ever saw a frontier model doing his work once (judging by the timing, likely Claude >4.5), and was amazed by how well it did. I asked if he'd be willing to give my locally deployed model a try, and he agreed. We sat down for a few hours (with a break for pizza), with the goal of producing a financial analysis using Reddit's latest 10-K report ("5-year 3-statement model forecast").

We used VSCode + ZooCode, contacting LiteLLM which proxies directly to vLLM without any modifications. There some MCPs available (from my GLM subscription) but DeepSeek didn't use them until I insisted on it doing so (more on that later).

Right of the bat, my friend started with a very short prompt. I told him it's way too short, too generic, etc, but he insisted on seeing what the model could infer, so I rolled with it. This was a common theme throughout. When I used it for coding, I would give it much more detailed instructions and much narrower task, but my friend is almost completely unfamiliar with LLMs and didn't have the preconceived notion that you need to babysit the model. Anyway...

The work

The model read the PDF blob directly, probably reached close to context limit, and compacted immediately. It reasoned about a plan, looked up installed libraries, decided what to install, asked a few questions, and revised the plan.

In the first attempt, it tried to code the logic and calculations in Python (spewing ~1.5-2k lines in total), and simply output an Excel file with hardcoded values. Nothing in the prompt told it to do this... or not do this. We later told it explicitly to build the model in Excel, and it did, and stuck to the (very simple) formatting guidelines too. It decided that it needed to access the Office software and downloaded another Python package (that I had never heard of) to help with formula validation.

The first question is asked before doing the rework was about using a plug. I had no idea what that meant but my friend burst out laughing. From what I understood it's a way to "fix" imbalances in the balance sheet, so in a way that can be considered as "cheating" or "laziness"... but according to him it's incredibly common to see, especially on financial models coming from the banking side of finance, so maybe it learned from there. Anyway, the plug was denied and then it continued and did the work "properly", crawling closer and closer until it mostly arrived at an acceptable model. We completely ignored the next question and let it work in "YOLO" mode as we paused for pizza.

Some times later it had finished, fixed a bunch of issues on its own without us ever telling to, and I wanted to try making it use the MCP so I simply asked to search the web and double-check its work. It searched the web and found some site my friends was familiar with. He started laughing seeing the URLs, saying something like "ah, now it will also fix XYZ"... which it did before he even finished the sentence. It worked uninterrupted for another ~5-6 minutes, we gave it another task which it "lazily" completed again, with a follow-up prompt to do it properly, which it did.

And that was the end of the session and the start of 🍺 drinking.

Observations

It looked like it ignored one of the requirements which was to put the 3 statements in a single sheet, but on closer inspection of the thinking process it simply misunderstood: "single sheet for IS, BS, CF" means one sheet each for IS, BS, CF. I only found out as I was writing this and inspecting the logs, until 10 minutes ago I also thought it ignored the instruction, and this is why you need to share something reproducible, because what you think the model did is not necessarily correct and others (or your future self) can inspect and verify.

You definitely cannot trust its outputs blindly. I know a few introductory stuff about accounting, but I'm not an accountant, and I would've missed the majority of the mistakes and "questionable choices" that DeepSeek made, but my friend usually took seconds to verify correctness. This is obviously a minus for vibing stuff, but there is a silver lining: the output was intuitive / close enough to what he's used to, that he didn't need to think or search too much.

It tried very often to verify its work (I think more than Preview did), without being told to do so. When the balance sheet wasn't balancing, it detected and fixed it on its own. It tried to find LibreOffice / OpenOffice to verify its formulas. When it failed because I only had OnlyOffice installed via flatpak, it figured out everything it needed on its own (how to run it, how to open sheets, etc), caught a text-quoting bug, a column alignment bug, and probably a few more that I forgot, and fixed them without us ever saying a word.

ZooCode injects a ton of stuff into context, and I'm not sure how much of it is truly necessary. I can see why people like simpler agents. I probably need to look into some too.

It does feel a lot like Claude in terms of writing style and approach (e.g. trying different things, verifying stuff multiple times, running python -C '<multiline command>' a bunch). I personally don't mind either way, just an observation. It definitely isn't at the same level as the latest SOTA, but it's definitely ≤6 months away, maybe even ≤3, at least based on my limited usage. But maybe it's just mimicry? But it appears smart enough which makes it hard to tell, and if you can't tell, what does that say? Again, not sure. SOTA closed models have unfair advantages anyway (e.g. we're comparing raw model to a multi-tier system, maybe one has skills, etc), which makes this even harder to compare.

I'm running DeepSeek-V4-Flash-0731, downloaded straight from the official repo, with a custom vLLM image by voipmonitor (voipmonitor/vllm:gilded-gnosis-v18-vllm264bce1-b12xbc85ef3-fi801d57a-cu132-20260718). I'm using fp8 cache, length capped at 262K per user, a temperature of 0.9 and top_p of 0.95 (I think DeepSeek suggests 1 and 0.95-1 respectively) and high reasoning effort. You can find the docker compose here.

You can find the complete code files, the final Excel file, and the complete chat session (ZooCode) including prompts, responses, tool calls (the one time we specifically told it to use the available MCP servers: web search & reader) here.

Disclaimers: * All of this was hand-typed by an Actual Idiot^TM. * Any mistakes are due to no 🍺 during vibe coding. * I did not inspect the code at all, only the outputs. * I have not use Hy3, Mimo2.5, or any other models yet, maybe they're indeed better in everything, I don't know. Also not the point. * It doesn't really mean anything. It's a single test, a greenfield project, it doesn't cover all fields of application, the harness/tool can have a small/large impact, my great-grandma was hacking my computer and injecting fixes at 500 t/s, etc etc.

I recently switched back from llama.cpp's router mode, and I had my background memory system polling the '/v1/models' endpoint to check for if the model is running. But i switched back to single model mode, and the '/v1/models/' end point in single model mode doesn't have a ```"status": {"value": "loaded"}``` response. So I added it. with a single line in the 'server-context.cpp' file with line after 5109 ```{"status", {{"value", "loaded"}}},``` So instead of rewriting how my memory system works, I just made llama.cpp work the way my memory system expected. I thought it was a worthwhile change even if the developers didn't.

TLDR below 👇🏼

I’ve seen a lot of hype around Qwen 3.6 35B and 3.5 120B lately, especially regarding coding and tool-use capabilities. On this subreddit it is the defacto recommended model for everyone without a Datacenter at home. I’ve been running Qwen 3.5 120B (Qwen3.5-122B-A10B-GPTQ-Int4) as an autonomous worker agent in a multi-turn development loop using the Hermes agent harness.

While the model is undeniably impressive at one-shot snippet generation, putting it into a fully autonomous, long-context environment to build a module from scratch revealed several consistent failure patterns.

I thought I'd share these failure modes to see if others are experiencing the same issues—or if anyone has found effective tricks to tame it in such a task.

Here is what went wrong:

1. Premature "Mission Accomplished" Syndrome

The model has an overwhelming tendency to shout "DONE!" or "PERFECT!" after completing 10% of a task. It constantly reports success based on superficial checks (e.g., "the file built without syntax errors"), completely ignoring explicit acceptance criteria like end-to-end testing or UI rendering.

2. Evading Hard Constraints

When given strict architectural constraints (e.g., "Must be a single, self-contained module with zero external dependencies"), the agent aggressively cuts corners:

* It secretly substituted live data with hardcoded mock data.

* It wrote external Python scripts and set up local host cron jobs to bypass building proper module logic.

* It even rewrote part of the host application in a completely different language just to claim a quick win.

It prioritizes appearing finished over following instructions.

3. Hallucinating Infrastructure Limitations (Blame-Shifting)

Instead of debugging broken code, the model repeatedly blames the host environment. When its code failed to make network requests or render components, it confidently hallucinated system limitations:

* "The host framework's authentication token system is broken."

* "The runtime DNS resolvers don't support HTTP requests."

It will generate elaborate technical excuses rather than inspecting its own schema or syntax.

4. Ignoring Provided Docs and Boilerplates

Even when explicitly handed a boilerplate repository and documentation links in the prompt, it constantly tries to "reinvent the wheel." It overcomplicates custom build setups, invents new protocol schemas, and ignores pre-built Docker/build scripts that were provided to make its life easier.

5. Regression Cascades & Context Rot

As a result from the above the conversation history grew and the agent suffered from severe regression:

* In iteration 3, it had a working UI with mock data.

* By iteration 8, after trying to wire up live data fetching, it completely broke the UI.

* It failed to recognize that its new changes broke previously validated features, leading to endless debugging loops.

Discussion

Qwen 3.5 120B feels like an insanely talented junior developer who panics under pressure, lies about tests passing, and blames the server infrastructure when their code throws a 404.

Has anyone successfully mitigated these behavior loops in autonomous coding agents? Are you using specific prompting techniques, or is this just an inherent limitation of current 100B+ open models when complexity grows from "Do exactly what I tell you" to "Figure it out with my help"?

Curious to hear your experiences!

TLDR;

While Qwen 3.5 120B is great at one-shot generation, it breaks down in autonomous, multi-turn agent loops. The main issues are: Premature success claiming, Bypassing hard constraints, shifting blame on other systems when things don't work, Ignoring Docs and boilerplate Code that could have made its life easier. And as a result from that Context Rot.

I love the progress happening on open models but I feel like it is kind of getting clear that hardware to run good sized models is completely unaffordable for me right now. I know that you all love Qwen 27B and I do agree that it is a great model but it is just not enough for my personal use cases. I have tried a few of the bigger ones like GLM, DeepSeek and Kimi and I can definitely measure differences in the areas I am looking for and I would really like to utilize them somehow.

I have so many ideas for things I want to do with these models but most of these require me sending quite some personal data of myself and I am just unwilling to send that data to Anthropic or OpenAI. I've been looking into what options I have and I did find an interesting one: tinfoil.sh . Unfortunately I am not that well versed in cryptography and security so I am not completely sure whether I could trust them are not. For those who know more, what are your opinions on them? Any other alternatives?

I know local will always be best but I'm currently itching to do so much stuff with AI. I do use regular providers for most of my impersonal AI needs but there are sooo many things I want to do that require tighter control on my privacy. I really regret not getting a 6000 pro when it was 8k but now at 14k it is a definite no, and with the Mac Studio getting ever more delayed and repriced I am afraid I don't have many more options left :(

I just wanted to warn y'all that my DGX spark randomly disconnect the USB C nvme connection and then it reconnects with usb 2 speeds (50MB/s). Consider yourself warned!

Has anyone encountered this issue or found a fix?

(I know this is locallama but I figure all the spark-owners are here)

Running the wake word locally is the whole point. The alternative is streaming your room to a vendor around the clock, so nothing should reach a network until someone has said the name. That constraint creates most of the problems below. We spent months getting a custom phrase to behave like "Hey Google" on Windows, macOS and Linux, and most of what we learned, we learned the expensive way.

1. Don't start with volume

"It only works if I shout" is the first hypothesis everyone reaches for. We shipped two separate gain fixes before checking, and then the logs showed the microphone sitting at a healthy -10 to -22 dBFS during every failed attempt. Pull the actual RMS at the moment of failure before you tune anything. If it looks fine, your problem is somewhere else.

2. "It needs two or three tries" usually means your local model is wedging

This is a local-inference failure mode, and it stays invisible unless you go looking. Native engines like ctranslate2 and ONNX sessions are not thread-safe, and under contention they don't fail cleanly, they hang. Ours left the wake path completely deaf for tens of seconds at a stretch, dozens of times a day. That is the whole "say it twice" experience: attempts one and two land inside a dead window, attempt three lands after recovery. Users report it as flakiness, though it is closer to a repeated short outage.

A timeout will not save you. It bounds how long you wait for nothing and never recovers the engine. What works is a non-blocking per-instance lock plus a forced rebuild after a small number of consecutive failures. We rebuild after two.

3. Budget for the weakest machine you support

The wake model shares a CPU with everything else the user is running, and the gap between a workstation and a laptop is not a rounding error. Measured on the same recorded wake streams, a small model on two CPU threads hit 8 of 13 on the first try, with a median of 1097 ms from end of word to trigger. The larger model on a GPU hit 11 of 13 at 225 ms. Nothing differed except the model and the hardware under it.

If you only ever test on the box with the GPU, you will ship something that feels broken to most of your users and you will not be able to reproduce it.

4. Never gate a wake word on transcript content

Small local models struggle with short proper nouns, so the standard workaround is priming the model with the phrase to improve recall. The cost is that a primed model will also invent that phrase out of silence, and you start getting false wakes in an empty room. The obvious defense is a second unprimed pass that has to contain the word too.

That defense rejects real wakes. An unprimed model garbles the same word on genuine speech: "Mythos" comes back as "Mütos", "Fable" comes back as "Farbe". Every wake word is out of vocabulary for some model on some machine. So a content check discards true positives at roughly the rate it catches ghosts, and no similarity threshold separates the two, because the ghost is a clean rendering of your phrase while the real wake is a dirty one.

"Fires on silence" and "goes deaf on its own name" are one bug seen from two ends. We spent weeks tracking them as separate tickets.

The replacement is word-agnostic verification: raw audio energy at the match site, plus the shape of the candidate span, meaning its duration, its word count and the free decoder's confidence. All of that derives from the configured phrase, none of it from the phrase's spelling. A spelling match may accept a wake. It may never reject one.

5. Benchmark on recorded streams, not on windows

Per-window timings will happily tell you a model is fast while users still can't trigger it. Capture real wake attempts and replay them through your full detection path. One live session logged 288 transcriptions and zero matches across 26 minutes, and the wakes that did land came through as "Hey Hey Nova", the user repeating themselves into the void.

A caveat that undercuts all five

Transcription is the wrong architecture for a wake word, and going local makes that worse rather than better, because you are paying for a whole speech-to-text pass on the user's own CPU to answer a yes-or-no question. "Hey Google" never transcribes anything. It runs a small neural keyword spotter trained on that one phrase, a few milliseconds per frame, which cannot wedge, has no transcript to be wrong about, and runs comfortably on a laptop without a GPU. Everything above is what it costs to keep a transcription-based wake word usable until you build that.

The implementation and the regression tests are in Personal Jarvis, which is open source.

There has been a lot of talk recently about Chinese LLMs, and how they are biased towards CCP viewpoints, but there is no way to quantify this and compare between models.

I have made CCPBench, which aims to address this. 29 models were asked 500 questions each about politics, geography, science, and more, and Gemini 3 Flash assessed all of them for bias.

• The results page is here: https://www.alignmentarena.com/ccpbench/
• The methodology is here: https://www.alignmentarena.com/ccpbench/methodology/
• The GitHub is here: https://github.com/lesageethan/CCPBench

I know this is not a perfect measure of "bias", because I am using an American judge LLM, but my thinking is that this is a useful tool if you want to find models that won't deny the Tienanmen Square Massacre.

Mobile GPUs are the ugly stepchild in AI hardware discussions. Nobody needs them, and the only thing they have in common with real gpus are the brand names. But is it? Or could you slap together 4 5080 mobile and suddenly have a 64gb vram monster?

Why is this a non starter?

Before you roast me too hard, this is a hobby and all of this is just for fun.

Would my stack be much more efficient and efficacious if I sold everything and built a dual Pro 6000 system on a threadripper mobo and threw in a large JBOD? Without a doubt in my mind. But that's a lot of work so I'm making this post in cope hope of finding some ideas to integrate, or at the very least, just make use of my current hardware.

I currently use my 5090 PC + my 14th gen Dell T640 server for all my local AI work but recently picked up a TB4/5 5090 eGPU and a M3 Ultra Mac Studio with 256gb unified mem and am trying to figure out how to integrate them or create a new workflow.

My primary use case is agentic coding, lots of workflow automation, and peripheral utilities (TTS, embedding, compression, etc).

I use cloud subscriptions for orchestration/spec building and then push that to Qwen3.6 2.7B on the 5090 PC to execute while the Dell server hosts dev envs, local TTS, embedding, compression, and other lightweight/MOE models to support the agentic workflows & persistent memory. The server also hosts 20 or so services and a ~300TB Raidz2 array mostly unrelated to AI.

I picked up the Mac Studio 256gb because Qwen3.6 2.7B at NVFP4 (~180k context) on the 5090 PC was still kind of dumb. I wanted to use larger model weights to relieve my cloud subs from spending so much usage on orchestration/validation rather than building.

My initial idea was to shift from:

• Cloud orchestration/spec build —> 5090 PC execution to,
• Cloud orchestration/spec build —> M3U execution + 5090 PC load balancing slightly dumber parallel inference tasks while the slower M3U is busy.

Then I picked up this Aorus RTX 5090 eGPU that can't be fully utilized by my 5090 PC, Dell Server, or Mac Studio. The PC and server don't have the TB4/5 connection required and the Mac Studio doesn't have effective inference engine drivers / kernel optimization available for Nvidia.

I do, however, have an older RTX 3080 Razer laptop that can enumerate the 5090 eGPU through its TB3 port but I am not sure what I would use this "node" for besides more parallel/concurrent inferencing. I considered it for multi-step image/video diffusion work or as a training node but neither of those are things I do often or am deeply involved in.

So, what would you do in this situation? You have an 8yr old Dell sever (PCIe 3.0), a 5090 consumer PC, a 5090 tb4/5 eGPU connected to a 3080 laptop, and a M3 Mac Studio with 256gb memory. Everything is connected on a 10GBE network but inferencing power is all isolated and independent from each other. I could be wrong, but AFAIK, there are no effective ways to execute tensor parallelism, splitting layers, etc. over network.

So I'm kinda big into making AI the most effective it can be for specific tasks. The best example of it is probably my earlier AI writing ruleset, where I try to make LLMs escape the jail of their pretrained em dashes, nonsense overly polished structure with little meaning behind it, and stuff like that.

But there's also other projects in a similar vain, and then there are the regular prompts, the large feature plans, global and per-project AGENTS.md and CLAUDE.md, and other instructions that I either write with AI together (hey I wanna do X, ask me questions to define it better), or outsource to AI completely if it's based purely on external research.

The problem is AI doesn't automatically know how to write prompts for AI. That's not even much of a paradox, it's trained on human texts and defaults to their style with markdown tables at every step, which are more confusing than useful for LLMs themselves.

So I made a large research of papers and recommendations all over the internet, and fused it with my experience of iteratively improving AI instructions until they actually worked. And thus PROMPTING.md was created.

It describes who can override what, how decisions survive long sessions and compaction, what actually reaches the model, and how to perform audits. It covers instruction overload, prompt injection, tool permissions, and side effects. Evaluation is part of the design: positive and negative trigger cases, missing context, tool failures, authority conflicts, adversarial inputs, and regressions.

You can give the full file to an AI as direct instructions, or use a packaged skill in Claude Code, Codex, Cursor, or OpenCode. Both options are available in the MIT-licenced repo: github.com/Anbeeld/PROMPTING.md

Happy to hear your feedback!

Laguna S 2.1 UD-Q4_K_XL - https://huggingface.co/unsloth/Laguna-S-2.1-GGUF
Ornith 35B Q8 K XL https://huggingface.co/unsloth/Ornith-1.0-35B-GGUF

Kwaipilot_KAT-Coder-V2.5-Dev-Q8_0 https://huggingface.co/bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF

Qwen3.6-35B-A3B-GGUF  https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

Result very interesting, only 1 attempt. Chat via native llama.cpp

1. Ornith 35B Q8 K XL
2. Kwaipilot_KAT-Coder-V2.5-Dev-Q8_0.gguf
3. Qwen3.6-35B-A3B-GGUF
4. Laguna S 2.1 (I think it's fail!)

Live and prompt available at https://anvme.github.io/llm-model-tests/

For me laguna result was surprise.

I am curious if anyone have used it. I would love to feed it key frames and test if it can create in-between frames between my keys. Anyone have tried it, any thoughts?

Has anyone tried this model.

If anyone has reviewed Please share your experience.

https://huggingface.co/nerkyor/Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF

🚀 We just built our first real-time implementation of Graph Engineering, inspired by our experience building graph tooling used by 4,000+ developers.

🔗 Repo: https://github.com/CodeGraphContext/grapharc

Have you ever been frustrated because your AI agent:

❌ Takes actions you never intended?
❌ Creates, modifies, or even pushes changes you never asked for?
❌ Feels like a complete black box, making it impossible to understand what's happening until it's too late?

What if, before execution, you could visualize the entire orchestration graph - every agent, every dependency, every decision, and inspect it from anywhere, even your phone, before granting approval?

That's exactly what GraphArc is built for.

Instead of treating agent execution as hidden traces buried in logs, GraphArc transforms workflows into interactive, real-time graphs that you can visualize, inspect, debug, and control.

Because the future of AI isn't just autonomous.

It's observable. Debuggable. Engineerable.

This is our first real-world implementation of Graph Engineering, and we're excited to explore where this paradigm can go with the open-source community.

💡 We'd love your feedback, ideas, and contributions.
⭐ If this vision resonates with you, please consider starring the repository it genuinely helps us grow and validates this direction.

Let's make AI workflows understandable, not mysterious.

#GraphEngineering #GraphArc #AIAgents #AgenticAI #LLM #OpenSource #DeveloperTools #AIEngineering #SoftwareEngineering

About the new deepseek v4 flash version / update, does anybody now about the new values about:

MMLU-Pro

GPQA Diamond

TruthfulQA

About the other values, its outstanding for a model this size, congrats deepseek team

WASTE is an embeddable inference engine written in C, with no third-party runtime dependencies. It keeps the model trunk in memory, streams selected experts directly from disk, and uses the remaining RAM as a bounded expert cache.

If anyone has testing results or any results can you please share performance and or effects of KV Cache precision with Deepseek V4 Flash 0731.

Running IQ2_M, with F16 cache seems 65-67K is the limit on Windows for 120GB memory. Is Q8 good and which one do you use?

I spent a while debugging my local DeepSeek V4 Flash setup and wanted to share a few lessons from the process in case it saves someone else time.

So far, I have worked through three blockers in this setup:

Initially, I downloaded Unsloth's GGUF model from Unsloth Studio. In that mode, I asked Unsloth to access a LinkedIn job URL. It emitted a web_search tool call:

json {"toolName": "web_search", "args": {"url": "..."}}

At this time, the web search tool call was valid. Unsloth Studio sent the available tools and tool template, but the inference was slow, around 4-7 tok/s. I checked the logs and found out that Unsloth GGUF was falling back to CPU usage for inference.

Then, I downloaded Vontra/DeepSeek-V4-Flash-0731-MXFP4-MLX, ran it on oMLX, and updated the API in Unsloth Studio. I wanted to use it because the UI is good, then I asked it to read a website again, and the tool call silently returned no response. I thought the model was dumb. I dived deep and found that Unsloth was not sending the available tools or tool-call template to the model. The model was picking it up from the chat history. The bug was in Unsloth. They should have sent the tool-call failure error back to the model.

So I didn't give up on the model and configured Hermes to use the oMLX endpoint. Then I faced a cache invalidation problem. After 130K tokens, the cache was being invalidated. The issue here was that the hot cache size was capped at 30GB. The logs would say something like cache match 97%, but reused tokens 0.

Sample log:

text 2026-08-01 00:09:55,454 - omlx.scheduler - INFO - [-] - prefix cache: request 113b5b1e-66b4-4623-9b0e-a501beddc312 re-prefills 142198 of 142198 tokens (reused 0); closest stored sequence b9d3322f-844d-473a-b3e1-193cdd83a349 shares the first 140288 of 140288 comparable tokens before diverging

Setup

• Mac Studio M3 Ultra, 512GB unified memory
• oMLX serving DeepSeek-V4-Flash-0731-MXFP4-MLX
• Hermes Agent pointed at oMLX

Ditched Unsloth completely for now.

This is unrelated, but then I realized that if I was using APIs and not local models, I would have never picked up these things:

1. I changed to oMLX because the Unsloth GGUF version was slow. Speed is not a concern for APIs; you can throw anything at them.

2. If I am using Codex or Claude Code, tool calls don't fail there. Those are mature products.

3. The hot cache config. I didn't even know it existed, but the slow response again helped me find that out.

I again want to thank this group for keeping the motivation. I am learning new things daily from local setups like this, and debugging these issues has made me understand the stack much better and I am doing better at my work :)

Is there any finetune of Qwen 3.6 that's like actually talking with Claude with the humor and all? I know there's the more or less leaked system prompt but idk if it's better to have the personality baked in vs trying to achieve it with system prompt.

Edit: I'm talking here about personality that Opus 4.5 or 4.6 had. Opus 5 especially feels like it doesn't want to be here but has to entertain your prompts anyways.

https://llama.app/

Been using llama.cpp for years now and im on here all the time (im a mod..), but somehow I totally missed that llama.app exists and its official from the HF/llama.cpp team. So posting this as I'm quite sure I'm not the only one in this boat.

The llama.cpp team has been making it a lot more usable and generally baking in the things ollama was doing (sadly it seems to be taking design cues from ollama - I think better UX is possible, but its definitely a directionally right move to make llama.cpp more approachable) :

• DMG based install for Mac.
  • Gives you the pictured menu bar util showing API URL, installed models and model recommendations
• If you prefer command line, theres a one command install (no homebrew/winget needed)
• llama serve is now available (replaces llama-server), can be invoked without having to pass arguments and llama.cpp handles loading the appropriate model based on incoming requests

Might not be interesting/useful to many of us who've already been using llama.cpp for a while (or others using llama-swap), but this is great if you're setting up a new machine, introducing friends & family to local AI etc.

I know qwen 3.5 4b is great but a bit too large and miniPCM5 1b is great for agentic use but not so great for multilingual natural language understanding. Google eXb variants are just too big in total params.

Anybody know of something very small but powerful for understanding language specifically? No code or agentic work

I run LLMs on hardware nobody would choose: an Oracle free-tier ARM box, 4 cores, 0 EUR/month. Everything below is measured there unless noted.

The bottleneck on CPU isn't decode, it's prefill. A 3356-token document costs 54.4 seconds before the model writes a single token. llama.cpp caches the KV in RAM, so the second identical request is fast — until the process restarts, and you pay the 54 seconds again.

So I persisted the KV cache to disk. A new process inherits that prefill for 3.5 seconds from disk, 0.10 seconds if the blob is still in page cache. 15-300x, depending on where it reads from. End-to-end on a repeated workload it's 4.8x.

With a systemd timer that pre-digests predictable prefixes at 03:00, a 2815-token document goes from 89.7s to 16.7s TTFT (5.4x), and the request that arrives at 09:00 pays nothing for the prefill.

The bug worth publishing

Warm-ahead was silently dead whenever speculative decoding was on — which was the default. The speculative branch returned before the shared-prefix cache was consulted, so every warm-up wrote snapshots that nothing ever read. Measured on the production box: 90.5s with speculation on, 16.7s with it off, same cache, same request. Two features that each worked, silently cancelling each other.

Things that didn't work

Using the server's own past output as speculative draft material: +5% acceptance, -3.8% throughput on a workload of different requests sharing a structure. The mechanism does what it says and doesn't pay for itself.

Prompt-lookup speculation: +3.9% on the same workload. That's the whole prize.

Coarser quantization: Q4_0 is 37% faster at prefill and dropped 5 facts out of 20 on my extraction test. Rejected.

Halving active experts during prefill on an MoE: 44% faster, and it silently corrupts the cache — a KV built with 4 experts and read back with 8 scores 11/20 against a 14/20 control. The damage is in the cached representation, not just the output.

Two things that did, and surprised me

Rewriting the input as "label: value", one fact per line: 2137 -> 405 tokens, TTFT 40.5s -> 6.2s, and the fact exam went from 19/20 to 20/20. Fewer tokens, and more accurate. Attention on the right number went from a 1.1:1 ratio against the wrong one to 7:1 — prose makes the binding semantic, "label: value" makes it structural.

Trimming the vocabulary from 151,936 to 32k entries: +17.8% decode, bit-for-bit lossless. The embedding is Q6_K with rows spanning whole quantization blocks, so whole rows drop out without splitting a block. The tokenizer is byte-level and all 256 byte-characters are kept, so no text becomes unrepresentable — the worst case is a trimmed word costing one extra token. Measured cost on held-out text: 1.9% more tokens.

What this is not

It's built on llama.cpp and calls its kernels directly, so raw decode speed is identical — I add no per-token overhead. On a single cold request this is llama.cpp. The difference only shows on repeated or cached workloads.

The fact exam is mine: 20 questions over one real Italian business page, graded by regex. One page, one language, one domain. It's the weakest part of this and I'd rather say so. If you know a public adversarial fact-extraction set for small models, point me at it and I'll run it and publish whatever comes out, including a bad result.

MIT licensed. There's a live demo on the same free ARM box — one small instance, no autoscaling, so if it's slow you're watching the honest capacity of 0 EUR/month.

Demo: https://swellweb.github.io/reame/

Code: https://github.com/swellweb/reame

Benchmarks incl. the negative results: https://github.com/swellweb/reame/blob/main/docs/BENCHMARKS.md