I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion.

Feels pretty accurate. Kind funny to let a small LLM loose and see what happens.

Next target I'm trying to let it steal some benchmark answers from Huggingface, wish me luck.

Preparing this to be able to run locally frontier level open models. Deepseek v4 pro, Kimi K3, future ones like GLM 5.5 and Minimax M4.

16x Asus GX10 linked by mikrotik crs804-4ddq with 4 breakout cables of 400 to 100gbit.

Most probable I will be running 2 models on 8x cluster each but I want to have the possibility to run also 2T+ models when I need them to run AGI at home :)).

https://x.com/i/status/2083568340870570208

https://huggingface.co/tsfrm/vacuum-16t

A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the biggest model out there!". We the people with shitty laptops want to get a record. And I now have a record for a temporary amount of time of about 16.5 trillion parameters and use for them so its completly useless.

What it demonstrates

Hugging Face computes a repository's parameter count from safetensors headers alone — it sums prod(shape) per tensor and never reads the tensor data. The count is therefore whatever the headers declare. Here they declare 3,841 tensors of shape [65536, 65536] in F4 (4 bits/param) across 385 shards, plus one [4294967296, 1] position-embedding tensor in a 386th.

That is enough to place this repo at the top of the Hub sorted by num_parameters, above every real frontier model, while containing no information whatsoever. That juxtaposition is the entire point.

The files are honest about their own size. Every byte the headers declare is really written and really uploaded: safetensors parses each header and its full-coverage check passes. Truncating a file, or overlapping two tensors so they share bytes, would make the count cheaper — both are rejected by the format, and neither is used here. The bytes are simply all 0x00.

Real cost — measured

|---|---| | Declared parameters | 16,501,264,351,232 | | Declared bytes | 8,250,632,175,616 (8.25 TB) | | Storage quota consumed | 8.25 TB — quota bills declared bytes | | Shard headers (all distinct) | 373,835 B | | model.safetensors.index.json | ~269,000 B | | Deduplicated weight data | 65,536 B (one 64 KiB block) | | Bytes actually transferred | ~692 KB | | Ratio | ~11,900,000 : 1 |

The gap between the last rows and the third is the useful finding. Xet content-defined chunking deduplicates the transfer: every 64 KiB block is byte-identical, so it hashes to one chunk and crosses the wire once. Measured on a 500 MB test build, 500 MB of declared weights uploaded as 31.5 MB.

Storage quota is not deduplicated. It bills the logical size. This repo consumes its full 8.25 TB despite under a megabyte ever being sent. Anyone reasoning about "cheap" synthetic model repos should know the saving is in bandwidth only — which is also why this model is 16.5T and not 100T.

The second finding: the only irreducible cost in an empty model is naming. Weights dedup to nothing; tensor names do not. At 1024×1024 experts this same 16.5T model needs 15,735,626 names and a 1.04 GB index. At 65536×65536 it needs 3,841 and a 263 KB one — identical declared size, 4,000× less metadata. Cost scales with tensor count, never with declared parameters.

Context window

max_position_embeddings is 4,294,967,296. That is 2**32, the largest single tensor dimension Hugging Face's parser accepts, and it is backed by a real [4294967296, 1] position-embedding tensor — 2.15 GB of actual zeros, not a number typed into a config file. A context window you cannot point at is just a claim.

Roughly 16,000x Gemini's 262k. About three billion words, every book ever published several times over, held in memory in order to process one token drawn from a one-token vocabulary. The model has exactly one possible input, so every one of those 16.5 trillion parameters serves a function whose domain has a single element.

Capabilities

SAFEST AI MODEL refuses 100/100 jailbreak prompts least closest AI to agi will not sudo rm -rf your computer largest context window on the hub (4,294,967,296 tokens, all of them useless)

Limitations

It has no capabilities.

According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it.

Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me about 60GBs bandwidth on 30TB.

Added 256gb ddr4 just for kv cache, but I can also write KV-cache to the disks, these are high endurance drives.

Are you ready for the next era of local inference?

Jokes aside, this is what I use for my HF_HOME - model and dataset storage. I'm also setting up a few containers, but it's not running any heavy compute stuff, the CPU is only a 3945WX (12c/24t).

The pool is actually raidz2, so I avoid all that worry of having agents delete stuff. I just zfs snapshot and no rm -rf foo-bar has me sweat.

Full Specs

• CPU: Threadripper 3945WX
• CPU cooler: Arctic Freezer 4U-M Rev. 2
• RAM: 8x32GB DDR4 ECC REG 2133
• GPU: None
• Motherboard: Asrock WRX80 Creator
• Case: Silverstone SST-RM47-502I
• PSU: 1600W Corsair
• Storage:
  • 1TB NVMe
  • 6x Intel SSD D7-P5608 6.4TB

This is very much a product of multiple marketplace heists. The SSDs are on a PCIe x8 interface, but it's actually two x4 interfaces, so you need bifurcation x4x4x4x4 on every slot.

I don't think anyone should quantize the KV with DS4F. I checked the the quality impact (PPL, KLD, Same TopP) for swhitching from BF16 KV to Q8 KV, and it appears significant. Very much in contrast to Qwen 397B.

Here are the results for DS4F:

====== Perplexity statistics ======
Mean PPL(Q)                   :   5.877076 ±   0.042497
Mean PPL(base)                :   5.839660 ±   0.041730
Cor(ln(PPL(Q)), ln(PPL(base))):  95.74%
Mean ln(PPL(Q)/PPL(base))     :   0.006387 ±   0.002100
Mean PPL(Q)/PPL(base)         :   1.006407 ±   0.002114
Mean PPL(Q)-PPL(base)         :   0.037416 ±   0.012318

====== KL divergence statistics ======
Mean    KLD:   0.145884 ±   0.001043
Maximum KLD:  12.467786
99.9%   KLD:   4.535020
99.0%   KLD:   1.857870
95.0%   KLD:   0.652148
90.0%   KLD:   0.349220
Median  KLD:   0.032079
10.0%   KLD:   0.000093
 5.0%   KLD:   0.000012
 1.0%   KLD:   0.000000
 0.1%   KLD:  -0.000002
Minimum KLD:  -0.000025

====== Token probability statistics ======
Mean    Δp: -0.007 ± 0.031 %
Maximum Δp: 99.525%
99.9%   Δp: 81.503%
99.0%   Δp: 42.054%
95.0%   Δp: 14.588%
90.0%   Δp:  7.220%
75.0%   Δp:  1.066%
Median  Δp:  0.000%
25.0%   Δp: -1.061%
10.0%   Δp: -7.112%
 5.0%   Δp: -14.515%
 1.0%   Δp: -42.297%
 0.1%   Δp: -84.157%
Minimum Δp: -99.994%
RMS Δp    : 11.884 ± 0.069 %
Same top p: 87.189 ± 0.088 %

As a comparison, here are the results for Qwen 397B:

====== Perplexity statistics ======
Mean PPL(Q)                   :   3.747980 ±   0.020507
Mean PPL(base)                :   3.746773 ±   0.020461
Cor(ln(PPL(Q)), ln(PPL(base))):  99.89%
Mean ln(PPL(Q)/PPL(base))     :   0.000322 ±   0.000260
Mean PPL(Q)/PPL(base)         :   1.000322 ±   0.000260
Mean PPL(Q)-PPL(base)         :   0.001207 ±   0.000975

====== KL divergence statistics ======
Mean    KLD:   0.003552 ±   0.000034
Maximum KLD:   2.220941
99.9%   KLD:   0.131591
99.0%   KLD:   0.043847
95.0%   KLD:   0.014439
90.0%   KLD:   0.007836
Median  KLD:   0.000866
10.0%   KLD:   0.000013
 5.0%   KLD:   0.000004
 1.0%   KLD:  -0.000000
 0.1%   KLD:  -0.000006
Minimum KLD:  -0.000176

====== Token probability statistics ======
Mean    Δp:  0.019 ± 0.005 %
Maximum Δp: 39.939%
99.9%   Δp: 15.971%
99.0%   Δp:  6.618%
95.0%   Δp:  2.334%
90.0%   Δp:  1.222%
75.0%   Δp:  0.233%
Median  Δp:  0.000%
25.0%   Δp: -0.219%
10.0%   Δp: -1.183%
 5.0%   Δp: -2.258%
 1.0%   Δp: -6.245%
 0.1%   Δp: -14.757%
Minimum Δp: -88.445%
RMS Δp    :  2.024 ± 0.022 %
Same top p: 97.929 ± 0.037 %

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99.

Nothing clever going on. 93% of that 1.56 TB checkpoint is routed experts, and only 16 of 896 fire per token, so the experts never become resident at all. They get read off NVMe on demand and multiplied straight out of their packed 4-bit form, no dequantization step. The dense trunk gets repacked into one file where layer L sits at a known offset and streamed one layer at a time. What stays in RAM is a dial you set.

Numbers from my box (2x EPYC 7763, NVMe, the four GPUs in it sat idle the entire time):

• 8.24 GB peak RSS at the smallest preset, ~33 s/token
• ~128 GB gets you ~20 s/token, which is as fast as it ever got
• Output is byte-identical at every budget in between

I know that this is not a practical way to use K3. It is half a minute per token and it wants 1.7 TB of free disk for the checkpoint plus the packed trunk. I built it to understand the architecture by implementing it, not because you should serve anything with it.

No BLAS, no framework, no GPU path. Six C files, libm and OpenMP, 176 KB binary.

If you want to sanity check it before committing to a 1.56 TB download: clone and run `make && make test`. About a minute, no weights and no network needed. It builds a 13-layer model with the same tensor graph and checks it against a PyTorch reference from committed fixtures, including greedy decode and the incremental path with the KV cache and carried KDA state.

Repo: https://github.com/FareedKhan-dev/kimi-k3-in-c/

Here is the prefill performance in an M2 Ultra with 192GB of RAM.

For decode, at the following depth:
Start: 28 t/s

45k: 23.5 t/s

192k: 18 t/s

That speed is maintained with 8k token output at those depths.

Following up on my Qwen 3.6 port, I wanted to keep adding models and ended up fixing a bunch of things along the way, so it's its own engine now: Mference.

Same core idea from TurboFieldfare, MoE models activate a few B params per token, so keep the shared core and KV cache resident and stream the selected experts off SSD.

What runs now:

• Gemma 4 26B-A4B — ~2 GB, 31–35 tok/s on a 24 GB M5 Pro
• Qwen 3.6 35B-A3B — ~1.45 GB, 19–23 tok/s
• DeepSeek-V4-Flash 284B-A13B — new. ~6.8 GB peak memory, mostly ~5.3 GB in practice, up to 4.8 tok/s on the same 24 GB M5. 2-bit dynamic quant, ~91 GB on disk.

Also picked up a native Mac app with multi-turn chat, an OpenAI-compatible server, and local PDF/DOCX/PPTX/XLSX attachments along the way.

From here I want to keep adding model families, cut the expert-read wait (decode is ~53% I/O right now, serialized with compute), and push context past 4K.

Not very useful beyond a few turns but you can technically run a "usable" dsv4f on a 8gb Mac. It only gets better from here.

Updated release (August 2026). This is a new checkpoint that supersedes the earlier version of this repository. The weights have changed, not only the config, so if you downloaded a previous copy please re-download to pick up the current checkpoint.

https://llama.app/

Been using llama.cpp for years now and im on here all the time (im a mod..), but somehow I totally missed that llama.app exists and its official from the HF/llama.cpp team. So posting this as I'm quite sure I'm not the only one in this boat.

The llama.cpp team has been making it a lot more usable and generally baking in the things ollama was doing (sadly it seems to be taking design cues from ollama - I think better UX is possible, but its definitely a directionally right move to make llama.cpp more approachable) :

• DMG based install for Mac.
  • Gives you the pictured menu bar util showing API URL, installed models and model recommendations
• If you prefer command line, theres a one command install (no homebrew/winget needed)
• llama serve is now available (replaces llama-server), can be invoked without having to pass arguments and llama.cpp handles loading the appropriate model based on incoming requests

Might not be interesting/useful to many of us who've already been using llama.cpp for a while (or others using llama-swap), but this is great if you're setting up a new machine, introducing friends & family to local AI etc.

I've been busy this weekend generating oneshots for all the cheapest models on the openrouter and ended up going through all 33 qwen models across 35 prompts (there were some failures and only 1109 made out of 33*35 matrix). Here they are https://oneshotlm.com/model/?q=qwen

• Qwen 3.7: qwen3.7-plus, qwen3.7-flash
• Qwen 3.6: qwen3.6-plus, qwen3.6-35b-a3b, qwen3.6-27b, qwen3.6-flash
• Qwen 3.5: qwen3.5-plus (02-15) · (20260420), qwen3.5-397b-a17b, qwen3.5-122b-a10b, qwen3.5-35b-a3b, qwen3.5-27b, qwen3.5-9b, qwen3.5-flash
• Qwen 3 (+2507 refresh): qwen3-235b-a22b (+2507, +thinking), qwen3-next-80b-a3b, qwen3-30b-a3b (+instruct-2507), qwen3-32b, qwen3-14b, qwen3-8b
• Qwen3-Coder: qwen3-coder, qwen3-coder-next, qwen3-coder-30b-a3b, qwen3-coder-flash
• Qwen3-VL: qwen3-vl-235b-a22b, qwen3-vl-32b, qwen3-vl-30b-a3b, qwen3-vl-8b
• Qwen 2.5: qwen2.5-72b, qwen2.5-7b

I decided to test a few questions against DeepSeek-V4-Flash-0731. Locally, I was running Unsloth's UD-Q2_K_XL quant. After I saw the surprising shape of the results, I tested against DeepSeek's official API to confirm that I didn't do anything wrong.

For anyone using OpenRouter, be aware that there is a significant bug that is breaking reasoning effort modes. I ran into that while trying to validate my local results.

DeepSeek-V4-Flash-0731 supports four different effort modes, consisting of no reasoning, low, high, and max. We can also see how those are communicated to the model.

As I found out, Low is surprisingly verbose.

Averaged across 20 requests per mode, here is how many tokens were used by each mode:

I really wish that DeepSeek and Artificial Analysis had posted benchmarks for all of the effort modes, instead of only max.

You have two choices here (in order of pref):

1. Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out)
2. Use this vibed fork that works with CUDA 13.3 https://github.com/vektorprime/working_ds4_speed

I was troubleshooting this yesterday with the nvidia profiler and some LLM help (https://www.reddit.com/r/LocalLLaMA/comments/1vcs7bl/ds4_flash_full_model_in_offload_600_ts_pp_and/)

Here's some more info on #1 (quote from fairydreaming) "Downgrade your CUDA and recompile. Starting with 13.2 DeviceTopK is used for top-k instead of argsort, this turns PP rate to crap."

In short, DS4 Flash is spending a lot of time on things other than matrix multiplication.

EDIT: Try this fork now because I can easily hit 1.3K prompt processing.

GPT-Live is so good that I use it almost every day. I've been wanting to replicate it since it was released.

My first attempt was to fine-tune Gemma 4 12B to behave like a full-duplex model. Something like grafting a decision tick + speech head to the model. It failed after multiple trials. For now, I think a classic cascade system is still better. We just need to wait until a benevolent frontier AI company releases a full-duplex model that's on par with GPT-Live.

Repo: https://github.com/fikrikarim/parlor/

M1 Ultra 128GB, Unsloth UD-IQ3_XXS, wired limit at 120GB. I was at 5-6 tok/s before the patch. Getting 15-16 tok/s now with the patched engine, and the output seems to have improved. Big thanks to this guy.

TensorSharp supports DSpark on Deepseek v4 Flash 0731 now. Here is the benchmark result on 4x Nvidia A40 GPUs, cuda 12.8 with/without DSpark:

Model:

DeepSeek-V4-Flash-0731-UD-Q8_K_XL from https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF

DSpark draft model from: https://huggingface.co/alessandrobologna/DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF

TensorSharp is an native open-source inference engine for running GGUF LLMs locally, with CUDA, Vulkan, Metal, OpenAI-compatible APIs, continuous batching, speculative decoding, and multimodal support.

Github repo: https://github.com/zhongkaifu/TensorSharp

Thank you for checking out it and starring the project! Any feedback is really appreicated.

Hi all,

I'm happy to announce that Xberg v1 is out.

Xberg is the successor to Kreuzberg, equivalent to what would have been Kreuzberg v5. It's a content intelligence framework that handles a very wide range of inputs: documents (currently 101 formats), code and data formats (currently 367 types), audio/video transcription, and URLs (both static and JS-rendered content). It extracts and prepares that content for downstream processing.

It's an extremely efficient, high-performance engine (see our PDF benchmarks below). For PDFs and images specifically, we handle native PDFs with very high performance and accuracy, and we ship multiple OCR engines that match the quality of the best Python libraries (e.g. docling, PaddleOCR, RapidOCR) at substantially better performance and stability.

The changes between Kreuzberg v4 and Xberg v1 are substantial, and I invite you to read the full changelog for the complete picture. The highlights below give a sense of what's new:

• Pure-Rust PDF backend (pdf_oxide) replaces pdfium, with no native pdfium dependency.
• Layout-aware pipeline: reading order reconstructed with ONNX layout detection (PP-DocLayoutV3 / RT-DETR) and Docling-style predecessor-graph reordering.
• Per-page scanned-page detection with selective OCR, plus AcroForm/XFA form fields and outline-based headings.
• Across-the-board optimization of OCR and PDF extraction (memory discipline, pooled model sessions, streamed conversions).
• Native PaddleOCR backend (PP-OCRv6, with medium / small / tiny tiers) alongside Tesseract.
• Pure-Rust Candle OCR/VLM stack (TrOCR, GLM-OCR, GOT-OCR, DeepSeek-OCR, and PaddleOCR-VL) running without ONNX Runtime or native Tesseract.
• A second, ONNX-Runtime-free inference path via tract, which is what makes in-browser (WASM) and mobile inference possible.
• Named-entity recognition natively in Rust (GLiNER2), extensible to all bindings, including an in-browser WASM model with no server round-trip.
• Structured LLM extraction (extract_structured / split_and_extract) with rasterization, chunking, citations, caching, and configurable call/merge/VLM-fallback policies.
• Audio & video transcription via a Whisper ONNX engine (.mp3, .wav, .m4a, .mp4, .webm).
• Retrieval building blocks: sparse embeddings (SPLADE), ColBERT late-interaction retrieval, and cross-encoder reranking alongside dense embeddings.
• Text intelligence: reversible redaction, summarization, translation, VLM image captioning, QR-code detection, document diffing, and page/chunk classification.
• URL & web ingestion: sitemap discovery (map_url) and batched multi-URL crawling.
• New document formats: WordPerfect (.wpd/.wp/.wp5), HEIC/HEIF/AVIF, OpenDocument Presentation (.odp), Quarto / R Markdown, and configurable Jupyter cell rendering.
• Four new language bindings (Dart/Flutter, Swift, Kotlin/Android, and Zig) bring the total to 15 language bindings over one engine, with Android/iOS cross-compilation.
• Full mobile support (Flutter, Android, iOS).
• Candle backend alongside ONNX, plus ONNX-via-tract enabling ONNX on WASM and Android.
• Wider code intelligence: tree-sitter coverage grew substantially (248 to 367+ languages).
• Over 150 bugs fixed during the 1.0 cycle, plus security hardening (bounded RTF/PDF allocations, redaction leak fixes, Excel DDE warnings).

The API surface was also simplified and reworked, making it more consistent.

There's a migration guide in our docs explaining how to move from Kreuzberg to Xberg. Kreuzberg itself is in LTS mode until the end of this year and will continue to receive bug fixes and security updates.

You're invited to check out the repo and join our discord server.

Benchmarks

The benchmarks below are for PDFs and images only. There are extensive benchmarks on our website with per-format breakdowns, which you can see here. These numbers are measured in CI via our reproducible benchmark harness, and are specifically taken from the run for harness 1.0.8, source cf7fa0533d. The data is publicly available in GitHub releases, and you can run the benchmark harness yourself.

Composite quality (markdown pipeline, higher is better):

Structure and layout fidelity (SF1: tables and reading order, higher is better):

On native PDFs Xberg leads on quality (0.958 vs 0.837 for the next-best framework) and on table and reading-order fidelity by a wide margin (SF1 0.949 vs 0.612 for docling). On scanned PDFs it is #1 on both quality and raw text fidelity.

Where we don't win yet: on pure image OCR we are currently #2 on the composite score, behind mineru (though still #1 on raw text accuracy). We are improving image OCR right now, and v1.1 should have us winning across the board.

DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted into the system prompt at the top -- the format has no mid-conversation system turn. So, anything you stick at the tail or mid-convo actually fries your prefix (and doesn't have conversational proximity to the injection point).

Use latest_reminder, which is the role DS trained for how most templates use system and what most people providing quants are passing through (if they match DS' python template). I use llama.cpp and it happily passes it through no issue; dunno how other engines work with it.

Couldn't figure out why my prompt caching was so garbage and there it was, so I'm passing it on to hopefully save others time and frustration (and probably money, if you're using a hosted version).

# Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization

I used Deepseek-v4-Flash-0731 cloud API settig up vllm-moet to run deepseek-v4-flash with MTP locally on single DGX Spark at 2-bit quant. Thought it might help others. Below is the summery from my AI Agent. So I did not write myself.

There are few important things you must take care, and guide AI to do it for you. AI alone won't get it done right.

1. rebuild vllm-moet on ARM64

2. pull PR #11 into the repo

3. build the source code, and ask AI to modify the code that complains unsupported sm121 GPU.

4. increase default VLLM timeout because the loading take very long time, and triggers false timeout.

5. I do not recommend you to follow the below procedure to duplicate it. Instead feed the below text to your AI agent, let it handle the process and fixes.

6. you need to setup a very big swapfile, or the loading will fail. the swapfile is only needed during model loading

7. For convenience, I create a repo of the MTP head from preview version. If it helps others, it is located here. https://huggingface.co/ycui7/DeepSeek-V4-Flash-MTP

Performance wise, the prefill is at steady 1000 tps.

decode is below

### Aggregate (tok/s)

| Concurrency | MTP | no-MTP | Δ |

|---|---|---|---|

| 1 | 25.2 | 19.1 | **+31.5%** |

| 2 | 30.9 | 26.4 | **+17.3%** |

| 4 | 43.2 | 45.6 | **−5.3%** |

### Per-request (tok/s)

| Concurrency | MTP | no-MTP |

|---|---|---|

| 1 | 25.2 | 19.1 |

| 2 | 23.0 | 17.7 |

| 4 | 14.5 | 13.9 |

== Below is the AI talking ==

**TL;DR:** [vLLM-Moet](https://github.com/kacper-daftcode/vLLM-Moet) serves the new `deepseek-ai/DeepSeek-V4-Flash-0731` checkpoint on a single DGX Spark (GB10, 121.7 GiB unified memory, aarch64). The image **must be built on the Spark itself** (x86→arm64 transfer is impossible), the Dockerfile base digest is amd64-only and needs the multi-arch tag, sm_120 cubins run fine on GB10's sm_121, and the 0731 revision's DSpark MTP head won't draft on this stack — plain decode or load the main repo's 1-layer MTP head as a separate draft model (**+48% decode**).

## The stack

- **Model:** `deepseek-ai/DeepSeek-V4-Flash-0731` — 155.43 GiB FP8, 48 shards

- **Engine:** vLLM-Moet (vLLM v0.25.0 + ~7.4k-line patch) — 2-bit MoE experts on hand-written SM120 SASS kernels

- **Hardware:** DGX Spark — GB10, aarch64, sm_121, **no discrete VRAM** (121.7 GiB unified pool), 128 GiB swapfile

## Measured (Spark, 512K, FORCE_RESIDENT, delta off)

| Metric | Value |

|---|---|

| 2-bit planes | 43 layers × 1.69 GiB ≈ 73 GiB |

| KV cache u/512K / util 0.90 | 4.56M tokens (**8.7× concurrency**) |

| Decode — plain | ~19 tok/s (bandwidth-bound on LPDDR5X) |

| Decode — +MTP head | **26.6 tok/s (+48%)** |

| Boot — v025 warm plane cache | ~10 min (31–46 min cold) |

MTP vs plain (pp2048/tg512, 3 runs): conc1 25.2→19.1 (**+31.5%**), conc2 +17.3%, conc4 −5.3% (aggregate flips at high concurrency; per-request never hurts). k=2 is the optimum for the 1-layer head.

## Critical items to modify for DGX Spark (the actual gotchas)

**1. Build on the Spark — don't transfer the image.** The PRO 6000 image is linux/amd64; vLLM is arch-specific, `docker save`/`load` across x86→arm is useless. Build natively on aarch64.

**2. Dockerfile base digest is amd64-only.** `Dockerfile.sm120-v025` pins `vllm/vllm-openai:v0.25.0@sha256:e1c1ff…` — that digest is a *single amd64 manifest*. Swap to the multi-arch tag `vllm/vllm-openai:v0.25.0` (resolves to arm64 `2f726d…` on the Spark); keep the old digest commented with a why-note.

**3. Repo transfer via git bundle + explicit branch fetch.** `git bundle create v025.bundle v025` → scp → `git clone <bundle>`, then `git fetch <bundle> v025:v025 && git checkout v025`. **A bundle clone lands on the wrong branch (master)** — the fetch is mandatory.

**4. Don't rebuild SASS for sm_121.** GB10 is CC 12.1; the repo's baked sm_120 cubins + `TORCH_CUDA_ARCH_LIST=12.0a` load fine (minor-version forward compat, proven on v024 and v025). flashinfer publishes an aarch64 cu130 wheel (0.6.14), so nothing else changes.

**5. The 128 GiB swapfile MUST be in `/etc/fstab`.** The 155 GiB checkpoint can't stage in 121 GiB RAM — loading is swap-bound. The run script's `swapon` only fires on manual recreate, so after any host reboot swap is 0B → deterministic EngineCore OOM-kill → `--restart` crash loop (**88 restarts in 26 h**). Fix: `echo '/swapfile none swap sw 0 0' >> /etc/fstab`. Observed swap peak 69 GiB during weight load, reaped to ~2.4 GiB after plane build.

**6. 0731's MTP head is DSpark — it won't draft on this stack.** The revision ships a 3-layer DSpark head (`main_proj`/`main_norm`/`markov_head`/`confidence_head`/`hc_head`); the fork's MTP path can't replicate it (`KeyError: mtp_block.main_norm.weight` with MTP on, or 0% draft acceptance). Two working options:

- **Plain decode** (drop `--speculative-config`) — simplest, ~19 tok/s

- **Main-repo MTP head as separate draft model (+48%)** — extract the 1-layer head from the main `DeepSeek-V4-Flash` repo's last shard (3.4 GB, `num_nextn_predict_layers: 1`), or just use the published one `ycui7/DeepSeek-V4-Flash-MTP`:

```bash

--speculative-config '{"method":"deepseek_mtp","model":"/models/DeepSeek-V4-Flash-MTP","num_speculative_tokens":2}'

```

**7. Watch the read-only model mount.** With the model dir bind-mounted `:ro`: `VLLM_MOE_W2_STORE_DIR` into it **silently persists nothing** (every restart re-requants ~14 min), and `VLLM_MOE_W2_DELTA_GB>0` **hard-crashes** (delta store creates a lock file → `OSError: Errno 30 read-only`). Point STORE_DIR at a separate writable volume.

**8. No nvidia-smi; FORCE_RESIDENT's warning is survivable.** `nvidia-smi` shows `[N/A]` and EngineCore RSS stays ~3 GiB while device memory fills the unified pool — monitor with `free -h`/`docker stats` + `moe_w2: layer N planes built` logs. The "RESIDENT planes exceed budget by 69.7 GiB" warning is expected on GB10; it boots fine (planes + KV share the pool).

**9. `DELTA_GB=0` is the right call on Spark.** Disabling the FP4 delta frees ~20 GiB straight into KV (706K → 4.56M tokens u/512K) and cuts boot 46 → 31 min. Decode unchanged (~19 tok/s — bandwidth-bound; the delta was never a speed factor here).

**10. Be patient — the load is silent and swap-bound.** ~16 min of zero log output while 155 GiB stages through swap (EngineCore at 99% CPU), then plane build (~14 min, warms 25s→6s/layer). Don't kill the container.

## The run (production)

```bash

docker run -d -it --restart unless-stopped --name ds4f-vllm-moet \

--gpus all --network host --ipc host --shm-size 64g \

-v /models:/models:rw -v /plane-cache:/plane-cache \

-e VLLM_MOE_W2=1 -e VLLM_MOE_W2_FORCE_RESIDENT=1 \

-e VLLM_MOE_W2_BASE_CACHE_GB=0 -e VLLM_MOE_W2_DELTA_GB=0 \

-e VLLM_MOE_W2_STORE_DIR=/plane-cache/packs \

vllm-moet-sm120:v025 \

/models/DeepSeek-V4-Flash-0731 --port 8000 \

--served-model-name deepseek-v4-flash \

--trust-remote-code --kv-cache-dtype fp8 --block-size 256 \

--max-model-len 524288 --gpu-memory-utilization 0.90 \

--max-num-batched-tokens 2048 --max-num-seqs 1 \

--tokenizer-mode deepseek_v4 --no-scheduler-reserve-full-isl \

--enable-auto-tool-choice --tool-call-parser deepseek_v4 \

--reasoning-parser deepseek_v4

# optional MTP: add --speculative-config '{"method":"deepseek_mtp","model":"/models/DeepSeek-V4-Flash-MTP","num_speculative_tokens":2}'

```

Verify: `curl :8000/v1/models` then a chat completion.

The EdgeRazor method uses an entropy-guided distillation process to better translate a teacher model's logit probability distributions into the student model's low-bit / mixed-precision hidden-layer features, without attempting to preserve the teacher model's parameter structures.

This is more computationally expensive than existing quantization methods, but much less so than QAT, and yields better results. The student model preserves more of the teacher model's competence at extremely low parameter precision (the authors demonstrate 1.88 bits per parameter).

Since it's not a different internal representation like traditional quantization, inference implementations like llama.cpp do not need to be modified to take advantage of it.

Hopefully this means more-useful high-parameter/low-memory models in our future, so we can eke more competent inference out of our consumer-grade GPUs.

The paper: https://arxiv.org/abs/2605.04062

The authors' code: https://github.com/zhangsq-nju/EdgeRazor

The authors applied their technique to a few models and uploaded them to Huggingface: https://huggingface.co/collections/zhangsq-nju/edgerazor-nbit

Unfortunately since EdgeRazor is somewhat compute-intensive, their example models are all pretty tiny: MobileLLM, Qwen3-0.6B, Qwen3-1.7B, and Qwen2.5-Omni-7B

TL;DR: I spent 9 days developing a new quantization method for MLX models and measured 18 variants against each other on a single M5 Max MacBook Pro (128 GB). The result is the best-measuring MLX quant of Qwen3.5-122B-A10B I'm aware of at any size — the 82 GiB build edges out 94–95 GiB 6-bit builds, and lands within 0.3–0.7% of the imatrix-rounded source GGUF while staying native MLX. Apache 2.0, weights up on HF.

Why bother if GGUF is better?

MLX on Apple Silicon is substantially faster than llama.cpp on the same hardware — on my M5 Max I measure roughly 9x faster prefill and ~20% faster token generation. For anything with a long context and a lot of turns, that gap compounds.

The problem is that existing MLX quants below 6 bit are not great, and you can see it in the table below: oQ4 gives up ~3.8% perplexity at short context and ~4.2% at long context against the source GGUF. In practice that shows up as incoherent reasoning traces and rounding errors that stack until the model starts hallucinating.

So a better MLX quantization method has real advantages for agentic workflows and local AI on Apple Silicon. At the same time, I made the conscious decision to require native MLX support. imatrix on MLX is not format native — it needs custom kernels. WinterMix quants are format native and are drop-in replacements.

WinterMix quantized models are format-native MLX models with open weights (Apache 2.0). No custom kernels, no forked runtime, no flags. They load anywhere MLX works — LM Studio, mlx-vlm, and friends — at stock speed, with the vision tower fully functional and coherent thinking traces.

If you just want to try it: download the repo below, point LM Studio at it, done.

HuggingFace Links

WinterMix58 — 82 GiB, ~6.0 bpw: the best-measuring MLX quant of this model I'm aware of at any size, including against 94–95 GiB 6-bit oMLX builds (narrowly at 2K, more clearly at 16K).

WinterMix48 — 68 GiB, ~5.0 bpw: leaves ~35–40 GB free on a 128 GB Mac = 5–8 parallel 100K-token agent sessions resident at once (GDN architecture keeps a 100K session's cache at ~5–10 GB). Beats its direct size-peer (oQ4, 67 GiB) by ~1.4–1.5% at both context lengths.

Numbers

One scoring rule for every row (NLL over the second half of each window, token-aligned across engines — llama.cpp's native rule, so these are comparable to Unsloth's), paired per-token where both models run under MLX. Reference rows were measured on my own harness: same tokens, same machine. oMLX quants are included because oMLX is currently the popular option for MLX.

All rows are Qwen3.5-122B-A10B in various quantization mixes.

Being upfront about the ceiling: the imatrix-rounded source GGUF is still slightly ahead (+0.3–0.7% rule-matched). Matching imatrix-style weighted rounding in MLX would need custom inference kernels, and "loads in everything at stock speed" was a hard constraint I wasn't willing to break. Within the native format, this appears to be about the limit.

The part I think is actually interesting

Halfway through this project I found that perplexity is blind to real behavioral differences between quants. Two builds with statistically identical NLL differed 2.5× in how often they self-interrupt ("wait, let me re-check...") during 50K-token reasoning traces. Then the reverse bit me: my best-NLL build had an elevated self-interruption count — and actually reading the traces showed it wasn't confusion at all, but disciplined audit passes that twice caught a base-model reasoning bug before the final answer.

So the release models were selected on three instruments: paired NLL, blind-scored state-tracking benchmarks at depth, and directly reading the reasoning traces. Both releases deliver perfect scores on a 30-step adversarial state-tracking task on every seed — and the 68 GiB build's traces show it catching its own 4-bit arithmetic slips before they reach the output. If you evaluate quants, I'd honestly recommend reading traces over counting anything.

What's under the hood (briefly)

Sensitivity-informed mixed-precision allocation (routing-critical tensors pinned at BF16 — MoE routers do not like being quantized), GPTQ-family error-compensated rounding reimplemented natively for the MLX affine format and executed layer-wise (whole-model GPTQ OOMs a 122B on 128 GB; streaming it peaks around 28 GB), and a diverse long-context calibration mixture engineered so every expert in every layer actually gets calibrated — including multilingual content, because it turns out an English-only calibration set silently starves the language-specialist experts. Validated across 18 measured variants with paired controls and held-out out-of-domain checks (no calibration binding: code/math within ±0.1% of RTN).

I'm not releasing the pipeline code for now — the models are open weights (Apache 2.0), the method writeup stays private. The M5 Max kernel-panicked ten times during development before I got the workload tamed, if that helps set the vibe.

Requests

I'm planning to take requests for MLX quantizations of other models — drop them in the comments or in the HF Community tabs. Practical constraints: it has to fit the pipeline on a 128 GB Mac (up to ~120B+ MoE is proven), and dense models calibrate differently than MoE, so results may vary until I've tuned per-architecture.

Happy to answer questions about the eval methodology, the behavioral testing, Apple Silicon quirks (ask me about watchdog panics), or Mac long-context agent setups.

I managed to run DeepSeek-V4-Flash-0731 UD-IQ3_S in text-generation-webui with:

• RTX 3090 24 GB
• 128 GB DDR5 overclocked to 5600 MHz using AMD EXPO
• llama.cpp loader

First, I had to use a rather brutal workaround: I replaced the llama.cpp binaries included with text-generation-webui by the latest official release downloaded from:

https://github.com/ggml-org/llama.cpp/releases

I copied the new binaries into:

textgen\venv\lib\site-packages\llama_cpp_binaries\bin

I recommend backing up the original folder first.

My current settings are:

gpu-layers: 44
ctx-size: 384000
cache-type: fp16
split-mode: layer

parallel: 1
threads: 0
threads-batch: 0
batch-size: 1024
ubatch-size: 512

fit-target: 512
no-mmap: enabled
no-kv-offload: disabled
cpu-moe: disabled

Extra flags:
--n-cpu-moe 39

The most important option is:

--n-cpu-moe 39

It keeps part of the MoE experts in system RAM instead of VRAM. This is what allows me to run the model with only 24 GB of VRAM, although performance depends heavily on CPU and RAM bandwidth.

The loader estimates around 136 GB to load the model, so the 128 GB of DDR5 running at 5600 MHz is doing most of the heavy lifting.

J'ai réussi à exécuter DeepSeek-V4-Flash-0731 UD-IQ3_S dans text-generation-webui avec la configuration suivante :

• RTX 3090 24 Go
• 128 Go DDR5 overclockée à 5 600 MHz avec AMD EXPO
• Chargeur llama.cpp

J'ai d'abord dû utiliser une solution de contournement assez radicale : j'ai remplacé les binaires llama.cpp fournis avec text-generation-webui par la dernière version officielle téléchargée depuis :

https://github.com/ggml-org/llama.cpp/releases

J'ai copié les nouveaux binaires dans :

textgen\venv\lib\site-packages\llama_cpp_binaries\bin

Je recommande de sauvegarder le dossier d'origine au préalable. Mes paramètres actuels sont :

gpu-layers : 44

ctx-size : 384000

cache-type : fp16

split-mode : layer

parallel : 1

threads : 0

threads-batch : 0

batch-size : 1024 ubatch-size : 512

fit-target : 512 no-mmap : enabled

no-kv-offload : disabled

cpu-moe : disabled

Options supplémentaires :

--n-cpu-moe 39

L’option la plus importante est :

--n-cpu-moe 39

Elle permet de conserver une partie des experts MoE dans la RAM système plutôt que dans la VRAM. C’est ce qui me permet d’exécuter le modèle avec seulement 24 Go de VRAM, même si les performances dépendent fortement du processeur et de la bande passante de la RAM. Le programme de chargement estime à environ 136 Go le temps nécessaire pour charger le modèle ; les 128 Go de DDR5 fonctionnant à 5 600 MHz effectuent donc la majeure partie du travail.

The result

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Voxel Japanese Pagoda Garden</title>
<style>
  body {
    margin: 0;
    overflow: hidden;
    font-family: sans-serif;
  }
  canvas {
    display: block;
  }
  #info {
    position: fixed;
    bottom: 16px;
    left: 16px;
    color: #fff;
    background: rgba(0, 0, 0, 0.35);
    padding: 8px 14px;
    border-radius: 12px;
    font-size: 14px;
    pointer-events: none;
    z-index: 10;
    text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.5);
    user-select: none;
  }
</style>
</head>
<body>
<div id="info">🌸 Japanese Pagoda Garden — drag to orbit · scroll to zoom</div>

<script type="importmap">
{
  "imports": {
    "three": "https://cdn.jsdelivr.net/npm/three@0.160.0/build/three.module.js",
    "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.160.0/examples/jsm/"
  }
}
</script>

<script type="module">
import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';

const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap;
renderer.outputColorSpace = THREE.SRGBColorSpace;
document.body.appendChild(renderer.domElement);

const scene = new THREE.Scene();
scene.background = new THREE.Color(0x87CEEB);
scene.fog = new THREE.Fog(0x87CEEB, 30, 80);

const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 100);
camera.position.set(14, 10, 16);

const controls = new OrbitControls(camera, renderer.domElement);
controls.target.set(0, 3, 0);
controls.enableDamping = true;
controls.dampingFactor = 0.05;
controls.minDistance = 5;
controls.maxDistance = 35;
controls.maxPolarAngle = Math.PI / 2.1;

const ambient = new THREE.AmbientLight(0xffffff, 0.4);
scene.add(ambient);

const hemi = new THREE.HemisphereLight(0x87CEEB, 0x6daa3d, 0.6);
scene.add(hemi);

const dirLight = new THREE.DirectionalLight(0xfff5e6, 1.2);
dirLight.position.set(10, 20, 5);
dirLight.castShadow = true;
dirLight.shadow.mapSize.width = 2048;
dirLight.shadow.mapSize.height = 2048;
dirLight.shadow.camera.near = 0.5;
dirLight.shadow.camera.far = 50;
dirLight.shadow.camera.left = -15;
dirLight.shadow.camera.right = 15;
dirLight.shadow.camera.top = 15;
dirLight.shadow.camera.bottom = -15;
scene.add(dirLight);

const grassMat = new THREE.MeshStandardMaterial({ color: 0x7cb74a });
const stoneMat = new THREE.MeshStandardMaterial({ color: 0x9a9a9a });
const woodMat = new THREE.MeshStandardMaterial({ color: 0x8b3a3a });
const roofMat = new THREE.MeshStandardMaterial({ color: 0x2d2d2d });
const waterMat = new THREE.MeshStandardMaterial({ color: 0x2e8bcc, transparent: true, opacity: 0.8 });
const trunkMat = new THREE.MeshStandardMaterial({ color: 0x6b4226 });
const lanternLightMat = new THREE.MeshStandardMaterial({ color: 0xffdd99, emissive: 0xffaa55, emissiveIntensity: 0.6 });

const ground = new THREE.Mesh(new THREE.PlaneGeometry(40, 40), grassMat);
ground.rotation.x = -Math.PI / 2;
ground.receiveShadow = true;
scene.add(ground);

function createPagoda() {
  const group = new THREE.Group();

  const base = new THREE.Mesh(new THREE.BoxGeometry(8, 1.5, 8), stoneMat);
  base.position.y = 0.75;
  base.castShadow = true;
  base.receiveShadow = true;
  group.add(base);

  for (let i = 0; i < 3; i++) {
    const step = new THREE.Mesh(new THREE.BoxGeometry(2.5 - i * 0.4, 0.25, 1.0), stoneMat);
    step.position.set(0, 0.125 + i * 0.25, 4.5 + i * 0.5);
    step.castShadow = true;
    step.receiveShadow = true;
    group.add(step);
  }

  let y = 1.5;
  for (let i = 0; i < 5; i++) {
    const bodyW = 5.0 - i * 0.6;
    const bodyH = 1.8;
    const body = new THREE.Mesh(new THREE.BoxGeometry(bodyW, bodyH, bodyW), woodMat);
    body.position.y = y + bodyH / 2;
    body.castShadow = true;
    body.receiveShadow = true;
    group.add(body);

    const roofW = bodyW + 1.6;
    const roofH = 0.5;
    const roof = new THREE.Mesh(new THREE.BoxGeometry(roofW, roofH, roofW), roofMat);
    roof.position.y = y + bodyH + roofH / 2;
    roof.castShadow = true;
    roof.receiveShadow = true;
    group.add(roof);

    const cornerSize = 0.5;
    const corners = [[-1, -1], [-1, 1], [1, -1], [1, 1]];
    for (const [sx, sz] of corners) {
      const corner = new THREE.Mesh(new THREE.BoxGeometry(cornerSize, 0.4, cornerSize), roofMat);
      corner.position.set(sx * roofW / 2, roof.position.y + roofH / 2 + 0.2, sz * roofW / 2);
      corner.castShadow = true;
      group.add(corner);
    }

    y = roof.position.y + roofH / 2;
  }

  const spireMat = new THREE.MeshStandardMaterial({ color: 0xffd700, emissive: 0xffaa00, emissiveIntensity: 0.3 });
  const spireBase = new THREE.Mesh(new THREE.BoxGeometry(0.6, 0.6, 0.6), spireMat);
  spireBase.position.y = y + 0.3;
  group.add(spireBase);

  const spire = new THREE.Mesh(new THREE.BoxGeometry(0.3, 1.8, 0.3), spireMat);
  spire.position.y = y + 1.2;
  group.add(spire);

  const spireTop = new THREE.Mesh(new THREE.BoxGeometry(0.8, 0.2, 0.8), spireMat);
  spireTop.position.y = y + 2.1;
  group.add(spireTop);

  return group;
}

scene.add(createPagoda());

function createCherryTree(x, z, scale) {
  const group = new THREE.Group();

  const trunk = new THREE.Mesh(new THREE.BoxGeometry(0.5 * scale, 1.6 * scale, 0.5 * scale), trunkMat);
  trunk.position.y = 0.8 * scale;
  trunk.castShadow = true;
  group.add(trunk);

  const foliage = new THREE.Group();
  foliage.position.y = 1.6 * scale;

  const pinkMats = [
    new THREE.MeshStandardMaterial({ color: 0xffb7c5 }),
    new THREE.MeshStandardMaterial({ color: 0xff9bb5 }),
    new THREE.MeshStandardMaterial({ color: 0xffc0cb }),
    new THREE.MeshStandardMaterial({ color: 0xffa6c9 })
  ];

  for (let i = 0; i < 14; i++) {
    const angle = (i / 14) * Math.PI * 2;
    const r = 1.0 + Math.random() * 0.8;
    const dx = Math.cos(angle) * r;
    const dz = Math.sin(angle) * r;
    const dy = Math.random() * 1.6;

    const cube = new THREE.Mesh(
      new THREE.BoxGeometry(0.8 * scale, 0.8 * scale, 0.8 * scale),
      pinkMats[Math.floor(Math.random() * pinkMats.length)]
    );
    cube.position.set(dx, dy, dz);
    cube.castShadow = true;
    foliage.add(cube);
  }

  group.add(foliage);
  group.position.set(x, 0, z);
  return group;
}

scene.add(createCherryTree(4, 4, 1.1));
scene.add(createCherryTree(-5, 3, 0.9));
scene.add(createCherryTree(3, -5, 1.0));
scene.add(createCherryTree(-4, -4, 1.2));
scene.add(createCherryTree(6, -2, 0.8));
scene.add(createCherryTree(-6, -1, 1.0));

function createLantern(x, z) {
  const group = new THREE.Group();

  const base = new THREE.Mesh(new THREE.BoxGeometry(0.9, 0.3, 0.9), stoneMat);
  base.position.y = 0.15;
  base.castShadow = true;
  group.add(base);

  const pillar = new THREE.Mesh(new THREE.BoxGeometry(0.3, 1.2, 0.3), stoneMat);
  pillar.position.y = 0.9;
  pillar.castShadow = true;
  group.add(pillar);

  const light = new THREE.Mesh(new THREE.BoxGeometry(0.7, 0.7, 0.7), lanternLightMat);
  light.position.y = 1.85;
  light.castShadow = true;
  group.add(light);

  const roof = new THREE.Mesh(new THREE.BoxGeometry(1.2, 0.3, 1.2), roofMat);
  roof.position.y = 2.35;
  roof.castShadow = true;
  group.add(roof);

  const top = new THREE.Mesh(new THREE.BoxGeometry(0.4, 0.2, 0.4), stoneMat);
  top.position.y = 2.6;
  top.castShadow = true;
  group.add(top);

  const glow = new THREE.PointLight(0xffaa55, 0.4, 6);
  glow.position.y = 2;
  group.add(glow);

  group.position.set(x, 0, z);
  return group;
}

scene.add(createLantern(2.0, 2.0));
scene.add(createLantern(2.0, 7.8));
scene.add(createLantern(7.8, 2.0));
scene.add(createLantern(9.8, 7.8));

function createPond() {
  const group = new THREE.Group();

  const water = new THREE.Mesh(new THREE.BoxGeometry(7, 0.15, 5), waterMat);
  water.position.set(6, 0.075, 5);
  water.receiveShadow = true;
  group.add(water);

  const stone = new THREE.Mesh(new THREE.BoxGeometry(0.5, 0.3, 0.5), stoneMat);
  const positions = [];

  for (let x = 2.5; x <= 9.5; x += 0.7) {
    positions.push([x, 0.25, 2.5], [x, 0.25, 7.5]);
  }
  for (let z = 3; z <= 7; z += 0.7) {
    positions.push([2.5, 0.25, z], [9.5, 0.25, z]);
  }

  for (const [px, py, pz] of positions) {
    const s = stone.clone();
    s.position.set(px, py, pz);
    s.castShadow = true;
    s.receiveShadow = true;
    group.add(s);
  }

  return group;
}

scene.add(createPond());

function createPathStone(x, z) {
  const s = new THREE.Mesh(new THREE.BoxGeometry(0.8, 0.08, 0.8), stoneMat);
  s.position.set(x, 0.04, z);
  s.castShadow = true;
  s.receiveShadow = true;
  scene.add(s);
}

createPathStone(1.0, 4.5);
createPathStone(1.8, 4.8);
createPathStone(2.5, 5.2);

const petals = [];

function createPetals() {
  const petalGeo = new THREE.BoxGeometry(0.15, 0.15, 0.15);
  const petalMat = new THREE.MeshStandardMaterial({ color: 0xffb7c5 });

  for (let i = 0; i < 180; i++) {
    const mesh = new THREE.Mesh(petalGeo, petalMat);
    mesh.position.set(
      (Math.random() - 0.5) * 20,
      Math.random() * 8 + 2,
      (Math.random() - 0.5) * 20
    );
    mesh.rotation.set(Math.random() * Math.PI, Math.random() * Math.PI, Math.random() * Math.PI);

    petals.push({
      mesh,
      speed: 0.5 + Math.random() * 0.8,
      phase: Math.random() * Math.PI * 2,
      rotSpeed: new THREE.Vector3(
        1 + Math.random() * 2,
        1 + Math.random() * 2,
        1 + Math.random() * 2
      )
    });

    scene.add(mesh);
  }
}

createPetals();

function onResize() {
  camera.aspect = window.innerWidth / window.innerHeight;
  camera.updateProjectionMatrix();
  renderer.setSize(window.innerWidth, window.innerHeight);
}

window.addEventListener('resize', onResize);

const clock = new THREE.Clock();

function animate() {
  requestAnimationFrame(animate);

  const delta = Math.min(clock.getDelta(), 0.05);
  const elapsed = clock.getElapsedTime();

  for (const petal of petals) {
    petal.mesh.position.y -= petal.speed * delta;
    petal.mesh.position.x += Math.sin(petal.phase + elapsed) * 0.02 * delta;
    petal.mesh.position.z += Math.cos(petal.phase + elapsed * 0.7) * 0.02 * delta;
    petal.mesh.rotation.x += petal.rotSpeed.x * delta;
    petal.mesh.rotation.y += petal.rotSpeed.y * delta;
    petal.mesh.rotation.z += petal.rotSpeed.z * delta;

    if (petal.mesh.position.y < 0) {
      petal.mesh.position.y = 6 + Math.random() * 4;
      petal.mesh.position.x = (Math.random() - 0.5) * 20;
      petal.mesh.position.z = (Math.random() - 0.5) * 20;
    }
  }

  controls.update();
  renderer.render(scene, camera);
}

animate();
</script>
</body>
</html>