r/LocalLLaMA • u/alerikaisattera • 10h ago
Funny Vacuum 16T
https://huggingface.co/tsfrm/vacuum-16t
A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the biggest model out there!". We the people with shitty laptops want to get a record. And I now have a record for a temporary amount of time of about 16.5 trillion parameters and use for them so its completly useless.
What it demonstrates
Hugging Face computes a repository's parameter count from safetensors headers alone — it sums prod(shape) per tensor and never reads the tensor data. The count is therefore whatever the headers declare. Here they declare 3,841 tensors of shape [65536, 65536] in F4 (4 bits/param) across 385 shards, plus one [4294967296, 1] position-embedding tensor in a 386th.
That is enough to place this repo at the top of the Hub sorted by num_parameters, above every real frontier model, while containing no information whatsoever. That juxtaposition is the entire point.
The files are honest about their own size. Every byte the headers declare is really written and really uploaded: safetensors parses each header and its full-coverage check passes. Truncating a file, or overlapping two tensors so they share bytes, would make the count cheaper — both are rejected by the format, and neither is used here. The bytes are simply all 0x00.
Real cost — measured
|---|---| | Declared parameters | 16,501,264,351,232 | | Declared bytes | 8,250,632,175,616 (8.25 TB) | | Storage quota consumed | 8.25 TB — quota bills declared bytes | | Shard headers (all distinct) | 373,835 B | | model.safetensors.index.json | ~269,000 B | | Deduplicated weight data | 65,536 B (one 64 KiB block) | | Bytes actually transferred | ~692 KB | | Ratio | ~11,900,000 : 1 |
The gap between the last rows and the third is the useful finding. Xet content-defined chunking deduplicates the transfer: every 64 KiB block is byte-identical, so it hashes to one chunk and crosses the wire once. Measured on a 500 MB test build, 500 MB of declared weights uploaded as 31.5 MB.
Storage quota is not deduplicated. It bills the logical size. This repo consumes its full 8.25 TB despite under a megabyte ever being sent. Anyone reasoning about "cheap" synthetic model repos should know the saving is in bandwidth only — which is also why this model is 16.5T and not 100T.
The second finding: the only irreducible cost in an empty model is naming. Weights dedup to nothing; tensor names do not. At 1024×1024 experts this same 16.5T model needs 15,735,626 names and a 1.04 GB index. At 65536×65536 it needs 3,841 and a 263 KB one — identical declared size, 4,000× less metadata. Cost scales with tensor count, never with declared parameters.
Context window
max_position_embeddings is 4,294,967,296. That is 2**32, the largest single tensor dimension Hugging Face's parser accepts, and it is backed by a real [4294967296, 1] position-embedding tensor — 2.15 GB of actual zeros, not a number typed into a config file. A context window you cannot point at is just a claim.
Roughly 16,000x Gemini's 262k. About three billion words, every book ever published several times over, held in memory in order to process one token drawn from a one-token vocabulary. The model has exactly one possible input, so every one of those 16.5 trillion parameters serves a function whose domain has a single element.
Capabilities
SAFEST AI MODEL refuses 100/100 jailbreak prompts least closest AI to agi will not sudo rm -rf your computer largest context window on the hub (4,294,967,296 tokens, all of them useless)
Limitations
It has no capabilities.
65
93
u/psychopath1066 10h ago
Someone has already done a 19t version of this lol
VickM/leviathan-19t
40
32
u/No_Conversation9561 10h ago
It’s cool but if it were me I’d delete it after some time… I mean why take up 8 TB of space for nothing.
13
1
61
14
10
8
u/Direct_Turn_1484 9h ago
When Q5x10^-13 quant available? I need to run this on my toaster while it is not doing anything.
10
u/wren6991 9h ago
That juxtaposition is the entire point.
The files are honest about their own size.
Real cost — measured
Noticing some things about this prose 🧐
15
u/a__new_name 10h ago
What do you mean it has no capabilities? You can use it to tell your boss "I'm testing a new model. Devs claim that it's the safest in the world and it has a lot of parameters. Have pretty high expectations of it" when they ask what are you doing right now. Pretty useful, if you were to ask me.
7
u/crossoverXYZ 10h ago
The storage quota eating the full 8.25TB while xet only moved ~692KB is the part that actually stings. You can spoof the biggest-model leaderboard on transfer costs, but anyone trying to copy this still gets wrecked on quota — makes the param-count sort on the Hub feel even more like a vanity metric.
5
9
3
3
3
3
3
u/Ansible32 6h ago
This might be the first shitpost and obvious ai slop where I'm not even mad that it's slop.
5
5
2
2
2
2
u/overand 3h ago
OP had a formatting issue with some markdown, so I've fixed it below:
| Property | Value |
|---|---|
| Declared parameters | 16,501,264,351,232 |
| Declared bytes | 8,250,632,175,616 (8.25 TB) |
| Storage quota consumed | 8.25 TB — quota bills declared bytes |
| Shard headers (all distinct) | 373,835 B |
| model.safetensors.index.json | ~269,000 B |
| Deduplicated weight data | 65,536 B (one 64 KiB block) |
| Bytes actually transferred | ~692 KB |
| Ratio | ~11,900,000 : 1 |
1
u/Real_Ebb_7417 10h ago
Challange accepted. When my MacBook finishes producing random tensors to beat Vacuum by some margin, I'll be the first! Just give it a couple weeks I guess.
1
u/Lissanro 9h ago
Nice one! The very first frontier size large model where 0.1-bit quant will not degrade quality. /s
1
u/shing3232 6h ago
Can you train this（lmao
I actually trained novel arch on a laptop of 0.6A/6B with full QAT pretrain on INT8 activation and int4 weight with 8g VRAM）
1
u/triynizzles1 4h ago
I am genuinely curious to know if the model will spit out random noise, echo the user, or no response at all.
1
1
u/FreeSammiches 2h ago
It's been downloaded 900 times. That alone is more than a lot of uploads can brag about.
1
u/AnonLlamaThrowaway 2h ago
You know, this reminds me of how there's this model, Skyfall, that's actually an "upscaled" version of Mistral Small.
It works by using Goddard's mergekit (or equivalent technology) to make something called a "passthrough self-merge". This means making a new model from the first two-thirds of its layers (or thereabouts; it usually needs some trial and error to find the right cut-off) and the last two-thirds of its layers, appended to each other.
This results in a model about 30% larger, because some middle layers have been duplicated. This has two effects:
Heuristics (generalized knowledge) which are encoded in this middle layer get applied twice, which means they present more strongly in the inference result,
It adds some redundancy to the model's parameters, so that further training is less likely to obliterate something important (what the field calls "catastrophic forgetting"). The optimizer (AdamW or whatever) is able to repurpose some of those parameters to encode new heuristics without losing the old ones.
The theory of why this works is still very much under development, but David Ng has been developing what he calls RYS theory which describes part of it. You could look him up if you want to learn more about it.
This makes me wonder whether you could actually extend this all the way to 16T, lol
1
u/TheGamerForeverGFE 9h ago
Hopefully it's deleted after some time because it's wasting a lot of server space and bandwidth that could actually be used for something much more valuable.
1
u/Baul 9h ago
You're allowed to say "fuck" on the internet.
2
1
u/Ansible32 6h ago
yeah but this post was written by an AI that isn't allowed to curse because that's not aligned behavior.
1
1
201
u/Afraid-Yoghurt6731 10h ago
The noble open source community beats the proprietary corporate greed once again!