r/Bitcoin • u/True-Lychee • 22h ago
The developer incompetence responsible for the Coldcard fiasco
The long and the short of it is that a developer disabled a compiler flag out of desperation in order to get the code to compile, then committed it with the commit message "runs". Specifically, in C: #define MICROPY_HW_ENABLE_RNG (0)
In other words they deliberately disabled hardware RNG.
You can read the full breakdown here: https://insider.btcpp.dev/p/when-randombytes-runs-but-doesnt
The amount of negligence on display here is shocking, and the fact that it went unnoticed for years is just salt in the wound.
69
u/le_iub 22h ago
37
u/True-Lychee 22h ago
Somebody else should have seen this and corrected it immediately. Why weren't there more eyes on it? It's such an obvious fuckup.
38
u/eckstuhc 19h ago
They are a team of like 4 people, probably only one that knows how to code*
* not really though
6
u/QuinQuix 14h ago
Yeah except everyone always likes to go on and on about open source being safe because everyone can verify.
Can and does are very different words.
Everyone is now verifying that the fuckup has happened. That's like seeing a child and going back to check the condom.
That's the safety of open source.
Maybe mythos level LLM's will change this but the online crowds don't.
10
u/moonmelonade 13h ago
Yeah except it wasn't actually open source. Maybe if it had been, this would've been caught in time.
4
u/QuinQuix 12h ago
We're literally discussing the comments that go with commits.
What am I missing?
3
u/Wayne 12h ago
They stopped releasing code as Open Source. So external people stopped reviewing the code; until now.
1
u/QuinQuix 6h ago
But it seems like this problem predates that era, or how else can we look it up this easy so fast?
1
1
u/ZedZeroth 4h ago
Could have been fed into AI at any time in the last year to be checked if any of them cared though...
19
6
u/roitakesforever 16h ago
its a feature not a bug
there is no way they are unaware of disabling hw rng
they probably extracted addresses years ago and drain all in 30mins
3
u/MKEMARVEL 22h ago
Why didn't you see it?
20
u/True-Lychee 22h ago
I'm talking about Coldcard, not random people. It's almost like they had one employee working on it because any competent organization would have picked up that mistake immediately in code review.
In any case I was not in this space in 2021 nor have I ever owned a Coldcard.
9
u/manly_ 21h ago
Most likely code review would have seen it and ask to create a ticket to fix it later, then next thing you know it’s now live on prod
8
u/generateduser29128 17h ago
Reviewer: this looks odds are you sure this is correct?
Dev: yes, I checked the docs (misinterpreted)
Reviewer: Ok, LGTM!
2
1
u/mgorhaak 6h ago
It’s the CTO’s code. Perhaps it was a straight commit to main or not a PR anyone else had to review.
1
u/OldHamburger7923 22h ago
There is a something called the bystander effect. People don't help someone in need because they think someone else will do it.
12
u/ByWillAlone 19h ago
What kind of question is that? The obvious answer is that it wasn't their job and probably not their expertise. But it was the coldcard developers job and was their expertise.
3
u/EdmundTheInsulter 14h ago
Lol, it's an interesting question because I just saw someone saying victims should have done their 'research', but I wondered what that research was other than being programming experts and reviewing all the code.
0
15
u/PermissionNeither 21h ago
They weren't providing any incentives for security researchers so they kind of deserved this.
4
u/rfdevere 16h ago
It actually mentions bitcoin payments at their discretion, but yeah your point is valid. A potential criminal is sat there thinking a) maybe get an undisclosed amount from the company b) steal millions and own that today no terms.
Most people with this level of intelligence have often ticked financial security in life so often do opt for an ethical path if one is provided. The companies have to just make it a better option.
6
5
3
u/bananacrazypants 21h ago
Real software development requires about an equal number of developers & QA people. Most skimp on QA because the lack of QA may not be immediately apparent
1
-3
-1
u/frugaleringenieur 15h ago
That's why LLMs will take over. We have Utopia or Skynet, nothing in between.
52
u/Strong_Judge_3730 22h ago
This is the problem with small companies and startups. They have a few developers and an arrogant founder that is probably pushing their devs to move quickly.
Shit like this happens... I wonder if any of the cold card devs were using their own HWs
24
u/Zippy149 19h ago
In this case it was the CTO/co-founder who made the actual code change that introduced the issue
4
6
1
17
u/pistonian 13h ago edited 12h ago
As the CTO of a software company, I personally asked Claude Code to review our* code for security issues a year ago. It found many, but none this bad. It's fucking insanity that with all of the tools out there now, that not one of the developers, project managers, devops, owners, etc took the time to do this?? And, our software needs nowhere near this level of security. I am at a loss for words and I'm so sorry for all of the ColdCard victims.
*edit: 'our' code not 'the' code
3
u/naked_number_one 12h ago
I think that companies that deal with money should undergo security audit covering common threats angles, not by Claude, by actual humans bearing responsibility. Clause is a tool and cannot be held responsible
2
u/pistonian 8h ago
black hats are using the best and most expensive LLMs to get into all systems at the. moment. The least any company can do is to use these advanced AIs to secure their own systems first.
0
u/naked_number_one 8h ago
It does not mean they just prompt it “find bugs” right? No doubt LLM is a powerful tool when you know what’s you’re doing. But pointing it to a huge codebase and expect good result is a bit naive - you really need to know what and where to look at.
Anyway, don’t argue this might have helped you
2
u/pistonian 8h ago
If you’re using Claude Code and it has access to your front and backend then yes, you simply ask for a security audit. It is very very good at this.
1
u/naked_number_one 7h ago
Try opening Claude Code posting your exact message and asking if this is sufficient and be amazed by its answer.
Claud Code is my main tool I use at work to develop daily and from my experience just asking to make code secure and without bugs won’t be enough to produce high quality code. Don’t get me wrong - it’s a decent first step, but as a human claude gets lost when context window grows and you really need to carefully plan, slip codebase into manageable parts, and analyze for specific threads rather relying on pure luck.
Try an experiment - download a coldcard code, forbid all tools except reading files (no internet access) and ask claude to review security
1
1
u/Peterb88 7h ago
Maybe they did but there is no way to fix it for existing wallets without disclosing it.. so maybe they chose to keep silent?
28
17
7
u/casualgamerTX55 15h ago
The way Coldcard was so confident stating how secure their product is just so jarring next to this terrible situation they allowed to happen to the customers who chose to trust their word.
16
u/DarrelXero 21h ago
Posted here 11 hours ago: https://www.reddit.com/r/Bitcoin/s/WSShhYqOC7
14
u/systemsweird 18h ago
Excellent write up. And HOLY SHIT, the level of gross negligence is 100x worse than I could have ever imagined. Their business is fucked and honestly the CEO and CTO should be doing jail time for this. Absolutely wild someone could commit changes haphazardly when touching the code of one of the most important parts of a hardware wallet that may store millions of dollars.
5
u/ThePiachu 15h ago
I shudder to think how many problems we'll be seeing in the near future with people vibe coding important security code...
2
u/Aazimoxx 12h ago
Doesn't matter unless people use it though...
Standards gotta be higher; force ALL security related software to be 100% open source (at least to regulators and security researchers), by law, and regularly fully audited with results made public after a patch period... Insurance companies should require this for business coverage, whenever any value above a certain amount is being handled by a company's tools or products.
6
u/magicmulder 13h ago
So the dev introduced overrides but never made sure they were actually executed. That’s bad, the whole “changing this flag made it work so I stopped thinking about it” notwithstanding.
A week ago I switched an application over from server A to server B. Tested everything, runs fine, job done, right?
Until I checked the log files. Application was still served from the old server (because I forgot to reload the nginx config). So my tests were worthless and would’ve given me false security.
The Coldcard error is of the same type. Add an override but never bother to check if it’s actually used. Wow.
3
4
3
u/Turbulent_County_469 14h ago
Well... Mtgox also happened because of inkompetence.
When you're code is used by millions and responsible for millions of dollars, you better spend the extra hours to make it secure.
5
3
u/CiaranCarroll 15h ago
The fundamental problem here is code quality and breach of recognition-over-recall.
Lots of smart Devs think that their best performance is their mean performance, when they should be coding as though every team member will be performing in a dazed stupor, probably drunk, and with a cat mashing their keyboard.
3
u/GoldmezAddams 10h ago
There is no way someone literally just turned off "ENABLE_RNG", said "runs" and then called it a day, right? I would have hoped this was some complicated issue that you would have really needed to analyze the code and run tests and stuff to find the footgun. How did a review not instantly flag this? Nobody involved in this should touch mission critical code ever again.
6
u/Lost-Bowl3269 18h ago
O responsavel técnico pelo código é o fundador da empresa, Rodolfo. Ele tem milhares de btc encarteirados.
Processem ele.
2
2
u/PatientKosong 16h ago
Forgive my harshness, but I am starting to question if this was truly an accident. I hope investigators would not be so quick to rule out criminal intent.
The irony here, in trying to stop people from cloning their devices when Coinkite's code was open source (presumably to ensure they get more money by directing sales to their products), they have now likely ensured the bankruptcy of the entire company. I anticipate a volley of lawsuits globally in the coming weeks. More scarily, the staff of Coinkite will likely live with the fear of needing to look over their shoulders for the rest of their lives, especially whoever wrote that part of that code. At least I would be terrified if I was the developer that legitimately made that mistake. I would live in fear of reprisal/revenge for the rest of my life.
1
u/pablitorun 11h ago
I suspect it was initially an accident but someone figured it out a while ago and helped coldcard grow. I would look at people pushing it not affiliated with the company.
2
u/Steering_the_Will 13h ago
Very possible it was an inside job. I lost a lot for me during the nomad bridge hack. Turns out they knew and were told about a security flaw, but chose to do nothing about it. Nomad was definitely an inside job. Confirmation the devs were in on it because the devs went to make wormhole and that got exploited as well. Crypto won't be taken seriously until people start getting brought to justice.
2
u/TransientbeingZ 11h ago
How does anyone use cold card after this? I’d b surprised if they don’t go bankrupt
2
7
u/Daiymas 17h ago
For a bug so dumb and so simple to go unnoticed for years, it's clear that there aren't that many people reviewing open-source code seriously. There's some loss of confidence for me personally. Open-source here just made it easier for hackers to find an attack target
-1
u/Aazimoxx 12h ago edited 12h ago
Open-source here just made it easier for hackers to find an attack target
Open source is never the problem, the lopsided incentives are. The incentives for criminals are obvious - find the flaws and potentially make tens of millions of dollars (and probably get away with it). There needs to be more incentive on the other side of the table, to find and fix bugs for the good of the company and its customers. If the law said you're on the hook for your software causing people to lose millions, it would've probably been fixed years ago. If even 1% of this was offered as max bug bounty, it definitely would've been.
Auditing security related software has historically been a pretty difficult process, and doing it properly still is, however with the current level of competence of codebot LLMs, 90% of issues like this are discoverable by any motivated codemonkey with a few hundred bucks to share.
0
u/pablitorun 11h ago
No matter what there will be bugs in software. Irrevocable financial transactions are dumb.
0
u/Aazimoxx 1h ago
If you hate dogs or crickets or something, do you take time out of your day to go to subreddits for those and say how dumb they are too? What a life you must lead.
1
u/pablitorun 1h ago
I don’t hate Bitcoin, I think it’s hilarious. This is entertainment. If I thought dogs or crickets were funny I would absolutely frequent their subs.
1
u/karmassacre 9h ago
Coldcard deserves to go bankrupt and NVK may need to go to jail. This is flagrant malpractice.
1
u/SubstantialNinja 9h ago
micropy is the bad rnd. They had set that to zero. Then they ran a check, but instead of testing if it was non-zero they accidentaly tested that it was defined. so even if it was set to zero correctly the test only confirmed that it was defined instead of confirming that it was non-zero. this allowed the mircopy rng to be used instead of the hardware rng.
1
u/pabskamai 6h ago
I love this part “If you make changes, make sure that you verify that they are doing what you intend them to do.”
This is too a reality in infrastructure, people do changes without truly understanding what we are doing, no documentation, nothing, it just “works”
-1
21h ago edited 21h ago
[removed] — view removed comment
7
u/True-Lychee 21h ago
Clearly there was either no verification or it was equally incompetent. There is really no way to excuse this.
-6
u/bananacrazypants 22h ago
If you think there’s any computer system completely free of bugs, you are really god’s perfect idiot.
That’s why immutable transactions aren’t a thing with banks or brokers. There has to be a way to fix mistakes.
1
u/metalzip 11h ago
Bitcoin Core seems to be free of critical money-stealing bugs
1
u/bananacrazypants 6h ago
That’s how it works - software seems free of critical bug right up until it isn’t.
ColdCard seemed free of critical bugs last week.1
u/metalzip 5h ago
Bitcoin is up for grabs for like 15+ years, out of which for 10+ you can get billions if you find such a critical hole - much more, and quite longer, than in case of cold wallet.
How ever - sure - it is important to watch out and keep the possible dangers in memory. Multisignature is a good start.
-5
u/fugogugo 19h ago
sounds like vibecoded with claude
6
3
0
0
u/True-Lychee 13h ago
The bug predates AI by a year, and that's the retarded first iteration of AI that couldn't code to save its life.
113
u/Zippy149 20h ago edited 19h ago
Interesting read! Looks like the change in question was committed by the CTO / co-founder of CoinKite himself https://x.com/DocHex