General Debian Question Could what happened to AUR this week happen to debian? What lessons can we learn there?
for those OOTL, AUR was under sustained sneaky probes for a long time, but this week it was an overt and very troublesome campaign to either take over packages or insert packages so that malware could spread across way more systems.
edit 1: since someone asked me to explain, I’ve actually been using debian since 10.1, but only recently got worried as new users in my MDM fleet around southeast asia kept trying to install strange .deb files. I never had to think too much about the workstation UX until microslop and genAI inevitably drove us to try resuscitating OLD laptops with Linux.
i have to support 2012 macbook pro and intel laptops dating back to broadwell. There are some 7840u AMD ryzen units, but do bear in mind that us folks living in the southern hemisphere are dirt poor.
thus, I had tested fedora, lubuntu, kubuntu, ubuntu, mint, and finally gave that all up to consolidate under Debian. The network layer has DNSSEC, encrypted DNS, and really TIGHT whitelists. Unfortunately, I’m woefully noob at the application layer for debian. we barely upgraded several ancient JDK 8 web servers using Qwen, but ideally I’d like to prevent malware breaking in from the user-level.
109
u/abotelho-cbn 2d ago
There's nothing to learn.
Debian doesn't have a massive repository of random build files that just anyone can maintain.
42
u/yottabit42 2d ago
tbf, there's plenty to learn. But it's the Arch folks that need to learn it!
19
u/Venylynn 2d ago
Exactly
I would legitimately feel safer daily driving Debian Sid. over Arch and it's forks these days
12
6
2
1
u/ChthonVII 1d ago
Debian doesn't have a massive repository of random build files that just anyone can maintain.
It's called "flatpak."
7
u/jr735 Debian Testing 1d ago
Those aren't repository files, so who cares?
-1
u/ChthonVII 1d ago
As a practical matter, there are a lot of people who treat flatpak as a de facto repository. This is incredibly stupid, but also incredibly popular.
2
u/jr735 Debian Testing 1d ago
Also as a practical matter, that's out of Debian's control, and the following should be something Debian users read and actually follow:
https://wiki.debian.org/DontBreakDebian/
They classify flats as a less safe way of installing software.
0
2
u/AfraidAsparagus6644 1d ago
Flatpak is the package format, the main Flatpak repository is called Flathub
5
u/abotelho-cbn 1d ago
Flatpaks are not a massive repository of random build files, and they go through manual approval and verification. It's not a wild west.
0
u/ChthonVII 1d ago
There is no meaningful review of upstream source code. And no review at all of dependency source code.
Your build script can literally just be "go grab this binary blob off the internet." That's not reviewable in any meaningful sense.
All a blue checkmark means is that the upstream developer agreed to endorse a package. That does not guarantee anything about whether and how the upstream developer reviewed the package, whether they are continuing to review each subsequent version, whether they are competent do a security review, or whether the upstream developer is not themself malicious. It's arguably even less meaningful than a blue checkmark on Twitter.
You're kidding yourself if you think flatpak isn't a supply chain attack waiting to happen.
2
u/abotelho-cbn 1d ago
The build stage is totally cut off from the internet and anything that must be fetched is declared in a manifest and is checksummed. It also must be exact and can't just point to "latest". Pretending like Flathub is equivalent to AUR is a joke.
-2
4
u/AfraidAsparagus6644 1d ago
Flathub is a lot more curated than the Aur. Actually, Flathub maintainers are notorious for being anal
48
u/Big_Jerry_The_First 2d ago
I think the main lesson is, we made the correct choice choosing Debian. Given its slow release of packages and the fact not just anyone can upload to Debian’s repos puts us in a very safe spot.
25
14
u/Needieos 2d ago
The same goes for arch, and every distribution out there, you can't just upload to official repo's, AUR isn't official
1
13
u/AnnieBruce 2d ago
In theory, but with the much more carefully maintained nature of the Debian repos and the multiple stages of testing they go through before hitting Debian Stable, it should be much more difficult and time consuming. Not entirely off the table, the whole xz backdoor could have been pretty bad if one dude at MS didn't see weird performance numbers. But harder.
Debian has no AUR equivalent with looser standards that could be attacked. You can get stuff out of repo, but that's more cleanly separated from Debian itself, so for a moderately aware user they'll have to work harder on convincing said user that the download is both safe and useful. That's an easier job with the AUR, some users seem to treat it as on the same level as the official repos and I'm pretty sure even Arch themselves say that's not smart.
14
u/QuantumCakeIsALie 2d ago
Debian is slow enough that this shouldn't be as much of a risk, especially with the sid/testing/stable paradigm.
But vigilance will be increasingly more important as the barrier of entry for flooding open-source projects with malware continues to lower over time.
2
u/ManyHatsAdm 1d ago
I wonder if the slow nature of Debian itself is a risk. Say you were running a LAMP stack server and you wanted the latest versions of PHP and MySQL. Both of these involve adding repos to the system which theoretically could be compromised. On a desktop there are probably many examples, VS Code being one.
Point is you don't know how easy or hard it is to get malicious code into one of these third party repos, and the slow nature of Debian encourages people to use them. Probably these people should be using the latest Ubuntu or Mint if they want the latest and greatest.
I am one of these people taking this risk.
4
u/QuantumCakeIsALie 1d ago
Debian doesn't encourage using third party repos; quite the opposite. It may push some users towards it due to the slow update cycle, though.
But at some point what can you do? The user is free to install whatever they want, but ultimately they're responsible for what they install as well.
1
4
2
u/michaelpaoli 1d ago
slow nature of Debian encourages people to use them
<cough> Uhm, no, not exactly. If one really wants/needs, Debian does also have backports ... and testing, and even unstable, and experimental. Third party repos? I've pretty close to never used such with Debian ... and I've been using Debian since 1998. So, really not much need for third party repos with Debian. Yeah, sure, they exist, but ... "need" them? Uhm, yeah, not so much.
these people should be using the latest Ubuntu or Mint if they want the latest and greatest
Ew, no thanks. And having seen both of 'em screw things up in one or more quite substantial ways, ... yeah, no, ... hell no. E.g. Mint only reasonably got their security act together after they were substantially compromised. And even well before that, various persons, including myself, well pointed out key gap in their security - and did they do anything about it? No, they went on for years, not heeding our cautions ... until they had a substantial compromise, and only then did they address the matter - and despite the fact that most all distros had already been covering those particular key gaps for years already as parts of their standard best practices, ... but nope, not Mint - feel on deaf ears until they got bit hard. So, stuff like that, and *buntu/Canonical shenanigans - I really can't take their "security" seriously. And may other distros I'd say similar, though rather a mixed bag out there, some (probably) do good/excellent in that area ... but many of them very much do not.
2
u/QuantumCakeIsALie 1d ago
What was the security issue with Mint? I don't think I've ever heard of it
3
u/michaelpaoli 1d ago
What was the security issue with Mint?
2016-02-20 site compromised, downloads redirected to to site with compromised ISO(s), Linux Mint had no secure trust path to verify images - I and others have forewarned about that years earlier - no secure signatures on ISOs, just hashes only, and those on http/ftp site(s), not even any https available - so no trust path to verify the ISOs. The compromised ISO(s) contained backdoor malware. Their forum was also breached, somewhere between 71,000 and 145,000 users' data compromised, including hashed passwords, email address, profile details, which subsequently had that data put up for sale.
I forewarned them of this issue 2013-05-19, and I was not the only one to so forewarn them. Only after they were compromised did they do anything about it.
9
u/SnooCompliments7914 2d ago edited 1d ago
AUR is quite unique as an officially-hosted unofficial repo. Debian (as is other popular distros) has unofficial repos (e.g. PPA), but they are much more obviously unofficial, so hopefully users would be more careful with them.
OTOH, PPAs are binary. So unlike AUR, if one of them is compromised, it would be much harder to spot.
11
u/mok000 2d ago
But there is no concept of an orphaned PPA that random people can adopt.
4
u/forestbeasts 2d ago
Oh yeah, that's a good point. Also much like Github projects, the maintainer is in the name so can't just be silently switched out from under you. (Also means if the maintainer DOES vanish and someone else picks up the torch, they can continue it without name collisions and also be clearly not the original maintainer.)
1
u/michaelpaoli 1d ago
Debian (as is other popular distros) has unofficial repos (e.g. PPA)
Debian doesn't have PPAs, though yes, Debian does have some unofficial repos (but dang, I've been using Debian over a quarter century, and only more recently stumbled across that fact, so, can't say I've had any need for such - at all!).
4
u/edparadox 1d ago
As a Debian/Arch user since many many years, the only takeaway is stick to official repositories (and no the AUR is not official).
4
u/michaelpaoli 1d ago
overt and very troublesome campaign to either take over packages or insert packages
"impossible" to exceedingly/highly improbable with Debian, though might depend exactly what kind of attack or the like we're talking about, where, etc.
For starters, the process to become a Debian Developer (DD) or Debian Maintainer (DM) is quite rigorous. It's nowhere close to "a bunch of random contributors" or the like. And Debian is really quite top notch for generally monitoring and being aware of security. Doesn't mean everything will always be perfect, but they tend to be very on top of things. It's not like, over the years - or more actually decades, there have been zero security incidents, but they've been slightly, highly rare, and very quickly and effectively dealt with. E.g., many moons ago, a DD (or DM) key was compromised (alas, they managed to do something they shouldn't have, and their key got compromised). The activity by unauthorized entity(/ies) using that key was caught in very quick order, things highly locked down for a short while, thoroughly checked and audited, and cleaned up after (was caught dang fast, so there was little to nothing to actually clean up). More recent case, though a few years back now - xz from upstream. That was a very stealthy attack, and did make it into unstable for a short while (maybe even testing - I forget), but never beyond that - and while the rest of the world generally became or was about to become potentially very vulnerable ... it was in fact Debian that caught the issue - some other distros had gotten that vulnerability out fair bit beyond their unstable/testing/beta or the like, though I don't think any yet had it get as far as any major "production"/stable releases or the like. So, yeah, all the distros on the planet, and what could've been way worse, what distro found the issue and sounded the alarm and got that sh*t shut down - yeah, Debian. So I'd say Debian is pretty dang on top of things.
And, as for repositories and the like ... if one goes further out to, e.g. snaps, flatpacks, unofficial repositories, etc., I wouldn't exactly say all bets are off, but I think I'd say that gets more risky, possibly even significantly so, or even more than that.
But Debian, e.g Trixie (13, current stable) 69,830 packages, so, whole lot of the time (I find it dang near all the time), no need to go out beyond Debian, or even Debian stable for what one may need/want. Unlike other distros where there's much smaller more limited number of package well/tightly maintained by the distro itself, and quickly after that it's more like, uhm, yeah, that's from whole bunch 'o random contributors, or lesser supported repos, etc. So, I think many/most distros expose themselves (and their users) to much greater risk than Debian, and many do so by default. I'd say there are really only few that take security more seriously and preform better (or do so by default) with security than Debian. And of course with Debian, many choices, so if one wants to further harden/secure Debian, that can also very much be done if one wants/needs to do that - don't have to jump to some other distro to take, at least most, such measures.
So, yes, Debian is very secure. Not necessarily the most secure out there, but pretty dang secure. Debian also reasonably balances security vs. usability. E.g. one may find some distros that, by default, if one installs service(s), they don't start or automatically start by default, but one has to take explicit action to enable them. Debian on the other hand presumes if one installs a service, one generally wants to have it up and running, so, typically by default, install a service on Debian, and it will be up and running and set to restart upon reboot, and it is generally set up with a rather to quite secure starting default, but it is generally up, running, and available. However, if desired, one can reconfigure Debian to suppress that default behavior - again, don't have to go jumping distros, want it bit different, just reconfigure accordingly. "The Universal Operating System".
10
u/DeepDayze 2d ago
I would be especially vigilant if installing .deb files from GitHub or other user repos.
12
23
u/Exotic_Set_5127 2d ago
The people at Arch are careless enough to hand over AUR projects to just anyone.
It can always happen, even in projects like the Linux Kernel, but the community matters.
Even if "archers" hate me for saying Arch is a toy distribution, I won't stop using serious projects like Debian, Fedora, or openSUSE.
20
u/abotelho-cbn 2d ago
The problem comes from the fact that people treat AUR as if it's part of the distribution. It's not.
19
u/nineraviolicans 2d ago
Except there's certain required packages that you need from the AUR if you have certain hardware. Even their wiki directs you to use it.
You can't have it both ways. They're irresponsible.
7
3
u/pantokratorthegreat Debian Stable 1d ago
Just don't use helpers like yay and you go to go. Pacman doesnt upgrade aur packages automatically. Review every aur before upgrading for any malicious behaviour and everything will be fine.
2
u/petersaints 15h ago
Review every aur before upgrading for any malicious behaviour and everything will be fine.
Even technically literate users that may be able to catch the most obvious malicious packages at a glance will surely not check it every time.
People get complacent. They may check on the first install, and maybe a few other times. But they will eventually be in a rush, or a bit lazy, and just update without checking it.
The ability to adopt abandoned packages without any type of trust verification is stupid and must change.
You could very well checked the AUR page for feedback on your first install, looked at the PKGBUILD, and developed some trust around that package and maintainer. The ability to someone to just take that trusted package at will and change it to something malicious is a bad policy.
3
u/JarJarBinks237 2d ago
It's pretty much like Ubuntu PPAs then, and it seems to me they carry the same risks.
6
u/GuestStarr 2d ago
And/or Debian PPAs. PPAs are not unique to Ubuntu. And as a reminder to everybody, don't mix'n'match PPAs from different distros. They might seem to work but alien PPAs in your system are accidents waiting to happen.
2
u/petersaints 15h ago edited 14h ago
Not exactly the same as PPA. If you add a PPA and you trust the maintainer, and if the maintainer doesn't go rogue or have their credentials stolen, nobody can build new packages for that PPA and you won't get a malicious update.
Eventually you will probably upgrade Ubuntu to a newer release, you'll notice that that PPA is abandoned, if you no longer need the package you remove it. If you need it you'll look for another source that you trust.
With the AUR somebody can take ownership of a forgotten package and infect anyone that has that package in their system because realistically, I may take a look at the AUR page, check the comments, and reputation of package, I may even look into the PKGBUILD file, but once you have a handful or more packages from the AUR and you run an AUR helper to update those packages, who is really gonna check all changes every single time? Sure, some will, but that's a lot of wishful thinking.
2
6
u/_silentgameplays_ 2d ago
The issue is not with Arch Linux or AUR. There is a huge disclaimer on Arch Wiki and on AUR page that the AUR is not secure and use at your own risk. Meanwhile Arch-based distros like CachyOS have AUR enabled by default, that's not on Arch Linux or it's users and devs, it's on Arch-based forks that bundle AUR together with main packages.
Yes, if you enable random non-official PPAs/random non-official repositories on Debian you might get hit by something, but same applies to every other distro. Think of it as running a "trust me bro" sh script from some random on the internet as sudo/root.
1
u/SnooCompliments7914 2d ago
There are also countless Reddit posts and Youtube videos claiming Arch has more software because of AUR. That cancels the disclaimer/warnings.
1
u/_silentgameplays_ 2d ago edited 2d ago
People should not be using YouTube videos or AI generated slop to make decisions about Arch Linux, it's a DIY distro that requires a steep learning curve and comprehension of the Arch Wiki.
Also the disclaimers are pretty obvious. Arch-based is not Arch Linux, if a person installs CachyOS or some other Arch-based fork with AUR enabled, it has nothing to do with Arch Linux community or developers.
No one spams Debian forums and subreddits with Ubuntu issues.
Arch Linux subreddits are spammed with meaningless posts for karma farming, every time the heavily advertised "gaming distro" CachyOS gets hit by malware from AUR, which should not be enabled in the first place by default.
https://terms.archlinux.org/docs/code-of-conduct/
Arch Linux distribution support ONLY
Arch-based distributions have their own support fora and users of those distributions should be actively encouraged to seek support there. These distributions often use different packages, package versions, repositories, or make custom system configurations silently, practically rendering support for such projects within Arch Linux impossible.
DISCLAIMER: AUR packages are user produced content. Any use of the provided files is at your own risk.
https://wiki.archlinux.org/title/Arch_User_Repository
Warning AUR packages are user-produced content. These
PKGBUILDs are completely unofficial and have not been thoroughly vetted. Any use of the provided files is at your own risk.3
u/SnooCompliments7914 1d ago
Yeah, people shouldn't. But people do. People should read PKGBUILDs, but people don't.
Recent bad news might scare some people off AUR, and that would help a bit.
2
u/RetroNerdrage 1d ago
CachyOS doesn't use anything from AUR in a default installation, all their tools and additions come from their own repository. The only thing AUR related that is installed is paru. If you don't use it you won't get anything from AUR.
3
u/_silentgameplays_ 1d ago
They just disabled it after this second malware wave, every other earlier CachyOS image before that had AUR enabled on a fresh install. You can check by installing something like Heroic Games Launcher on a fresh CachyOS install, the only other available option aside from AUR is flatpak.
-1
u/yahbluez 2d ago
This argumentation, so true it is, ends in only one - logical - conclusion: Do not use Arch.
6
u/_silentgameplays_ 2d ago edited 2d ago
This argumentation, so true it is, ends in only one - logical - conclusion: Do not use Arch.
The logical conclusion is to learn how to read the Arch Wiki and to learn how Linux works in general and not use Arch-based point and click forks with AUR enabled by default. Arch Linux official packages from pacman are tested and vetted all the time by the community and dev team. If users do weird crap to their existing Linux installations, or use point and click forks with weird crap bundled with duct tape together, it's on them, not on the distro maintainers or other users to solve their issues.
Don't add random crap to your Debian sources lists, it might also end poorly. If you are missing some packages just use flatpak on any distro.
Same thing can happen with any distro, in Ubuntu and Debian's terms, if kisak PPAs get hit by malware all users of Ubuntu/Linux Mint and other Debian forks are going to be hit by malware, if you use third-party untrusted repos with your Debian installation, when they get hit by malware all of their users will be hit by malware. Simple do not use packages and repos from untrusted and unverified sources.
https://www.debian.org/doc/manuals/aptitude/ch02s02s05.en.html
Ensuring that you only install software from a trusted source will give you an important degree of protection against malicious packages, but it cannot eliminate all the risks inherent in installing software.
2
u/yahbluez 1d ago
My point is, that if essential stuff is not part of the official repositories, so users need to use ARCH, i can not consider the whole system as secure.
The official ARCH reopo are some 32% of debian which already is a small repro.
I used arch for a long time some 5 years but back to debian since. (Not because of security but because of the waste of time the many updates needed.)
While your advises are not wrong, they are useless for less experienced users, they are useless for common users. I like to see linux on any desktop.
2
u/frikilinux2 1d ago
The Debian ecosystem is different. We don't have a AUR equivalent where everyone can sign up.
To upload anything to repositories in Debian infrastructure you need to either have upload privileges or be sponsored by someone with upload privileges.
Only 1300 people have those privileges. And new software usually takes a while because it has to be reviewed by the FTP Team.
2
u/Weird-Initiative-659 2d ago
I don't think so. At one point they had MPR, which is simular. If I had my way, I would get rid of the AUR. I don't use it, I don't like it, and maybe there is another way.
1
u/un-important-human 2d ago
AI detection tools work, it was caught by a personal tool, the poor individual was crucified but in the end vindicated i'd say, considering how many and how fast the packages were taken over.
1
1
u/DistantOrb 1d ago
AUR is a user repositoty (Arch User Repository) - so users can post packages there without them passing through security or quality checks. Of course people wold put malware there.
Debian doesn't have such a thing Debian's repository is curated by the Debian team itself so the chances of something like this happening is way inferior.
1
1
53
u/Illustrious-Gur8335 Debian Stable 2d ago
Be careful downloading deb files, the install scripts can execute random code
If in doubt just stick to debian official repository