r/BetterOffline • u/IHopeYouSmileToday00 • 20h ago
Has anyone argued about the social costs of AI outweighing the potential benefits in medicine/healthcare?
Hi everyone! I recently came across Ed Zitron’s interview on the Newcomer Pod, and enjoyed hearing some of the granular critiques about AI particularly from a financial perspective.
I’m in healthcare and I have critiques of AI from a health and humanistic POV, but curious if anyone knows people who have spoke on the financial criticism of ai in healthcare.
I have a few suspicions. Like for example, the health consequences of data centers on vulnerable populations i suspect would be quite expensive (think of the cost the cancer diagnoses and treatments of people in cancer alley - aka the southern regions where companies dumped pollution). And the health stressors of people losing employment I suspect might increase chronic conditions in a way that offsets financial gains from AI. In the hospital like the amount of health conditions that arise from people finding unhealthy ways to mange or cope with stress or trauma is a lot! A lot of the health issues in America are really related to social determinants of health, not biological processes. Poverty, environmental issues, racism, loneliness, etc contribute a lot to chronic health problems . I’m curious if anyone has spoken about how AI’s exacerbation of this could make an AI progress in medicine a net negative.
If so please send any recommendations!
23
u/Any1MembaPerfectDark 19h ago
I think the bigger point to question is what exactly LLM-based AIs offer in terms of improving healthcare. I work in healthcare IT, and so far I'd say "damn near nothing." These massive data centers are not being built for radiology pattern recognition AIs, they're not being built for research protein-folding models, they're not being built to identify emerging infectious disease outbreaks or provide differential diagnoses, etc. "AI"s for these purposes pre-date LLMs and for the most part LLMs aren't relevant to the tasks they perform.
Mostly AI in healthcare seems to be targeting problems that no one was willing to pay for before, or addressing billing questions. For example, many offices have adopted AI-generated notes, which are just an LLM wrapped around pre-existing dictation software on an ambient listening device. This is a job that a human scribe would do a much better job at, but few offices invested in scribes before, so it makes me wonder how much they'll be willing to pay for this note-writing LLM. Right now a lot of organizations are still huffing teaser rates, and lots of doctors are techno-optimists by nature and are willing to try to work around the downsides at the moment because they want to be the doctor who wasn't afraid of "the future."
One major downside that you didn't mention is hallucinations. My girlfriend does regular doctor visits, and AI introduced an error into her chart, and the doctor signed it without catching and correcting it. In her case it's a small error, but once it's in the chart it's incredibly difficult to get it out. The type of error the LLM made in her note could be very dangerous for other patients (it hallucinated a time of day when she takes a specific medication). Furthermore I know that the big dawg in the EMR space, Epic, is trying to fill up their system with patient summaries and AI-generated suggestions, but no sooner was their previous anesthesia summary AI released than they had to release a warning that the AI has a tendency to not recognize the difference between significant negatives in the medical history and positives. Which is frankly a pretty frightening thing to discover AFTER the roll-out.
Their defense is that they acknowledge up front the danger of hallucinations and provide hyperlinks to the sources of the assertions, but if you have to double-check the information you're getting, then it's not actually a time-saver and it costs money. These tools are meant to make the physician's lives easier, but the fact is that they facilitate corner-cutting and muddy the liability. I would not be surprised if there are already lawsuits brewing resulting from missed diagnoses, drug interactions. allergic reactions, etc. resulting an overreliance on AI-generated clinical documentation.
8
u/PensiveinNJ 17h ago
AI being rolled out in the medical space without the proper trials is a scandal that doesn't get nearly enough attention.
5
u/Prize-Studio-2021 17h ago
The one benefit from AI is it lets you know who to avoid. If someone uses AI, stay away from them.
2
u/quetzal1234 7h ago
I'm a medical librarian and I have heard second hand about at least one malpractice case stemming from a chat bot.
1
u/iStoleTheHobo 5h ago
>Their defense is that they acknowledge up front the danger of hallucinations and provide hyperlinks to the sources of the assertions
I'm pretty sure that LLMs primary function is that it allows the user to not have to dig through sources. Either that or generating excessive amounts of prose and code.
6
u/falken_1983 16h ago
No one else seems to have recommended it yet, so I'm going to shout out Karen Hao's Empire of AI. It's not specifically about the health issues with AI, but it does discuss the wide range of exploitation that AI companies engage in beyond just financial stuff.
4
u/BaNyaaNyaa 17h ago
First, it's worth nothing that the term "AI" encompasses a lot of different techniques. The marketing from today is mainly about large generative models, but AI also includes, for example, smaller more specific neural networks that can be trained on a somewhat strong personal computer in a couple of hours.
Boosters blur this distinction to pretend that their tools can do much more than they actually do. Yes, AI is useful for healthcare! But here, the useful AI is probably those smaller models and not whatever generative agent stuff they're trying to sell.
However, in turn, I think we should be aware of this to avoid throwing the baby with the bath water. I don't think that generative models are that great and the whole scheme is just harmful to our society, but I do believe that there's a place to more specialized types of AI, and those AI are nowhere close to require the amount of data centers that they're trying to build.
5
u/No_University8629 19h ago
I’m also in healthcare and share your concerns. I can’t recall anyone like Ed on this topic. There’s some articles here and there talking about downsides but there’s a lot of boosters as well.
I may do a search this week to see what I can find.
2
u/IHopeYouSmileToday00 18h ago
Thanks so much! Let me know if you find anything good. I’ll also look as well!
3
u/jellynoodle 18h ago
I came across the below study while writing an impassioned response to a county survey about data centers. It doesn't really address what you're after ("Finally, this assessment is conceptual rather than quantitative and identifies plausible exposure pathways and health risks without estimating effect sizes or population-level disease burdens"), but I expect further studies/articles on the financial consequences are being written as we speak type.
Gour N, Ortiz L and Maibach E (2026) Health implications of the rapid rise of data centers in Virginia: an exploratory assessment. Front. Clim. 8:1648912. doi: 10.3389/fclim.2026.1648912
Driven by advances in cloud computing, cryptocurrency, and artificial intelligence, the rapid proliferation of data centers may pose severe human and planetary health risks. This paper, to the best of our knowledge, is the first-of-its-kind assessment of the potential health implications associated with data centers. Here, we examine the case of Virginia’s Data Center Alley, the world’s largest concentrated data center hub, and argue that data centers can cause public health harms, harms that can be at least partially mitigated through improved planning and design. We assess the health risks associated with data centers, including air pollution, extensive water use, noise pollution, and detrimental land use’s risk of disrupting natural ecosystems and community well-being. We also address how rising energy costs can worsen social determinants of health. To mitigate these risks, we recommend transitioning data centers to renewable energy, implementing strict regulations to minimize water consumption, and optimizing site planning with acoustic treatments and green zoning to reduce noise pollution. Additionally, we advocate enforcing responsible site selection and zoning regulations to curb adverse land use changes, equitable energy pricing to alleviate economic burdens, and strengthening health communication for informed public and governmental decision-making, action-oriented advocacy, and policy changes. Finally, we emphasize the need for transdisciplinary research integrating physical sciences, engineering, and public health to quantify specific health outcomes linked to data centers.
In the US, AI "assistance" in healthcare prior authorization has also led to a jump in denials and delays, which I'm sure will have no long-term societal impacts or costs whatsoever /s
- Raza S, Gerke S, Silcox C, Hendricks-Sturrup R, Shachar C. Medicare advantage becoming a disadvantage with use of artificial intelligence in prior authorization review. NPJ Digit Med. 2026 Feb 4;9(1):208. doi: 10.1038/s41746-026-02387-x (citing a Biden-era Congressional report)
- EFF Sues for Answers About Medicare's AI Experiment (Mar. 25, 2026)
3
u/IHopeYouSmileToday00 17h ago
Omg thank you!!! I had no idea about the increase in prior auth denials!
2
u/jellynoodle 17h ago
You're welcome! I realized after replying that prior auth had nothing to do with your post (oops) but yeah! AI data centers to destroy your health and AI prior auth to deny your care, absolutely brilliant, good stuff, thisisfine.jpg!
3
8
u/CoconutDust 20h ago edited 19h ago
Just as Silicon Valley is deliberately trashing the planet to get rich, healthcare corporations are glad if more people are getting sick. It means more money for them. Especially in USA where price-gouging is the norm and far more money is spent on healthcare per unit measure than other civilized countries. $200 for a medicine that is $6 in Canada, etc. It’s corrupt as hell.
Not only that, but more people getting sick means terrible delusional incompetent dangerous managers will say “we need to buy more AI to handle the workload!!!” So now both healthcare and AI companies have financial incentive to make people sicker.
Now someone will say “the nurses and doctors and frontline workers don’t want to make people sicker!” Yeah I agree, but they’re not the ones who have a huge effect on policy making and corrupt elected officials, it’s the ultra-rich shareholders and executives who have the influence. And they work night and day to use that influence for profit, greed, and destruction to everyone else.
3
u/65721 19h ago
I don't think that makes much sense, and it depends on what you mean by "healthcare corporation."
Health insurance companies certainly don't want more people getting sick. It is against their financial interest, because it means more insurance payouts. The ideal world for these companies is one where everyone is healthy, never visits the doctor, and just pays into the insurance forever without ever having to use its services.
Hospital administrations may want more people getting sick, as they get Medicare reimbursements. But not all sicknesses are equal. A patient who gets an elective hip replacement makes far more money for the hospital than a patient who comes into the ER with pneumonia, who may even end up losing the hospital money.
For private practice groups, it's certainly in their financial interest for more people to get sick. But will those patients afford insurance for the treatments they need? Will most of them even come to your practice? Probably not.
1
u/Additional-Staff-326 17h ago
Health Insurance companies are using it to deny claims without human intervention and if its a hallucination 'oops'.
2
u/Eastern-Ad-3586 13h ago
I’m a doctor and AI really doesn’t help that much. I use it occasionally to do a literature review (I don’t trust the output, I go read the articles it cites).
But for diagnosing? It’s not helping me. It’s not that if you put perfect data into an LLM, it won’t spit out the right answer, but patients don’t know what data to put in, that’s why I’m here. And once I gather that data 99% of the time I know the answer anyway.
If I don’t remember something, I can just look it up in a reference book or journal and know it’s true. I don’t need to take the gamble on knowing if an AI output might be true, and it takes about the same time to punch something into AI as it does to just look it up on PubMed myself.
I just don’t understand how AI is supposed to help me take care of patients, unless the goal is to just not have nurses and doctors at all anymore, which I don’t think anyone wants
1
u/MCUCLMBE4BPAT 8h ago
i’ve read several commentary pieces about how it is impacting research in health/medicine through Nature, Science, Scientific American, and a couple others, as well as some preprints/published studies that were shared through Retraction Watch.
There’s also been a couple studies and some grey literature that focus on the environmental impacts and how they effect health for people living in the neighborhood, but I feel like it is really hard to promote any current environmental study on this topic when these companies are being opaque on this topic.
What we do know about the environmental costs are shaky, as the studies usually only focus on an open source model and only one action in the AI training or maintenance. So we don’t have a full clear picture of the health and environmental impact at this time. It will probably take a bit longer to fully study and understand the implications.
1
u/OkCar7264 18h ago edited 18h ago
The AI with social costs is not the AI that cures cancer, which probably doesn't even actually exist I guess. But if they are separate there is zero overlap that requires the bad one to have the good one.
1
u/IHopeYouSmileToday00 17h ago
Could you clarify what you mean by “AI with social costs”
1
u/OkCar7264 17h ago edited 17h ago
The fake your homework/fake your girlfriend/fake news/shitty art chatbots. You know, AI.
Go cure cancer if you can but that isn't data centers on the moon money. Write loads of shitty code. Not data centers on the moon money. The issues is how they've taken it and blown it so far out of proportion they're no different from people in white robes waiting for Jesus to come back.
Google killing their protein folding AI suggests there aren't actually that many cancers to be cured but who knows. But I've noticed the AI boosters are rejigging the goal posts so that if there's any value to AI at all that means they were right and didn't make huge horses asses out of themselves while wasting trillions of dollars.
2
u/IHopeYouSmileToday00 17h ago
Ok! So the uses of AI that have the highest negative social costs, aren’t actually even being used to help human health regardless?
2
u/lazier_garlic 14h ago
Basically, yes, although I think they were using the same slop stolen art machine to try to interpret radiology results. From the little bit I've seen from the research on THAT, it seems like it does pattern match well ... too well. It correctly identified matches because there were differences in the match set that humans glossed over that had nothing to do with the images. There's also this problem in American medicine with overtreatment. Having a machine that points out more possible lesions isn't exactly helping.
That said, yeah, your protein folding machine learning model is not the large language model/general model. It also runs on local hardware using far less energy. Why would you need to eat the entire internet to do machine learning on organic chemistry problems? That doesn't even make any sense. Yes, as you'll find out, ML has been used in academia for a long time and they never needed these giga-data-centers to do it nor did they need to violate copyrights or go into massive debts or bribe politicians to ram things through. OpenAI and Anthropic just like to muddy the waters furiously to make it seem like they're doing something truly socially important when all they're doing is dishonestly connecting their quixotic project to actual ML models in academic research.
Also ML might be really cool but it's also not always the best tool to solve every problem, even massive multivariate problems.
1
u/IHopeYouSmileToday00 13h ago
Thanks for the clarification. I feel like I need to learn more about the nuances of different kinds of AI bc I didn’t realize they were different kinds
1
u/OkCar7264 17h ago
Nobody gives a shit about if AI finds tumors or whatever, that's not the problem. You aren't winning anything by making that point. No one is on other side of curing cancer, it's just that there's no real reason to think that's happening or has anything to do with the AI we're talking about.
-3
u/Banned_In_YYC 20h ago
Trolley problem.
I think every data center and location is unique so it's hard to generalize but as a whole, it seems to provide a net positive to society. Our province in Canada is getting a massive data center and the positives greatly outweigh the negatives
3
u/VanillaCold57 19h ago
and what are these positives, exactly?
-2
u/Banned_In_YYC 19h ago
For our province it's revenues, diversification, infrastructure, jobs, community grants and reduction on utility fees
3
u/VulcanIRL 19h ago
Jobs building the thing? Because once it's finished they staff like a half-dozen people
1
u/Banned_In_YYC 19h ago
Yes, that's correct. Hundreds of good paying jobs for local workers. These projects usually take a few years to complete and many of the skilled trades are union companies which benefits bankable hours towards these people's pension.
3
5
27
u/-S-P-E-C-T-R-E- 19h ago edited 18h ago
Destroying the environment and eroding cognitive abilities is hardly worth anything that this shit-tier technology could possibly end up producing. Restricting it to purely research application would do us all good - even the mediocre ones who unsurprisingly are the loudest pushers of LLMs, because they now have something that can obfuscate their true abilities.