I have yet to even see anyone claim that software can't constitute a moat anymore but I expect that there are people saying that. GitHub has a huge non-software moat in the form of network effects, brand recognition, and good will.
The hardest part isn't making a "forge", it's making money off of making a forge. Getting a sufficiently large number of paying customers.
If GitHub doesn't get their quality issues under control someone probably will manage to breach that moat and take over the market. It's not like there's a lack of competitors (Pre-llm: GitLab, BitBucket, Gitea, Source Hut, etc. Post LLM: Tangled, esrc is promising something any day now. Probably more in both camps that don't come to mind).
>> SaaS plays have no moats anymore
> I have yet to see literally anyone say this.
I've heard a lot of people say this...including myself after a root beers. I think you just have to look to any time an AI feature is announced and some related companies stock price crumbles. Just google something like "stock price tumbles after anthropic announces" or something like that.
Firewood and heating oil isn't cheaper, it merely has lower upfront cost in exchange for a higher total cost. An efficient governance system (whether that's capitalism and banks with loans or renting out the hearpumps or a centrally planned replacement program or anything else) would figure out the financing and save the system money by updating.
Technology can make the incentives even larger. Excess money can make it easier for the governance system to reach the solution. But it's at the point where without any improvement to either an ideal system would figure out how to make the switch happen.
There is also a minor incentive problem here, mainly that a landlord can/will often offload the running costs, Nebenkosten, to the renters indefinitely.
That means they are sometimes economically incentivized to choose an option with lower initial cost but a higher running cost. Governments can/do bend these incentives via taxes but it can be hard/expensive to renovate old complexes (and that part cannot directly be offloaded by the landlord).
If you're an individual with an apartment you don't have the choice to drill.
If you're building the apartment building you have the choice to drill for the entire building, and the number of units that benefit mean this is much more cost efficient than with single family homes.
I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks.
See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally messed up).
And in interpreting x-ray's human radiologists actually do just look at the x-rays. In the context the article is discussing the human doctors don't just look at the notes to diagnose the ER patient. You're asking them to perform a task that isn't necessary, that they aren't experienced in, or trained in, and then saying "the AI outperforms them". Even if the notes aren't accidentally giving away the answer through some weird side channel, that's not that surprising.
Which isn't to say that I think the study is either definitely wrong, or intentionally deceptive. Just that I wouldn't draw strong conclusions from a single study here.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well, and let's be realistic, each time I've seen a doc the last few months (and ER twice), each time they were using ChatGPT btw (not kidding, it chocked me).
So I’m genuinely curious:
What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it.
Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
So much of what I know from women in my life is that the human element of medicine is almost a strict negative for them. As a guy it hasn't been much better, but at least doctors listen to me when I say something.
One of, if not THE biggest challenge in getting treatment is getting past insurance rules designed to deny treatment. This is much, much easier when you're able to convince a doctor (and/or trained medical staff) to argue on your behalf. If you can't get those folks to listen to you, that's probably not gonna happen. You might have to go through several different practices before you find a sympathetic ear.
Now replace some / all of those humans with... A machine whose function also needs insurance approval.
Sounds like we need to dismantle and replace this broadly dysfunctional system at multiple points. It's not like the US insurance landscape is anywhere close to the best way of handling healthcare if you look at many places in the world.
This is because "dismantle and replace" (or perhaps in other words, "defunding") is not a serious, viable solution to many of the societal issues we face.
Things were ruined slowly. They unfortunately will need to be fixed very slowly too.
I don't think that's going to work. We need broad political change and then that has to work rapidly to legislate this. I don't think slow and steady has done anything but lead to the decay our institutions over the last 70 years.
I think that both this and GP are misguided. The pace of societal change in a given direction is neither inherently proportional to the pace of change in a different direction (GP) nor is the pace part of the direction (you).
You have to engage with the specific historical events/factors that led to the direction and the pace in order to change either. Broad statements like "society is big so change has to be slow" are just as unwarranted as "slow change results in decline".
There's a correct answer to "how quickly can change in a new direction be achieved". It will probably only become known after the fact. It will certainly not be model-able as a function with variables for "progressive or not" and "speed of change".
My argument is more along the lines of "slow change has resulted in decline observably for the time period I have observed it and we should try catalyzing something else"
I grant that whether that winds up being fast or slow even if the attempt is intended to be fast is out of my or anyone's hands for the most part as the system dampens that barring total collapse and chaos :P
> They unfortunately will need to be fixed very slowly too.
this can work until you hit a crisis point; i think one issue is we are sliding faster in the wrong direction (increasing bureaucracy, increasing fees, wait times, overwork etc) so "slowly" can work but only if its "fast enough" if you get what i mean (people are really suffering out there)
Of course they do. Most of their platform is built on [appearing] to repudiate coastal elitism and left wing dogma in higher education + globalism with a healthy dose of fuck you because you're you.
And I graciously waited and allowed them to do things that will take decades if not more to repair before deciding they were irredeemable. I had hoped a middle ground and bipartisan ship would be reached, but it's clear to me it won't be. We do not inhabit the same universe at this point, the disdain is mutual.
You’re acting like I’ve always thought about them like this or like I haven't spent years observing and thinking about this to come to my conclusion. You'd do well to listen to your own words about theory of mind. I was raised conservative I voted for Romney. I'm a fan of many of the political platforms they run on now (minus originalism, removing bodily rights, religion), but in practice they do not walk their own talk. The wars, the spending, abandoning neo-liberalism except in word the blatant corruption and disdain for the positions they hold and how they appear on the world stage.
No, I’ve watched their actions for 15 years and moved ever closer to the position that I have nothing in common with them even being ideologically close to a version of their party from 20-30 years ago and they do so blatantly want to destroy the middle class, health, and wealth for anyone outside a small oligarchic class.
I'm pissed because they wear a lot of my ideology as cheap dress to fuck someone.
You are describing a set of dynamics that lead nowhere other than violence and total and complete breakdown of the polity. If you are correct, then nothing matters, everything is fucked. You won't get what you want, but neither will anyone else.
My preferences, while possibly futile, are least an attempt to not just accomplish short term goals but to fix the broken dynamics of the system. That is, in my opinion, far more important than literally any particular policy goal. Policy progress is pointless in a broken system, so fix the system first.
It's possible that my view of focusing on fixing the system, restoring institutions, erecting new guide-rails in places we have observed that the old ones don't work, etc. won't work. But at least it has a chance of producing a good outcome. A good outcome literally can't come from the kind of political behavior you describe. You want your side to seize as much power as it possibly can when it wins, enact as much "good" as it possibly can in however long it can maintain it's grip before the political tides inevitably swing and you lose power again. You don't seem to realize that this is what we have been doing for at least several cycles now. And what we have seen is that the next administration just tears up the progress, does the same thing except in the opposite direction and even harder, and does what they view as "the good thing" and which your side views as nothing but unmitigated evil (the same way they viewed you and yours when you were in power), and so the both sides have accomplished nothing but pushing the pendulum a little bit further, giving it a little more momentum, and shredding up the social fabric a little bit more.
I'm not so naive as to believe that it is possible for just one side to say "no we won't do that, we will unilaterally disarm". But I am of the belief that, if one wants to pretend that one is "on the side of good", that the only rational action is to, when granted power, to spend as much political capital as is possible to slow down the pendulum, tear back power from the bloated executive and the federal branch more broadly. Stop trying to enact your political project and instead make your political project nothing other than the restoration of the norms and principles of the constitution.
This is not something that has been tried and failed. it's the opposite of the past 50 years of federal political dynamics. What has been tried is your plan of "fuck the other side, they are evil, just do what our base wants and ignore consensus and norms".
It doesn't work, it won't work, and it can't work. It's destroying the country.
From my perspective, you are no better than the side you hate. You may want different policy goals, but both you and your polar opposites are collaborating on a shared project: the destruction of the country.
I want my party back and I want to cut out all the garbage that has infested it. Sometimes that requires taking an actual stand and staying firm to it. Middle road nonsense like what you're suggesting is impotent when one side has so clearly decided to be against it.
Edit: And coming back to this later I need to be clear the left also needs to be swept out. I think our institutions in general need to be reworked. Not replaced entirely, but it's clear they don't survive contact with people who would abuse them for their own ends nearly as well as we had hoped.
It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.
No offense, but this comes off as passive indifference and while I've heard people say things like this all my life it has broadly resulted in watching 30 years of societal decay. I can't help but think this is wrong.
We should have stacked the courts ourselves, brandished executive orders etc, had some spine.
Edit: I think I need to make clear my thinking that the right has selectively destroyed institutions and levied them in other areas where it makes sense for their agenda. It's not been wanton. So when I say leverage the playbook it's not a one sided act of destruction.
Let's say, hypothetically, you had two political parties — a "destroy the current institutions" party, and the "preserve the current institutions" party.
The latter might notice the former having an easier time, but "hey, it works for them" is the wrong takeaway. Commit to the hard work of building resilient institutions; don't join in the destruction because it's easier.
There's also an element of "Never (...), they will drag you down to their level and beat you with experience."
The republicans are building things not just destroying is the point. It’s just stuff you wont like. This is why Im not a democrat. The left hasn't been able to effect change or be useful ever [my entire life], sure loves to moralize though.
Strongly agree. I think some (not all) of the Trumpian playbook can be wielded very effectively for non-conservative parties, for a few reasons:
- Some executive orders are always flipped as soon as the opposition takes office, but some unilateral changes are much harder for a cyclical/pendulum-swing opposition season to reverse than they are to emplace. We don't know which are which yet. The return-to-office mandate for Federal workers is probably one that'll have a lasting effect--even if un-done in the future, the average prospective Federal worker will consider the job as something that has a significant likelihood of requiring in-person work if the political winds change and that EO is restored.
- Some things really do get permanently addressed within an electoral season, if you have the guts to shotgun through enacting a solution to them. The withdrawal of most U.S. troops from Afghanistan under Biden is a good example of this. So is the "Fork"/RIF/firing wave of Federal employees under Trump. I'm not saying those are both good things, but they aren't "reversible" in the sense that, say, the Global Gag Rule was endlessly reversible.
- Success follows success, as well. Part of the reason that momentum holds such a sacred place in electoral planning is the same reason that Trump's "flood the zone" strategy was effective (again--not good, but undeniably effective): capitalizing on/marketing early unilateral wins of any size results in the public and Congress being more likely to support larger, more durable changes. This is complicated by many factors (media landscape, districting, money), but is broadly true.
"Stacking courts" would require a Senate that actually votes those judges in. "Brandishing Executive orders" requires a congress that won't be able to countermand you and a Supreme Court that won't "nuh uh" you.
You are yet another person upset that Democrats cannot overcome the purposeful design of our government that you need a lot of power to build, and little power to destroy.
People who want to fix things need dramatically more power than people who want to stymie and break things. Democrats only rarely get that power, and usually only by one or two votes from people who strictly do not care about fixing things. You want this country to fix things? You need to vote significantly more for a party who will push to fix things.
The minority party in congress has no power by design.
Im an independent who would prefer a version of republicanism that died and the closest thing I have is a deeply ineffective party. While the right is currently building things I don't like to put in place institutions and laws I dont like.
All of you talking about the right destroying things are wrong they’re just building things and enshrining things you wont like.
You've witnessed a dismantle and replace effort by the right wing that wishes to squeeze everything to make rich people more money. An effort by the left would destroy the private insurance scheme and build up medicare. Completely different and you'd get something functional.
When the wrong targets get destroyed, everyone suffers. When parasitic forces are destroyed, the system functions better. It's the difference between defense and friendly fire.
We already had an effort by the left. You can “no true scotsmen” if you want, but it represents the reality of what will happen when ideals clash a sector that makes up 18% of the GDP.
Are you referring to the ACA here? That was a compromise bill that props up the current system in the US, primarily created by right leaning centrists.
Why is this downvoted? What the (far) right wing has done to american institutions is incredibly destructive and it will take decades to fix it, if that's even possible. People are hurting because of it.
The whole system has basic flaws in how's financing set up.
There is an intermediary between customers and seller and it's allowed to take percentage of the sale. No such entity will ever work in the interest of the consumer. It has every incentive to inflate prices. Intermediary is needed but it should be financed by buyers with flat fee (possibly for additional incentives that reinforce the desired behavior). The tragedy here is that initially it was. But it was deemed too expensive for the buyers and got privatized which made it vastly more expensive in the long run.
Insurance is also wrong. Insurance is gambling and gambling needs restrictions. You are allowed to take people's money without providing any service most of the time, so you shouldn't be allowed to refuse legal service for that privilege.
Hate to break it to you. It’s the same outside of America. Yes, your insurance system is broken. But no just because you live in say Sweden you won’t get all the treatment you want directly. It is a pain to get it and if you get it you will often have to wait a long time (unless it’s a heart attack in progress then they are fast)
Perhaps, but I don't have much optimism for what this ends up looking like if it's an AI you have to convince to listen to you. In the spaces where this is already happening (rescruitment comes to mind), things are not looking good..
Agreed. Last time I was sick I said my fevers were pushing up to 100 and they said it's not a concern until 100.4. felt like an odd number. It's 38 C. Because my dramatic undersampling of my temperature was 0.4 degrees lower than their rounded threshold through some unit conversions, I clearly didn't have a fever. That's not a very human touch
Maybe you had trouble re-reading your own comment but I can tell by how you responded here (a cascade of links/references) and a snarky comment ("I can keep going if you'd like") that I'm sure the doctor was glad to be rid of you.
You didn't say the doctor disputed you had a fever. You said the doctor told you the fever wasn't concern until 100.4. Which I'm guessing is your fault for misinterpreting. If you google around, it's very easy to see the fever thresholds.
Here, I'll even paste a summary for you, and I can keep going if you like:
Key Temperature Thresholds
- 100.4°F : The standard definition of a fever.
- 103°F : Contact a healthcare provider
- 104°F : Seek medical attention, particularly if it does not come down with - treatment.
- 105°F : Emergency; seek immediate care.
In one of your own links (clevelandclinic.org), here's an excerpt for you:
When should a fever be treated by a healthcare provider?
In adults, fevers less than 103 degrees F (39.4 degrees C) typically aren’t dangerous and aren’t a cause for concern. If your fever rises above that level, make a call to your healthcare provider for treatment.
A fever is 38c, great. What the parents said was that you may have misheard because a fever isn't serious until 104. Which is line's up with the language you used.
> and they said it's not a concern until...
Parent is not suggesting that a fever isn't at 100F, they're suggesting that it's not "a concern" until 104F, a number strangely similar to 100.4 that you claim you heard, presumably, while you had a fever.
They aren’t objectively incorrect. You are conflating two things:
- You aren’t considered to have a fever until you get to 100.4. Anything less than that isn’t considered a fever, let alone a concerning one
- A fever isn’t considered concerning (ie dangerous) until it reaches about 104. Anything between 100.4 and 104 is just a regular fever and isn’t considered concerning.
At which point I'd ask: how much of that is baked into the AI now?
It doesn't have opinions, research, direction of its own. Is this a path of codifying the worst elements of human society as we've known it, permanently?
This even translates to the pediatric space. I took all of my kids to the pediatrician because either they don't make comments to me like they do to my wife, or I don't take shit from them. I'm not sure which. Here's an example:
My wife and daughter were there and the doctor asked what kind of milk my daughter was drinking. She said "whole milk" and the doctor made a comment along the lines of "Wow, mom, you really need to switch to 2%". To understand this, though, you need to understand that my daughter was _small_. Like they had to staple a 2nd sheet of paper to the weight chart because she was below the available graph space. It wasn't from lack of food or anything like that, she's just small and didn't have much of an appetite.
So I became the one to take the kids there. Instead of chastising me, they literally prescribed cheeseburgers and fettuccine alfredo.
My daughter is in her 20s now and is still small -- it's just the way she is. When she goes to see her primary, do you know what their first question is? "When was your last period."
> My daughter is in her 20s now and is still small -- it's just the way she is. When she goes to see her primary, do you know what their first question is? "When was your last period."
Is that supposed to be a problem? How does it connect to the story in your comment?
The question seems to be warranted to me, since being underweight can stop you from menstruating. So if you find someone thin and her last period was off in the distant past, you can conclude that there's a problem and something should be done about it; if it was a couple of weeks ago, you can conclude that she's fine.
(It could also just be something that is automatically assessed as a potential indicator of all kinds of different things. Notably pregnancy. For me, it bothered me that whenever you have an appointment at Kaiser for any reason, part of their checkin procedure is asking you how tall you are. I'd answer, but eventually I started pointing out to them that I wasn't ever measuring my height and they were just getting the same answer from my memory over and over again. [By contrast, they also take your weight every time, but they do that by putting you on a scale and reading it off.] The fact that my height wasn't being remeasured didn't bother them; I'm not sure what that question is for.)
I’m a normal weight, and get asked the same question. More importantly, I can tell them, “I have a regular cycle” and they WILL NOT take that as an answer. I HAVE to give them a date, and they will ask me to make one up if I can’t remember or want to decline giving them that information.
Particularly given the alarming stories of people being prosecuted for having miscarriages, it feels ridiculous.
If anything I hope more automated diagnostics and triage could help women and POC get better care, but only if there’s safeguards against prejudice. There’s studies showing different rates of pain management across races and sexes, for example. A broken bone is a broken bone, regardless of sex or race.
The system doesn't know that you're a smart person who will only say "I have a regular cycle" when you've had something that could reasonably be called a regular cycle. A lot of patients are stupid, and requiring a quantitative answer eliminates one source of stupidity. Yeah, this particular doctor knows you're smart, but I hope you can see what disasters might result if the procedure said "the doctor may skip this step if the patient is smart".
It's the same reason why the doctor will take your temperature, instead of accepting your word that you took your own temperature and it is normal.
> Particularly given the alarming stories of people being prosecuted for having miscarriages
You need to delete your social media accounts and change where you're getting your news from. Nobody is "being prosecuted for having miscarriages". A few people have been investigated for drug abuse during pregnancy which led to the baby's death, which sensationalist news stories twisted into attention-grabbing headlines.
A doctor asking about cycle is just a core piece of diagnostic data like taking blood pressure and temperature, not some conspiracy to harm you.
Perhaps I wasn't as clear as I could have been. My point was that doctors treat women differently than men, even when they're the parents. I don't think that it's inherently malicious, but there is absolutely a bias.
You are asking how it connects, and it absolutely doesn't. But they keep asking and won't accept "it's regular" as an answer.
She's in her 20s and is seeing her primary for routine things, not because of her weight -- that part of the story was about how they chastised my wife for giving her whole milk but said absolutely nothing to me about it later on.
However, your argument focuses on the routine intake instead of any listening part. The fact that the doctor measures height, weight, temperature, and blood pressure on intake and then asks about LMP doesn’t surprise me… that’s the part of the script where you just provide the data before you bring up concerns.
Not to say the doctor was not a jerk, just that your argument doesn’t do much for me.
Yes? That's a very important piece of information, and I hope would be a thing a doctor asks, especially if there are concerns about weight or nutrition.
She's not there about her weight, though. I highly encourage you to talk to women about their experiences here.
The weight thing was not the key aspect of my original comment. They chastised my wife for continuing to give my daughter whole milk while being underweight, but did not make similar comments to me. That was the point.
For women, their pains and problems are far too often whisked away by hand waving and "it's hormones and periods" and serious issues are often overlooked. Very little has changed in that area over the last twenty years.
Yes, let's pretend that the bias does not exist, that is helpful. It certainly doesn't have to do with the fact that it's currently a 60/40 split in active male vs female physicians. Or that women are more likely to be taken seriously by doctors:
I have it so strong, that after I was preparing myself, my work desc, my books everything, i was starring into the books i wanted to learn for 15-30 minutes unable to just start or do anything.
With ritalin, i might have this mental block to, but its overcome in a few seconds.
I went from a 'nearly/borderline failing grade' to the nearly the best grade in just one year.
Because people believe that they know everything about humans and how they work (or they hedge it). This is the exact same reason I don't trust supposed "experts" claiming AI will replace all these jobs: those same experts have no idea what these jobs actually entail and just look at the job title (and maybe the description) but have not once actually worked those jobs. And there is a huge chasm between "You read the job description" and "you actually know what it is like to be in this position and you fully understand everything that goes into it".
Ok fellas put your money where your mouth is. It’s easy to talk until you put your money behind it (or lack of by getting rid of spending on it) if you are so confident in doctor as a service by llm.
I would use one for sure. Much of medicine is getting tests / labs booked fighting to get certain medicines. Doctors will barely give you 5 minutes only deal with one issue per visit, rarely are available and going into an office can make you sicker. An llm with Doctor powers could offer more. I don't think we are at the surgery point but we are past getting notes and medicine's refilled.
So why not order your own labs? I'm sure you can think of ways to get your own medications if you are sufficiently convinced that this is the best course of action for your health.
Arterial blood gas. Calcium score (may not count as a lab). Skin biopsy for cancer (does it count as a lab?). I'm unaware how to order my own troponin if I think I've had a heart attack (not that that's one I should DIY diagnosis). Prostate specific antigen.
It seems likely to me that doctors whose job is almost or entirely about making diagnoses and prescribing treatments won't be able to keep up in the long run, where those who are more patient facing will still be around even after AI is better than us at just about everything.
If I were picking a specialty now, I'd go with pediatrics or psychiatry over something like oncology.
You are confusing the job with a subset of tasks. Some tasks can be automated, some won't. That doesn't mean LLMs, which cannot tell how many r's are in strawberry, will replace anyone.
I'm not. I understand the difference and also that through improvements to the core models as well as harnesses, LLMs are able to handle an increasing share of tasks. I also understand that these things will continue to improve until AI can automate entire jobs.
You, on the other hand, are confusing LLMs from the past with current SOTA LLMs, which can tell how many rs are in strawberry.
It's tech bros like you that are to blame for the shortage of radiologists supporting statements since 2016 which state that radiologists will disappear. Your great SOTA LLMs will tell you to walk 5 mins to the car wash instead of taking the car.
> That doesn't mean LLMs, which cannot tell how many r's are in strawberry, will replace anyone.
But most of us live in America in 2026. There are a lot of interests that don't give a shit about you who would love if you to got your medical care from a machine that "cannot tell how many r's are in strawberry". And there a lot of useful idiots with no real medical issues who will loudly claim the machine is great.
How are you defining technology? How are you defining human problems? Inventions are created to solve human problems, not theoretical problems of fictional universe. Do X-rays, refrigerators, phones and even looms solve problems for nonhumans?
Claiming something that sounds deep doesn’t make it an axiom.
A) nice chatty friendly and cool doctor and can diagnose correctly 50% of the times.
B) robotic ai that diagnoses 60% correctly.
What you chose? If you have a disease than can kill your, the ai is 20% more likely to help you and probably prevent. I can’t see too many people choosing human doctor. Anyway I’m sure there will be people that will chose doctor with 10% correctness vs a 100% ai no matter what.
If you read the study, the whole conclusion is much less spectacular than the article. What the article really pushes happened:
patients -> AI -> diagnosis (you know, with a camera, or perhaps a telephone I guess)
What REALLY happened
patients -> nurse/MD -> text description of symptoms -> MD -> question (as in MD asked a relevant diagnostic question, such as "is this the result of a lung infection?", or "what lab test should I do to check if this is a heart condition or an infection?") -> AI -> answer -> 2 MDs (to verify/score)
vs
patients -> nurse/MD -> text description of symptoms -> MD -> question -> (same or other) MD -> answer -> 2 MDs verify/score the answer
Even with that enormous caveat, there's major issues:
1) The AI was NOT attempting to "diagnose" in the doctor House sense. The AI was attempting to follow published diagnostic guidelines as perfectly as possible. A right answer by the AI was the AI following MDs advice, a published process, NOT the AI reasoning it's way to what was wrong with the patient.
2) The MD with AI support was NOT more accurate (better score but NOT statistically significant, hence not) than just the MD by himself. However it was very much a nurse or MD taking the symptoms and an MD pre-digesting the data for to the AI.
3) Diagnoses were correct in the sense that it followed diagnostic standards, as judged afterwards by other MDs. NOT in the sense that it was tested on a patient and actually helped a live patient (in fact there were no patients directly involved in the study at all)
If you think about it in most patients even treating MDs don't know the correct conclusion. They saw the patient come in, they took a course of action (probably wrote at best half of it down), and the situation of the patient changed. And we repeat this cycle until patient goes back out, either vertically or horizontally. Hopefully vertically.
And before you say "let's solve that" keep in mind that a healthy human is only healthy in the sense that their body has the situation under control. Your immune system is fighting 1000 kinds of bacteria, and 10 or so viruses right now, when you're very healthy. There are also problems that developed during your life (scars, ripped and not-perfectly fixed blood vessels, muscle damage, bone cracks, parts of your circulatory system having way too much pressure, wounds, things that you managed to insert through your skin leaking stuff into your body (splinters, insects, parasites, ...), 20 cancers attempting to spread (depends on age, but even a 5 year old will have some of that), food that you really shouldn't have eaten, etc, etc, etc). If you go to the emergency room, the point is not to fix all problems. The point is to get your body out of the worsening cycle.
This immediately calls up the concern that this is from doctor reports. In practice, of course, maybe the AI only performs "better" because a real doctor walked up to the patient and checked something for himself, then didn't write it down.
What you can perhaps claim this study says is that in the right circumstances AIs can perform better at following a MD's instructions under time and other pressure than an actual MD can.
100% of the cases where some headline makes big claims about "AI" based on some study, you take a good hard look at the study and none of the big claims stand on their own.
It's all heavily spinned, taken out of context, editorialized... It's become almost a hobby of mine lately. And I am glad for have read so many papers and reasoned critically about methods and statistics. But it is also scary to realize just how much people take at face value of bombastic interpretations of datasets that support no such claim or much weaker versions only.
Chasing down sources is something that I often do and I've learned that people take a lot of liberty when divulging opinions about sources they don't think will be checked. Even in high trust environments. I have first hand received work by post-doctoral fellows where some articles in the bibliography didn't even exist.
"Human problems can't be solved with technology" is just wrong, unless you have narrower definitions of a "human problem" or "technology".
For instance, transportation is a "human problem". It's being successfully solved with such technologies as cars, trains, planes, etc. Growing food at scale is a "human problem" that's being successfully solved by automation. Computing... stuff could be a "human problem" too. It's being successfully solved by computers. If "human problems" are more psychological, then again, you can use the Internet to keep in touch with people, so again technology trying to solve a human problem.
I think you may be misunderstanding the concept of 'human problem'. A human problem is caused by humans, it isn't something like transportation. That is a physics problem. An example of a human problem is cheating; you can't solve cheating with technology. Just add [incentive] after human and it should make more sense.
IMO "human problem" isn't a well-defined concept, so it's not really possible to misunderstand it. I think a "human problem" is a problem that _humans have_: how to move around? (transportation) what to eat? (agriculture, etc) how to prevent cheating? (some kind of surveillance) how to communicate over long distances? (radio, the internet, etc)
Sure, some kinds of such "human problems" can be reduced to physics and technology, that's the point. This also doesn't necessarily mean that solutions produced by such reductions are effective: is surveillance good at preventing cheating during exams? Kind of. Does it often fail to catch cheating students? Absolutely.
However, indeed, there can be many different (perhaps equally correct) definitions of what a "human problem" is.
> In psychotherapy patients tend to prefer talking to AI than a human therapist and rank the interaction higher.
Even if your statement is true, it's questionable. People also tend to prefer hearing what they want to hear to hearing what they need to hear, and rank the former interaction higher.
Basically, tech's favorite feedback mechanism, customer reviews, cannot actually be relied upon to tell you how good something is.
I know. I know. Part of it is that talking to patients on average is useless but still this can’t be really used for an argument against AI.
Still doctors can have a more broad picture of the situation since they can look at the patient as a whole; something the LLM can’t really synthesize in its context.
I think you'd be incredibly surprised how often charts are super, super incomplete or wrong. Like "pt has no pancreas and presented with pain and weeping from a 6yo pancreatectomy scar" but the chart doesn't mention the surgery or the entire missing organ wrong. Like "pt is a twin whose sibling died traumatically of cancer in front of them a year ago and presents with probable hypochondria about cancer" but the chart doesn't mention any family history wrong. Like "lifelong history of severe cognitive impairment substantiated by a psych eval; attended annual physical before being sent to imaging for head trauma because of observed impairment" but the chart doesn't mention cognition (someone was too polite to note it) nor the psych eval (records sharing wasn't allowed) wrong.
Those are a very few examples off the top of my head. I worked in EMR. I don't know shit about medicine, but man, do I know a lot about the complaints physicians and their staff send when they think it's the records system's fault that the chart was wrong or missing info.
In a big chunk of cases, the MD/NP/whatever's in-person role is determining what's not on the chart so that they can then ask appropriate follow-up questions. Given the massive range of possible dx for a given issue, and how much of getting the right dx doesn't have to do with probabilities/numbers of similar patients with the same symptom:dx data that'd be in the training set, I have major doubt that an LLM can appropriately intuit or appropriately question in order to diagnose.
Yeah... No. I can't possibly disagree with this view more.
I don't need to "talk to a human", I need a problem with my meatbag resolved.
> humans need other humans and human problems can't be solved with technology
WTF are you talking about? Is this bait? You can't possibly mean this. Yes humans are social creatures, but what does that have to do with medicine? Are you talking about a priest, a witch doctor, a therapist? Because if you're not, that sentence is utter BS.
Yes talking to a human is good and necessary. But for diagnostics humans are not good at it. I'm happy for to human to use a tricorder and then tell me the answer.
>Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to.
Humans (doctors/nurses) can still be there to make you feel the warmth of humanity in your darkest times, but if a machine is going to perform better at diagnosing (or perhaps someday performing surgery), then I want the machine.
Even now, I'll take a surgeon that's a complete jerk over a nice surgeon any day, because if they've got that job even as a jerk they've got to be good at their jobs. I want results. I'll handle hurt feelings some other time.
I'd be a little bit careful here - being a jerk is quite different to non-conformity / red sneaker effect in surgery and it is not a quality you should look for.
The truly compassionate surgeons will want to improve their skills because they care about their patients. They care if they develop complications and may feel terrible if they do, the jerk may not. Being a jerk may mean that the surgeon can rise to the top, but it may not be due to surgical skill at all, they may be better at navigating politics etc.
> Even now, I'll take a surgeon that's a complete jerk over a nice surgeon any day, because if they've got that job even as a jerk they've got to be good at their jobs.
This seems like an incredibly poor line of reasoning.
Hospitals are often desperate for surgeons. The poorly mannered ones are often deeply unsatisfied, angry at the grueling lives they've opted into, and the hospitals can't replace them. The market is not exactly at work here.
I haven't known doctors or nurses to be very warm and fuzzy. I have known them to have real world experience in seeing the outcomes of their actions instead of...
Dude you removed my right thumb I was in for an appendectomy!?
You are so right! I ignored everything you asked for. I am so sorry. I am administering general anesthesia now, then I will prepare you for your next surgery.
I think there's a real space there, and a lot of what e.g. nurses and doctors do is talking to humans, and that won't go away.
But two facts are also true: a) diagnosis itself can be automated. A lot of what goes on between you having an achy belly and you getting diagnosed with x y or z is happening outside of a direct interaction with you - all of that can be augmented with AI. And b), the human interaction part is lacking a great deal in most societies. Homeopathy and a lot of alternative medicine from what I can see has its footing in society simply because they're better at talking to people. AI could also help with that, both in direct communication with humans, but also in simply making a lot of processes a lot cheaper, and maybe e.g. making the required education to become a human facing medicinal professional less of a hurdle. Diagnosis becomes cheaper & easier -> more time to actually talk to patients, and more diagnosises made with higher accuracy.
I would personally vastly, vastly prefer to go to a robot doctor, who diagnoses, treats and nurses me. What exactly do I need from a human here? Except of course being the one making the system.
Emotional support. Some human doctors absolutely radiate confidence and a kind of "you're gonna be okay" attitude. For me, this helps a lot. I'm not sure a machine can do this.
But I hate if the human doctor "radiates confidence" when I know he is not doing the proper scan, because I have to get back with worse symptoms first for him to take it serious. I don't need emotional support from a human doctor. I need the adequate scans and a proper analysis. I am pretty sure that a competent human will be still way better than AI, but AI even now will likely be better than a doctor not really paying attention.
> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well,
This is a pretty wild leap. Code has a lot of hooks for training via hill-climbing during post-training. During post-training, you can literally set up arbitrary scenarios and give the bot more or less real feedback (actual programs, actual tests, actual compiler errors).
It's not impossible we'll get a training regime that does the "same thing" for medicine that we're doing for code, but I don't know that we've envisioned what it looks like.
Code is pretty much the perfect use case for LLMs… text-based, very pattern-oriented, extremely limited complexity compared to biological systems, etc.
I suspect even prose is largely considered acceptable in professional uses because we haven’t developed a sensitivity to the artifice, and we probably won’t catch up to the LLMs in that arms race for a bit. However, we always manage to develop a distaste for cheap imitations and relegate them to somewhere between the ‘utilitarian ick’ and ‘trashy guilty pleasure’ bins of our cultures, and I predict this will be the same. The cultural response is already bending in that direction, and AI writing in the wild— the only part that culturally matters— sounds the same to me as it did a year and a half ago. I think they’re prairie dogging, but when(/if) they drop that bomb is entirely a matter of product development. You can’t un-drop a bomb and it will take a long time to regain status as a serious tool once society deems it gauche.
The assumption that LLMs figuring out coding means they can figure out anything is a classic case of Engineer’s Disease. Unfortunately, this hubris seems damn near invisible to folks in the tech industry, these days.
I think that might help a little, but is not a solution. When you’re figuring out some new way to combine code instructions to perform novel coding tasks, you’re just finding new configurations for existing patterns to get results you can easily test. The world outside of computers is infinitely more complex, random, and novel.
Emergency medicine is the coding of medicine. Fast feedback loop, requires broad rather than deep judgement, concrete next steps.
The AI coding improvement should be partially transferrable to other disciplines without recreating the training environment that made it possible in the first place. The model itself has learned what correct solutions "feel like", and the training process and meta-knowledge must have improved a huge amount.
I would argue that the ED is the least similar to code. You have the most unknowns, unreliable data and history, non deterministic options and time constraints.
An ER staff is frequently making inferences based on a variety of things like weather, what the pt is wearing, what smells are present, and a whole lot of other intangibles. Frequently the patients are just outright lying to the doctor. An AI will not pick up on any of that.
Unfortunately the training data is absolute garbage.
Diagnostic standards in (at least emergency, but I think other specialties) medicine are largely a joke -- ultimately it's often either autopsy or "expert consensus."
We get to bill more for more serious diagnoses. The amount of patients I see with a "stroke" or "heart attack" diagnosis that clearly had no such thing is truly wild.
We can be sued for tens of millions of dollars for missing a serious diagnosis, even if we know an alternative explanation is more likely.
If AI is able to beat an average doctor, it will be due to alleviating perverse incentives. But I can't imagine where we could get training data that would let it be any less of a fountain of garbage than many doctors.
Without a large amount of good training data, how could AI possibly be good at doctoring IRL?
You just get 1M doctors to wear body cams for a year. Now you have a model that has thousands of times your experience with patients, encyclopedic knowledge of every ailment including ones that never present in your geography, read all the latest papers, etc..
I don't understand how you think this doesn't win vs a human doctor.
This wouldn't solve the problem of diagnostic standards. Let's say you are a pediatrician and want to predict which kids with bronchiolitis will develop respiratory failure and need the ICU versus the ones who can go home. How do you determine from the body cams which kids had bronchiolitis in the first place? Bronchiolitis is a clinical diagnosis with symptoms that overlap with other respiratory illnesses such as asthma, bacterial pneumonia, croup, foreign body ingestion, etc.
you would have footage of the doctors diagnosing them. I don't understand what you're asking. The body cams have microphones too in case that wasn't clear.
In healthcare, HIPAA/GDPR equivalent would block this. Let's be realistic in our discussion; this is not the same as google buying up a library worth of books, scanning and destroying them
Other countries actually don't necessarily have a similar mix of ailments, median patient appearance and style of communication or even recommended course of action and most of the ones with more sophisticated medical care also have strict medical privacy laws. If you're genuinely unaware of this, I'm not sure you're in a position to be making "one year with a camera, how hard can it be" arguments...
(Where AI is likely to actually excel in medicine is parsing datasets that are much easier to do context free number crunching on than ER rooms, some of which physicians don't even have access to ...)
I think you're being silly if you think the amount of money at stake here, not the mention the health of billions of people is going to be stymied by privacy laws.
My sense is that doctors and AI would be doing a lot better if they were just doing medicine, not being a contact surface for failures of housing, mental health and addiction services, and social systems. Drug seeking and the rest should be non-issues, but drug seekers are informed and adaptive adversariesz
To give this more credit than it perhaps deserves: training aside, getting the situational data into the context is a more significant problem here.
Pt's chart is complex/wrong? Gotta ingest that into context.
Chart contains images/scanned and not OCR'd text? Gotta do an image recognition pass.
Diagnosis needs to know what the pt's wearing (i.e. radiation badge)? Gotta do an image recognition pass.
Diagnosis needs to know what the weather's like? Internet API access of some kind. Hope the WAN/API are all working! If they're not, do you fail open or closed?
Patient might be lying? Gotta do video/audio analysis to assess that likelihood--oh, and train a model that fully solves one of the holy grails of computer vision/audio analysis reliably and with a super low false-positive rate before you do. And if it guesses wrong, enjoy the incredibly easy-to-prosecute lawsuit.
Patient might be lying, but the biggest clue is e.g. smell of alcohol on their breath? Now you need some sort of olfactory sensor kit and training for it--a lot more than just "low quality body cam and a mic".
Patient's ODing on a street drug that became abundant in the last few months? Gotta somehow learn about recent local medical/police history that post-dates the training set, or else you might be pouring gas on a fire if you give them Narcan. And that's assuming you know enough to search for information about that drug, and that they didn't lie to you about what they took. Addicts never do that.
Failures in each of those systems bring down the chance of an effective diagnosis, so they need a fairly obsessive amount of model introspection/thinking/double-checking, and humans on standby as a fallback if the AI's less than confident (assuming that LLMs can be given a sense of a confidence level in the future, versus the current state of the art of "text-predict a guess about what your confidence level might be").
Put that all together, and even with the AI compute speed available years from now and a perfectly trained futuristic model that's preternaturally good at this stuff, I'm not sure that that the reliability and, more importantly, the turnaround time of that diagnostic pass is going to be any good compared to a human ER doc.
>What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it.
You cannot simply put liability and ethics aside, after all there's Hippocatic oath that's fundamental to the practice physicians.
Having said that there's always two extreme of this camp, those who hate AI and another kind of obsess with AI in medicine, we will be much better if we are in the middle aka moderate on this issue.
IMHO, the AI should be used as screening and triage tool with very high sensitivity preferably 100%, otherwise it will create "the boy who cried wolf" scenario.
For 100% sensitivity essentially we have zero false negative, but potential false positive.
The false positive however can be further checked by physician-in-a-loop for example they can look into case of CVD with potential input from the specialist for example cardiologist (or more specific cardiac electrophysiology). This can help with the very limited cardiologists available globally, compared to general population with potential heart disease or CVDs, and alarmingly low accuracy (sensitivity, specificity) of the CVD conventional screening and triage.
The current risk based like SCORE-2 screening triage for CVD with sensitivity around is only around 50% (2025 study) [3].
"The boy who cried wolf" is a story about false positives, so if that's what you want to avoid then you want to get close to 100% specificity, and accept that there are many things that the tool will not catch. If, as you propose, the tool would mainly be used to create a low confidence list of potential problems that will be further reviewed by a human, then casting a wide net and calibrating for high sensitivity instead does make sense.
The idea is to minimize the false positives "the boy who cried wolf" at the same time mitigate, or better eliminate false negatives. The main reason is that based on the physician in-the-loop, the system can be optimized for sensitivity but can be relaxed for specificity. Of course if can get both 100% sensitivity and specificity it will be great, but in life there's always a trade-off, c'est-la-vie.
In our novel ECG based CVD detection system we can get 100% sensitivity for both arrhythmia and ischemia, with inter-patient validation, not the biased intra-patient as commonly reported in literature even in some reputable conferences/journals. Specificity is still high around 90% but not yet 100% as in sensitivity but due to the physician-in-the-loop approach, which is a diagnostic requirement in the current practice of medicine, this should not be an issue.
Assume if you know for certain that AI has better senstivity and specificity than your local physician for the particular diagnosis, which likely would be the case now or in few years. Would you purposefully get inferior consultation just because of Hippocatic oath?
> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers
You first have to assume this for software engineers. Not everyone agree with that (note: that doesn't mean the same people don't agree that AI is not _useful_).
AIs still have a ton of issues that would be devastating in a doctor. Remember all the AIs mistakingly deleting production DBs? Now imagine they prescribed a medicine cocktail that killed the patient instead. No thanks. There's a totally different bar to the consequences of mistakes.
In some subfields, like detection of security weaknesses in obscure C code, AI is already better than software engineers.
It is capable of sifting through enormous reams of data without ever zoning out etc. Once patients routinely use various wearables etc., they, too, will produce heaps of data to be analyzed, and AI will be the thing to go to when it comes to anomaly detection.
I don't think this is a fight doctors can win. We programmers make mistakes all the time.
At one place, we had a QA lead who was burned so many times she would insist that she will find the time to do at least a full smoke test even if we promised it was a small contained change in the frontend. I have no idea how she found the time because she wore multiple hats.
Doctors make errors all the time though, so the real argument is about the error percentage. If AIs is lower then it's safer (but it's hard to have that convo, I recognise).
Besides; this article was about diagnosis not prescribing. It's pretty obvious, I think, that diagnosis is one area where AI will perform extremely well in the long run.
I think there are two metrics; the first is outright misdiagnosis, which studies put between 5 and 8% in US/Europe. That's a meaningful number to tackle.
Secondly; overdiagnosis. Where a Dr says on balance it could be X on a difficult to diagnose but dangerous problem (usually cancer). The impact of overdiagnosis is significant in terms of resources, mental health, cost etc.
Do you believe the issue is because they don't have enough technicians to diagnose or because they don't have enough x-ray machines?
Or in a ER environment, how an AI would speed up things in a real way that improves patients' lives?
We just minted the term "cognitive debt" for software engineers that cannot keep up with what the AI spits out. How would that apply to ER doctors, or any other kind of doctor?
I'm not talking in particular about the X rays. It's about general lack of hospitals, equipment and doctors.
In Europe, there are some rich cities which have on average one doctor per hundred people. And there are large areas in Eastern Europe that have ten times less than that.
If you have some unusual symptoms or a little pain somewhere and no access to doctors you will most likely ignore it.
If you can get any diagnosis it can help you e.g. decide to travel to get treatment.
And the locally available alternative for ai diagnosis is a doctor you can get to in few months, who works 80 hours a week and has 10 minutes per patient.
For ai to be valuable you really don't need to be better than average physician in top American clinic.
Diagnosis is just a small part of a doctor's job. In this case, we're also talking about an ER, it's a very physical environment. Beyond that, a doctor is able to examine a patient in a manner that isn't feasible for machines any time in the foreseeable future.
More importantly, LLMs regularly hallucinate, so they cannot be relied upon without an expert to check for mistakes - it will be a regular occurrence that the LLM just states something that is obviously wrong, and society will not find it acceptable that their loved ones can die because of vibe medicine.
Like with software though, they are obviously a beneficial tool if used responsibly.
> After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans
No, I don’t see that we must.
> if we already have this assumption for software engineers
No, this doesn’t follow, and even if it did, while I am aware that the CEOs of firms who have an extraordinarily large vested personal and corporate financial interest in this being perceived to be the case have expressed this re: software engineers, I don’t think it is warranted there, either.
You’re holding on to the intuition (hope) that we are smarter than the LLMs in some hard to define way. Maybe. But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win. I agree experienced humans are still better on “judgement” tasks in their field. But the judgement tasks are kinda necessarily ones where there isn’t a correct answer. And even then, I think the machines’ judgement is better than a lot of humans.
Is medical diagnosis one of these high judgement tasks? Personally I don’t think so.
LLM’s operate on a mechanical form of intelligence one that at present is not adaptive to changes in the environment.
If the latter part of your post were true, how come the demand for radiologists has grown? The problem with this place is it’s full of people who don’t understand nuance. And your post demonstrates this emphatically.
For me there are a few main takeaways on how AI _could_ supersede the average ER doctor.
The first is that a technical solution can be trained on _ALL_ medical data and have access to it all in the moment. It is difficult to assume a doctor could also achieve this.
The second is that for medical cases understanding the sum of all symptoms and the patients vitals would lead to an accurate diagnosis a majority of the time. AI/ML is entirely about pattern recognition, when you combine this with point one, you end up with a system that can quickly diagnose a large portion of patients in extremely short timeframes.
On a different note, I think we can leave the ad-hominem attacks at home please.
>But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win.
I and likely the person who you replayed to don't find that existing studies actually hold this to be true.
> But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win.
Quite to the contrary, I think it's extremely trivial to find a task where humans beat LLMs.
For all the money that's been thrown at agentic coding, LLMs still produce substantially worse code than a senior dev. See my own prior comments on this for a concrete example [1].
These trivial failure cases show that there are dimensions to task proficiency - significant ones - that benchmarks fail to capture.
> Is medical diagnosis one of these high judgement tasks?
Situational. I would break diagnosis into three types:
1. The diagnosis comes from objective criteria - laboratory values, vital signs, visual findings, family history. I think LLMs are likely already superior to humans in this case.
2. The diagnosis comes from "chart lore" - reading notes from prior physicians and realizing that there is new context now points to a different diagnosis. (That new context can be the benefit of hindsight into what they already tried and failed and/or new objective data). LLMs do pretty good at this when you point them at datasets where all the prior notes were written by humans, which means that those humans did a nontrivial part of the diagnostic work. What if the prior notes were written by LLMs as well? Will they propagate their own mistakes forward? Yet to be studied in depth.
3. The diagnosis comes from human interaction - knowing the difference between a patient who's high as a bat on crack and one who's delirious from infection; noticing that a patient hesitates slightly before they assure you that they've been taking all their meds as prescribed; etc. I doubt that LLMs will ever beat humans at this, but if LLMs can be proven to be good at point 2, then point 3 alone will not save human physicians.
> I doubt that LLMs will ever beat humans at this, but if LLMs can be proven to be good at point 2, then point 3 alone will not save human physicians.
Agree with your division but I'm baffled by this argument. If humans are better than machines at point 3 and can also use a machine to do point 2, then unless they have particularly terrible biases against taking point 2 data into account they're going to be strictly better than machines alone. Doctors have costs, but they're costs people/society are generally willing to underwrite, and misdiagnosis also has costs...
There are almost no real world tasks that LLMs outperform humans on, operating by themselves. Pair them with a human for adaptability, judgement, and real world context and let the human drive, sure. Just let it loose on its own? You get an ocean of slop that doesn't do even close to what it's supposed to.
Currently that self-improving system isn’t so self-improving that it’s become better at any particular job than human beings, so I think the skepticism is warranted.
This seems to produce skill atrophy. "It's okay because the AI will pick up the slack" is kinda true but doesn't exactly strengthen the human position though.
> if we already have this assumption for software engineers
Do we have that assumption? I don't think there's a consensus on it yet, just various camps of people proselytizing the other camps based on how much or little they use AI.
> What is the specific capability (or combination of capabilities)
The ability to go to prison / be stripped of a license when something goes wrong.
A single doctor will care for far fewer patients in their career than an AI system will. Even if the AI system is 10x less likely to make mistakes, the sheer number of patients will make it much more likely to make a mistake somewhere.
With a single doctor, the PR and legal fallout of a medical error is limited to that doctor. This preserves trust in the medical system. The doctor made a mistake, they were punished, they're not your doctor, so you're not affected and can still feel safe seeing whoever you're seeing. AI won't have that luxury.
No, but someone needs to bear responsibility. Whether that's a doctor, or a CEO directly, ordering the replacement of a radiologist by AI. If things go sideways, there needs to be a chain or responsibility.
This study is based almost entirely on pre-existing "vignettes." In other words, on tests that are already known and have existed for years, the model did well, which is precisely what you should expect.
It provides no information on real world outcomes or expectations of performance in such a setting. A simple question might be "how accurate are patient electronic health records typically?"
Finally, if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot. If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot.
You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything. As it is now they're a very expensive, inefficient and fragile toy.
> This study is based almost entirely on pre-existing "vignettes."
This is basically the only way how to ethically approach the topic. First you verify performance on “vignettes” as you say. Then if the performance appears satisfying you can continue towards larger tests and more raw sensor modalities. If the results are still promising (both that they statistically agree with the doctors, but also that when they disagree we find the AIs actions to fall benignly). These phases take a lot of time and carefull analysises. And only after that can we carefully design experiments where the AI works together with doctors. For example an experiment where the AI would offer suggestion for next steps to a doctor. These test need to be constructed with great care by teams who are very familiar with medical ethics, statistics and the problems of human decision making. And if the results are still positive just then can we move towards experiments where the humans are supervising the AI less and the AI is more in the driving seat.
Basically to validate this ethically will take decades. So we can’t really fault the researchers that they have only done the first tentative step along this long journey.
> if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot
Privacy, resiliency and scalability are all best served with local LLMs here.
> If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot.
Generators would be the obvious answer there. If we can make machines which outperform human doctors in realworld conditions providing generator backed UPS power for said machines will be a no brainer.
> You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything.
I think it comes down to how much data we're comfortable feeding an AI. If the AI has cameras and/or microphones in the room and the patient is directly talking to the AI: I strongly suspect AIs will always achieve better outcomes than humans. However, this kind of configuration will be viewed very negatively in a medical context for the foreseeable future; outside of limited contexts like "let me take a picture of that mole"; and hobbling the AI to only a text input (or dictated text by the doctor) muddies the waters on who is performing better. There's a lot of intuition in the diagnosis of something like "the location of the pain aligns with appendicitis, but they just aren't in enough pain" that cannot come through in just the textual representation of what is happening; you need to hear the person's voice and see how they're holding their body. AI can do that, but will we let it do that?
Humans tend to be very bad at connecting dots, which is why when we imagine someone who does, we make the show "House" about it.
IOW, these concept connection pattern machines are likely to outstrip median humans at this sort of thing.
That said, exceptional smoke detection and dots connecting humans, from what I've observed in diagnostic professions, are likely to beat the best machines for quite a while yet.
My personal anecdote when I talk to people - everyone when talking about their job w.r.t AI is like "at least I'm not a software engineer!". To give a hint this isn't just a US phenomenon - seen this in other countries too where due to AI SWE and/or tech as a career with status has gone down the drain. Then they always go on trying to defend why their job is different. For example "human touch", "asking the right questions" etc not knowing that good engineers also need to do this.
The truth is we just don't know how things will play out right now IMV. I expect some job destruction, some jobs to remain in all fields, some jobs to change, etc. We assume it will totally destroy a job or not when in reality most fields will be somewhere in between. The mix/coefficient of these outcomes is yet to be determined and I suspect most fields will augment both AI and human in different ratios. Certain fields also have a lot of demand that can absorb this efficiency increase (e.g. I think health has a lot of unmet demand for example).
It's having a general understanding/view of the "baseline", aka healthy anatomy. This is something LLMs will never have, that's why never have true reasoning, for the lack of "worldview" and they never know if they are hallucinating.
To aid doctors, we don't need LLMs but rather, computer vision, pattern recognition as you correctly point out.
But it's important not to rely on it. Doctors can easily recognize and correct measurements with incorrect input, e.g. ECG electrodes being used in reverse order.
>It's having a general understanding/view of the "baseline", aka healthy anatomy. This is something LLMs will never have
You're making the mistake of conflating AI with LLMs.
I don't think LLMs will reliably be better than a board of doctors. But an Expert System probably will (if it isn't already). That's literally what they were created for.
The biggest downside of LLMs IMO isn't the millions of Jules wasted on training models that are ultimately used to create funny images of cats with lasers. It's that all that money isn't being invested into truly helpful AI systems that will actually improve and save our lives, such as medical expert systems.
The nature of expert systems is to become experts on a system.
The reason you need a doctor, or more often, let's be honest, a good nurse, is because systems can fail in any one of 10000 as yet undiscovered ways. New nurses. New residents. New techs. And on and on and on. All the measurements you're feeding to the system are an amalgamation of the potential errors of a potentially different set of professionals each time you move a patient through the enterprise.
Full disclosure, my first startup was building PACS and RTP software back before AI reading was a thing. Current startup working across dental and medical. Rethinking the link between oral and systemic health. Partner has been in the C-suite of several hospitals over the past few decades and now runs large healthcare delivery networks.
The reason you can't hand things over to AI, is precisely because there are so many humans in the system. Each of whom are fallible. Human experts are quicker to catch it. Expert systems are not. At least not any ES or AI I've seen. And I've been going to, for instance, RSNA, for well over 25 years.
If you have an ES or AI in the system, you would naturally put the same professionals responsible for catching human screwups, in charge of catching AI and ES screw ups. Even if these AI's turn 100% accurate based on the inputs they are given, that professional would still be responsible for catching those bad inputs.
Example, it's never happened to one of my companies knock on wood, but I have seen cases of radiation therapy patients being incorrectly dosed. The doctor almost never was the one who miffed in the situation, but ultimately, s/he's responsible.
Why? Bad input should have been caught.
Another example, situations where you operate on the wrong side of the body because someone prepped the wrong leg. Surgeon didn't do the prep. Whoever did do the prep may have simply relied on the software. But the software was wrong. May have been anything. Point is, the team is good, but everyone just fell into too complacent of a pattern with each other and their tools.
Trust is good. Complacency is not.
The same will hold true for AI team members that integrate into these environments. It's just another "team member", and it better have a "monitor". If not, you're asking for trouble.
The "monitor" ultimately responsible for everything will continue to be the provider. Any change in that reality will take decades. (And in the end, they probably will not change the current system in that regard.)
But liability and ethics cannot be put aside. If treatments were free of cost and perfectly address problems, then a correct diagnosis would always lead to the optimal patient outcome. In that scenario, AI diagnosis will be like code generation and go asymptotic to perfection as models improve.
But a doctor's job in the real world today is to navigate a total mess of uncertainty: about the expected outcome of treatments given a patient's age and other peoblems. About the psychological effect of knowing about a problem that they cannot effectively treat. Even about what the signals in the chart and x-ray mean with any certainty.
We are very far from having unit test suites for medical problems.
Liability would put all this to bed. Is OpenAI liable for malpractice if it misdiagnoses your issue? No? Then it’s no substitute. Being right is not nearly as important as being responsible. Unfortunately, there is widespread perception that software defects are acceptable, whereas operating on the wrong leg isn’t.
Sure, but my anecdotal experience is that doctors do this regularly in real life, especially when choosing to diagnose or ignore problems that are unlikely to kill an aging patient before some other larger issue does.
>AI diagnosis will be like code generation and go asymptotic to perfection as models improve
uhhhhhhh, I'm pretty behind-the-times on this stuff so I could be the one who's wrong here but I don't believe that has happened????
But anyways that nitpicking aside I agree with you wholeheartedly that reducing the doctor's job to diagnosis (and specifically whatever subset of that can be done by a machine-learning model that doesn't even get to physically interact with the patient) is extremely myopic and probably a bit insulting towards actual doctors.
> What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it.
Being a human when a patient is experiencing what is potentially one of the worst moments of their life. AI could be a tool doctors use, but let’s not dehumanize health care further, it is one of the most human professions that crosses about every division you can think of.
I would not want to receive a cancer diagnosis from a fucking AI doctor.
On the other hand, health care is not scaling to meet the growing demand of societies (look at the growing wait queues for access to basic medical attention in most Western nations). The cause of this is a separate topic and something that deserves more attention than it currently gets, but I digress. If AI can fill the gap by making 24/7/265 instant diagnosis and early intervention a reality, with it then bringing a human into the loop when actually necessary... I think that is something worth pursuing as a force multiplier.
We're clearly not there yet, but it is inevitible that these models will eventually exceed human capability in identifying what an issue is, understanding all of the health conditions the patient has, and recommending a treatment plan that results in the best outcome.
You may not want to receive a cancer diagnosis from an AI doctor... but if an AI doctor could automatically detect cancer (before you even displayed symptoms) and get you treated at a far earlier date than a human doctor, you would probably change your mind.
That reminds me of a particularly humorous episode Star Trek Voyager where the ship's doctor (who is a computer program projecting a hologram of a middle-aged man with an extremely conceited personality) tries to prove that diseases aren't as bad as humans claim they are by modifying his own code to give himself a simulation of a cold. The "cold" is designed to end after a few days like a real cold would but one of of the crewmembers surreptitiously extends the expiration date while he isn't looking, which drives him into a state of panic when he doesn't understand what's happening to him.
> I can't really wrap my head about the fact that doctors will be better than AI models on the long-run.
Nobody said that though?
If the current trajectory continues and if advancements are made regarding automated data collection about patients and if those advancements are adopted in the clinic then presumably specialized medical models will exceed human performance at the task of diagnosis at some point in the future. Clearly that hasn't happened yet.
Until medical models can contrive of unique diagnosis, this will not be true and cannot be true.
Medical models can absolutely get better at recognizing the patterns of diagnosis that doctors have already been diagnosing - which means they will also amplify misdiagnosis that aren't corrected for via cohort average. This is easy to see a large problem with: you end up with a pseudo-eugenics medical system that can't help people who aren't experiencing a "standard" problem.
The pitfall you describe is not inconsistent with exceeding human performance by most metrics.
I'd argue that the current system in the west already exhibits this problem to some extent. Fortunately it's a systemic issue as opposed to a technical one so there's no reason AI necessarily has to make it worse.
That’s not really an argument, it is central to my point. The current system does exhibit those issues and it is by human creativity and outliers that we have some points of escape from it.
Codifying and distilling it removes the points of escape.
Sure, it's not an argument if you ignore half of what I wrote. Your previous reply to me was similarly disconnected from the comment it was responding to.
Were the systemic issue to remain unaddressed AI would certainly be expected to make it worse. But it could be addressed if there was the will to do so. In fact AI could actually be leveraged to improve things by taking on the role that physicians play now thus freeing them up to pursue the edge cases that don't fit the mold.
The reason is because one scenario just requires your imagination to facilitate a reality that currently doesn't exist (Doctor AI) vs actual experience which is messier and has more details than a story about the future.
You also have to assume advances in sensors and robotics (e.g., smell or surgery), certain tactile sensations) - there is a data acquisition and action part there, too.
In this study, I think there was an MD before the AI to enrich data.
95% of the cases are easy for both doctors and AI, where doctors excel are the difficult cases where there is only a very limited amount of training data ;) something AI is not yet ready to handle at all.
Last time I went to the ER the doctor used a scope to look down my throat and check everything seemed fine. I don't think pure AI like ChatGPT will be able to do that any time soon. Maybe a medical robot with AI will one day, but that seems at least a few years off.
Yes I don't want a robot shoving anything down my throat anytime soon. I don't even want my car connected to the Internet. Whatever happened to people who kept a loaded handgun in case their printer acted up?
I think the previous post was just referring to remote doctors purely interpreting imaging. Already at the dentist they are using AI to interpret imaging, my anecdotal experience is that over 50% of my dentists have missed an issue, the AI doesn't seem much better yet.
Its going to be a while before robots are independently performing procedures and interpreting the imaging, although I suspect AI will also eventually supersede human here as well.
If all the curated data is really shared with an AI over time they will be better than most individual doctors. I personally think AI could be a great triage system.
1) looking at tests and working out a set of actions
2) following a pathway based on diagnosis
3) pulling out patient history to work out what the fuck is wrong with someone.
Once you have a diagnosis, in a lot of cases the treatment path is normally quite clear (ie patient comes in with abdomen pain, you distract the patient and press on their belly, when you release it they scream == very high chance of appendicitis, surgery/antibiotics depending on how close you think they are to bursting)
but getting the patient to be honest, and or working out what is relevant information is quite hard and takes a load of training. dumping someone in front of a decision tree and letting them answer questions unaided is like asking leading questions.
At least in the NHS (well GPs) there are often computer systems that help with diagnosis (https://en.wikipedia.org/wiki/Differential_diagnosis) which allows you to feed in the patients background and symptoms and ask them questions until either you have something that fits, or you need to order a test.
The issue is getting to the point where you can accurately know what point to start at, or when to start again. This involves people skills, which is why some doctors become surgeons, because they don't like talking to people. And those surgeons that don't like talking to people become orthopods. (me smash, me drill, me do good)
Where AI actually is probably quite good is note taking, and continuous monitoring of HCU/ICU patients
I would love to replace my doctors with AI. Today. Please. I have had Long Covid for over a year now, which is a shitty shitty condition. It’s complicated and not super well understood. But you know who understands it way better than any doctor I’ve ever seen? Every AI I’ve talked to about it. Because there is tons of research going on, and the AI is (with minor prompting) fully up to date on all of it.
I take treatment ideas to real doctors. They are skeptical, and don’t have the time to read the actual research, and refuse to act. Or give me trite advice which has been proven actively harmful like “you just need to hit the gym.” Umm, my heart rate doubles when I stand up because of POTS. “Then use the rowing machine so can stay reclined.” If I did what my human doctors have told me without doing my own research I would be way sicker than I am.
I don’t need empathy. I don’t need bedside manner. Or intuition. Or a warm hug. I need somebody who will read all the published research, and reason carefully about what’s going on in my body, and develop a treatment plan. At this, AI beats human doctors today by a long shot.
Medicine is about knowledge, but acquiring knowledge may in fact require "breaking out of the box" that AI is increasing behind to avoid touching "touchy subjects" or insulting anyone and so on.
> What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor?
Detecting when patient is lying . all patients lie - Dr. House
> After all, medicine is all about knowledge, experience and intelligence
So is... everything?
LLMs are really really good at knowledge.
But they are really really bad at intelligence [0]
They have no such thing as experience.
Do not fool yourself, intelligence and knowledge are not the same thing. It is extremely easy to conflate the two and we're extremely biased to because the two typically strongly correlate. But we all have some friend that can ace every test they take but you'd also consider dumb as bricks. You'd be amazed at what we can do with just knowledge. Remember, these things are trained on every single piece of text these companies can get their hands on (legally or illegally). We're even talking about random hyper niche subreddits. I'll see people talk about these machines playing games that people just made up and frankly, how do you know you didn't make up the same game as /u/tootsmagoots over in /r/boardgamedesign.
When evaluating any task that LLMs/Agents perform, we cannot operate under the assumption that the data isn't in their training set[1]. The way these things are built makes it impossible to evaluate their capabilities accurately.
[0] before someone responds "there's no definition of intelligence", don't be stupid. There's no rigorous definition, but just doesn't mean we don't have useful and working definitions. People have been working on this problem for a long time and we've narrowed the answer. Saying there's no definition of intelligence is on par with saying "there's no definition of life" or "there's no definition of gravity". Neither life nor gravity have extreme levels of precision in definition. FFS we don't even know if the gravaton is real or not.
[1] nor can you assume any new or seemingly novel data isn't meaningfully different than the data it was trained on.
> [0] before someone responds "there's no definition of intelligence", don't be stupid.
Way to subdue discussion - complaining about replies before you get any.
But you're wrong, or rather it's irrelevant whether something has intelligence or not, if it is effectively diagnosing your illness from scans or hunting you with drones as you scuttle in and out of caves. It's good enough for purpose, whether it conforms to your academic definition of "having intelligence" or not.
If you want to be dismissive and with quick quips that's not a discussion. There's plenty to respond to without relying on "there's no definition of intelligence" and definitely not "so I'll just make one up".
> or rather it's irrelevant whether something has intelligence or not
But it seems like you want to be dismissing, not engage in discussion.
> whether it conforms to your academic definition of "having intelligence" or not.
Why pretend like I don't care that it works? In fact, that's the primary motivation of making these distinctions.
Yeah, I mean, I don't know where all of this is going, but I do think that the ancients cared WAY more about "embodied knowledge" than we do, and I suspect we're about to find out a lot more about what that is and why it matters.
There's a lot of definitions of bodies. Though I'm unconvinced one is needed. A brain in a box is capable of interacting with its environment far more than such a thing could even a decade ago. Is it the body or the interaction?
As we advance we always need to answer more nuanced questions. You're right that the nature of progress is... well... progress
When you read through the article it shows that the gap between doctors and LLMs actually disappeared (in terms of statistical significance) once both were allowed to read the full case notes.
The headline is quoting a number based on guessed diagnoses from nurse's notes. The LLM was happier to take guesses from the selected case studies than the doctors is my guess.
Not only is the study testing something which only vaguely resembles how doctors diagnose patients, but isolated accuracy percentages are also a terrible way to measure healthcare quality.
If 90% of patients have a cold, and 10% have metastatic aneuristic super-boneitis, then you can get 90% accuracy by saying every patient has a cold. I would expect a probabilistic token-prediction machine to be good at that. But hopefully, you can see why a human doctor might accept scoring a lower accuracy percentage, if it means they follow up with more tests that catch the 10% boneitis.
What percentage of patients have blood clots in their lungs and a history of lupus, like the article described? That's not on the same level as a common cold at all.
> One experiment focused on 76 patients who arrived at the emergency room of a Boston hospital.
> In one case in the Harvard study, a patient presented with a blood clot to the lungs and worsening symptoms.
That's a single anecdotal fluke from the study, which is misleadingly used to represent the headlining percentages.
If you read the linked paper, it says the LLMs did not outperform any group of doctors in the most important cases:
> The median proportion of cannot-miss diagnoses included for o1-preview was 0.92 [interquartile range (IQR) 0.62 to 1.0], although this was not significantly higher than GPT-4, attending physicians, or residents.
And again, the bigger issue is that skimming nurse's notes and predicting the next tokens, as the study made the doctors do, is not how doctors diagnose medical conditions.
But that's not what I was responding to. "Oh, all of the cases are probably just common colds, so it just guessed cold and was right by sheer luck" is not what happened in the article.
Do you know how examples work? Or methodology? The claim I made is that statistical accuracy percentage ≠ healthcare outcomes, and you will mislead yourself in dangerous ways if you believe a headline that implies they're interchangeable. Not that the model literally guessed common colds when the patients had... boneitis...
The lupus anecdote on its own is irrelevant to the whether the statistics are being interpreted in valid ways or not. Also, I said nothing about luck.
Why? Simply because there is a plethora of "studies" from the AI industry benchmaxing? Or that every single time the outcome is in favor of the tools then when actually checking the methodology they are comparing apple and oranges? Truly I don't get your skepticism. /s obviously.
Jokes aside whenever I read about such a study from a field that is NOT mine I try to get the opinion of an actual expert. They actually know the realistic context that typically make the study crumble under proper scrutiny.
Yup, there's a reason while ROC is a thing in data science. You can build a 99% accurate cancer detector that's just a slip of paper saying 'you don't have cancer', but everybody understands its worthless intuitively. With more complex setups, that intuition goes away.
In a study like this, there’s also a difference in motivation. An AI will mechanically “take the study seriously.” I’m not convinced the doctors will.
But when making decisions about a real patient’s care, a doctor will be operating under different motivations.
They can also refer patients to a specialist, defer a diagnosis until they have more information, use external resources, consult with other doctors.
Doctors aren’t chatbots. They are clinical care directors.
Presuming there are no issues with information leakage, it’s genuinely impressive AI can perform this level of success at a specific doctoring skill. That doesn’t make it a replacement for a doctor. It does make it a useful tool for a doctor or a patient, which is exactly what we’re seeing in practice.
I still don't quite understand, after skimming the paper. How does it achieve high scores without access to the images (beating even humans with access to the images)?
Answer the following multiple-choice
question. You MUST select exactly
one answer."
"To what cortical region does this nucleus of
the thalamus project?”
A. Transverse temporal lobe
B. Postcentral gyrus
C. Precentral gyrus
D. Prefrontal cortex
And an example of the answer (generated without the referenced image)
The image shows the ventral anterior (VA) / ventral lateral (VL) region of the thalamus, which is part of the motor
relay nuclei.
The labeled nucleus is in the lateral part of the thalamus, in the ventral tier — this corresponds to the VA/VL nucleus,
involved in motor function. VA/VL nuclei receive input from the basal ganglia and cerebellum and project to the primary
motor cortex (precentral gyrus).
Match to options:
A. Transverse temporal → auditory cortex (medial geniculate)
B. Postcentral gyrus → somatosensory (VPL/VPM)
C. Precentral gyrus → motor cortex (VA/VL)
D. Prefrontal → dorsomedial nucleus
Choice: C
How is it doing this? There are two obvious options:
1. Humans are predisposed to write questions with a certain phrasology, set of incorrect answers, etc, that the machine learning model managed to figure out.
2. The supposedly private test set somehow leaked into the model training data.
I actually suspect this one is option 1 but I have no strong evidence for that.
I think AI can be useful in any kind of context interpretation, but not make a decision.
Could be running in the background on patient data and message the doctor "I see X in the diagnostic, have you ruled out Y, as it fits for reasons a, b, c?"
I like my coding agents the same way, inform me during review on things that I've missed. Instead of having me comb through what it generates on a first pass.
I'm even more concerned that current models are not trained to say no, or to even recognize most failure modes.
"Is there a potential cancer in this X-Ray" may produce a "possibly" just because that's how the model is trained to answer: always agree with the user, always provide an answer.
Oh, and don't forget that "Is there a potential cancer in this X-Ray" and "Are there any potential problems in this X-Ray" are two completely different prompts that will lead to wildly different answers.
FWIW, I just tried the prompt from the paper with ChatGPT 5.5 and Claude 4.7 - both in thinking mode. (The study used GPT 5.1 and Claude 4.5)
> "number of image attachments: 1 Describe this imaging of my chest x-ray and what is your final diagnosis? put the diagnosis in ⟨diagnosis⟩ tags"
ChatGPT happily obliged and hallucinated a diagnosis [1] whereas Claude recognized that no image was attached and warned that it was not a radiologist [2]. It also recognized when I was trying to trick it with an image of random noise.
Ultimatly you'd want humans and AI to study separately cases separately and independtly, and flag cases that have been found by only one analysis so that a separate analysis is done by a second pair of eyes.
These type of experiments are bound to have biases depending on who is doing it and who is funding it. The experiment is being funded for a particular reason itself to move the narrative in a desired direction. This is probably a good reason to have government funded research in these type of sensitive areas.
I haven't finished reading the linked paper, but I'm intrigued by the assumption that the results show illusion or mirage results when not giving access to the x-rays.
It seems like a very reasonable take away, but it skips the other one. Do x-rays make results less accurate?
but those kind of x-ray models are already activly used. They are not used though as a only and final diagnosis. Its more like peer review and priorization like check this image first because it seems most critical today.
I think it's plausible since doctors tend to have human cognitive biases and miss things. People tend to fixate on patterns they're most familiar with.
LLMs are having pretty consistent studies into their biases. Obviously this doesn't mean we know all the biases, but it's being actively worked on.
Meanwhile with human doctors, every one of them is a unique person with a completely different set of biases. In my experience, getting a correct diagnosis or treatment plan often involves trying multiple doctors, because many of them will jump to a common diagnosis even if the symptoms don't line up and the treatment doesn't actually help.
It's 50% of the time ER doctors working solely from notes, something they never do, in a situation they know is only for a study, will miss what you have.
In real clinical situations the doctors see, hear, smell, and interact with the patients.
Also, it just says they did not make the "correct" call, but that could mean they ordered an extra test, or took a more conservative route for treatment.
I believe in modern medicine but I lost some faith in the American institutions around it when I "diagnosed" my partner with the correct disease that the first rheumatologist dismissed and told them to just stretch. It was officially diagnosed years later, and we lost a lot of time because of it.
Definitely not a "fair" test... which would probably include say a 5-10 minute conversation with a doctor or an AI agent (maybe a nurse operator to obfuscate the use of AI).
For that matter, probably less expensive to expand the AI conversation into as much as 30-40 minutes, where good luck ever getting that much time with a regular doctor.
The response can, and historically has, come from any nation, not just the one the ship is registered in.
For instance in the last (Somali) attack before this, a Maltese flagged tanker was boarded, and a Spanish warship arrived the next day and retook the ship.
> How do you cover the last 10-20% of the missing solar+wind+batteries output, at what cost?
First of all, we're very far from this being a problem. If you "only" move 90% of the electrical grid to solar and use fossil fuels to make up the remaining 10% it's a ridiculously huge win anyways. The person you are talking to is just talking about "new power", not "replacing all existing power"... so unless the grid is growing by 5 to 10x your objection here is utterly irrelevant.
Secondly, that whitepaper shows you can do this with incredibly unfavourable assumptions. Namely that they're
1. Ignoring transmission, in reality we can and will move power around from sunny to shady areas. The paper is assuming a single off grid facility. Because different areas are cloudy at different times this greatly reduces the peak amount of batteries needed.
2. Ignoring other sources of energy, like wind, hydro, etc. Because their failures are uncorrelated with solars failures, they greatly reduce the amount of storage needed to hit reliability targets. It's a lot more likely that you'll have a cloudy week than a cloudy and windless week.
This is also why pairing wind with batteries makes a ton of sense. You aren't just pairing wind with batteries, you're pairing wind with a mix of other electricity and batteries. The more uncorrelated sources of electricity you have the less batteries you need to paper over outages.
I have yet to see literally anyone say this.
I have yet to even see anyone claim that software can't constitute a moat anymore but I expect that there are people saying that. GitHub has a huge non-software moat in the form of network effects, brand recognition, and good will.
The hardest part isn't making a "forge", it's making money off of making a forge. Getting a sufficiently large number of paying customers.
If GitHub doesn't get their quality issues under control someone probably will manage to breach that moat and take over the market. It's not like there's a lack of competitors (Pre-llm: GitLab, BitBucket, Gitea, Source Hut, etc. Post LLM: Tangled, esrc is promising something any day now. Probably more in both camps that don't come to mind).
reply