Can AI substitute a human physician’s second opinion? Recently the Journal of Medical Ethics published two contrasting views: Kempt and Nagel advocate for using artificial intelligence (AI) for a second opinion except when its conclusions significantly diverge from the initial physician’s while Jongsma and Sand argue for a second human opinion irrespective of AI’s concurrence or dissent. The crux of this debate hinges on the prevalence and impact of ‘false confirmation’—a scenario where AI erroneously validates an incorrect human decision. These errors seem exceedingly difficult to detect, reminiscent of heuristics akin to confirmation bias. However, this debate has yet to engage with the emergence of explainable AI (XAI), which elaborates on why the AI tool reaches its diagnosis. To progress this debate, we outline a framework for conceptualising decision-making errors in physician–AI collaborations.

AI and XAI second opinion: the danger of false confirmation in human–AI collaboration

Stuckler, David
In corso di stampa

Abstract

Can AI substitute a human physician’s second opinion? Recently the Journal of Medical Ethics published two contrasting views: Kempt and Nagel advocate for using artificial intelligence (AI) for a second opinion except when its conclusions significantly diverge from the initial physician’s while Jongsma and Sand argue for a second human opinion irrespective of AI’s concurrence or dissent. The crux of this debate hinges on the prevalence and impact of ‘false confirmation’—a scenario where AI erroneously validates an incorrect human decision. These errors seem exceedingly difficult to detect, reminiscent of heuristics akin to confirmation bias. However, this debate has yet to engage with the emergence of explainable AI (XAI), which elaborates on why the AI tool reaches its diagnosis. To progress this debate, we outline a framework for conceptualising decision-making errors in physician–AI collaborations.
In corso di stampa
2024
Rosenbacke, Rikard; Melhus, Åsa; Mckee, Martin; Stuckler, David
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11565/4073220
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 5
  • ???jsp.display-item.citation.isi??? 4
social impact