“Algorithmic Bias and Physician Liability”
by Shujie Luan
Southeast University
Place: MA-330
Abstract
Artificial intelligence (AI) is increasingly being used in the development of clinical algorithms to support clinical decision making. Their potential bias, as reflected in disparities in accuracy across patient populations, has received growing attention. The U.S. Centers for Medicare and Medicaid Services (CMS) recently proposed an anti-bias liability rule stating that providers may be liable for erroneous medical decisions made in reliance on biased algorithms. In this paper, we model and analyze how such anti-discrimination liability rules affect both upstream AI development decisions (by an AI firm) and downstream AI deployment decisions (by a physician). The AI firm first decides on the algorithm’s accuracy for two types of patients, where training the algorithm for the disadvantaged patients requires higher development costs; in response, the physician decides whether and how to use the algorithm, which may be biased against the disadvantaged patient, when prescribing a treatment plan. Using a biased algorithm can help reduce clinical uncertainty, but may expose the physician to legal liability in the event of a treatment error. We show several results with important policy implications. First, anti-discrimination liability can lead to discriminatory use of AI by inducing the physician to (1) underuse AI and (2) disproportionately reject AI recommendations for disadvantaged patients.
Second, we show a non-monotonic effect of liability on the physician’s decision to use AI: as liability increases, the physician is less likely to use AI for disadvantaged patients and then more likely to use it. Finally, we show mandating equal accuracy can make all patients worse off, because it removes liability concerns and leads to more AI use, but the physician may overuse AI for disadvantaged patients.
Bio
Shujie Luan is a final-year Ph.D. candidate in Management Science and Engineering at Southeast University, specializing in Revenue Management & Pricing and Medical AI. Her research develops data-driven algorithms for assortment and pricing optimization to address real-world business challenges, while also exploring the societal impacts of Human-AI interaction in healthcare settings. Her work has been published in top-tier journals, including Production and Operations Management and Omega, and she has also secured multiple research grants to support her projects.
