← Registry
VerifiedBias in risk scoring2019
Widely used health risk algorithm scored Black patients as healthier than equally sick White patients
Consequence: In the studied health system, Black patients with the same risk score had more chronic illness than White patients. Correcting the gap would raise the Black share of patients auto-identified for extra care from 17.7% to 46.5%.
Claim
An algorithm used to select patients for care-management programs used past health costs as a stand-in for health needs. Because Black patients generated lower costs at the same level of illness, they received lower risk scores and were less often selected.
Evidence
A peer-reviewed study in Science (Obermeyer, Powers, Vogeli and Mullainathan, October 25, 2019) using records from a large academic hospital: 6,079 Black and 43,539 White patients, with the algorithm's inputs, outputs and objective all observable. At the 97th percentile of risk, Black patients had 26.3% more chronic conditions than White patients.
Causal strength
Direct and documented mechanism: the authors show the cost label causes the gap, and a revised label that blends health and cost predictions reduced the bias by 84% in their tests. The data come from one health system, so the exact figures describe that system.
Bears on
A case bears on an obligation when it informs how that obligation applies to AI. This is not legal advice and not a finding that anyone violated it.
Grade history
- November 21, 2026VerifiedInitial grade at publication
Sources
- Dissecting racial bias in an algorithm used to manage the health of populations, Science 366(6464)Primary · Peer-reviewed study
- PubMed recordSecondary · Index entry
Not affiliated with TruRate (trurate.com).