← Registry

VerifiedBias in risk scoring2019

Widely used health risk algorithm scored Black patients as healthier than equally sick White patients

Consequence: In the studied health system, Black patients with the same risk score had more chronic illness than White patients. Correcting the gap would raise the Black share of patients auto-identified for extra care from 17.7% to 46.5%.

Claim

An algorithm used to select patients for care-management programs used past health costs as a stand-in for health needs. Because Black patients generated lower costs at the same level of illness, they received lower risk scores and were less often selected.

Evidence

A peer-reviewed study in Science (Obermeyer, Powers, Vogeli and Mullainathan, October 25, 2019) using records from a large academic hospital: 6,079 Black and 43,539 White patients, with the algorithm's inputs, outputs and objective all observable. At the 97th percentile of risk, Black patients had 26.3% more chronic conditions than White patients.

Causal strength

Direct and documented mechanism: the authors show the cost label causes the gap, and a revised label that blends health and cost predictions reduced the bias by 84% in their tests. The data come from one health system, so the exact figures describe that system.

Bears on

A case bears on an obligation when it informs how that obligation applies to AI. This is not legal advice and not a finding that anyone violated it.

Grade history

  • November 21, 2026VerifiedInitial grade at publication

Not affiliated with TruRate (trurate.com).