Bias Complications: Removing human bias in AI can be challenging

The debate that has erupted over a study in the Journal of the American Medical Association (JAMA) that says a million Black adults in the US could receive timely treatment for kidney disease if a controversial ‘race-based correction factor’ is removed from algorithms used in diagnosis and decision to start medication shows how challenging the issue of removing human bias in AI could get.

Nature reports that the correction factor was introduced in the late-1990s to reflect research results that showed Black Americans tend to have higher levels of creatinine in the blood—high creatinine levels, along with other factors, lead to a lower estimated glomerular filtration rate (eGFR), which is associated with kidney disease. With this in mind, a multiplier of 1.2 was introduced in the calculation of eGFR for Black people in the algorithms used for diagnosis; this could cause kidney disease to get masked or seem less serious in certain cases.

While those who have called to drop the correction factor argue that stalling treatment for certain members of a marginalised community reflects racial bias, even though unintended, those who hold that creatinine predispositions need to be factored in, though not necessarily tied to race, say that over-diagnosis (as also false positives) due to removal of the factor will mean pushing a community with high levels of poverty into further poverty, and reduce access to vital medicines—renal disease makes administration of certain drugs, such as anti-diabetes ones, risky.

While the US reports a higher likelihood of kidney disease among Black people than other ethnicities, whether this has been worsened due to the correction factor, it is difficult to say. What this does indicate is how challenging tackling human-bias-contagion for AI is going to be.