Tech giant Amazon is set to announce a year ban on law enforcement authorities to use its controversial Rekognition facial recognition platform, Amazon said on Wednesday as per the Verge report.
The decision was taken after Joy Buolamwini, a researcher at the MIT Media Lab, and Timnit Gebru, a member at Microsoft Research maintained in his 2018 paper that the chances of biases and errors with such tech tools were higher against dark-skinned people. This is due to the fact that the data used to design the system had been collected from an overwhelming number of white men, according to the New York Times report.
Amazon’s announcement came two days after IBM announced that it would no longer research or develop and facial recognition system citing potential human rights and privacy abuses. According to their research, facial recognition tech, despite the advances provided by artificial intelligence, remained biased along lines of age, gender, race, and ethnicity, The Verge report added.
In a separate 2019 study, Buolamwini and co-author Deborah Raji examined Amazon’s Rekognition and found that the system is not immune to such discrimination. It mistook black women for men. The system worked with a near-zero error rate when analysing images of lighter-skinned people, the study found.
Amazon tried to undermine the findings, but Buolamwini posted a lengthy and detailed response to Medium, in which she said: “Amazon’s approach thus far has been one of denial, deflection, and delay. We cannot rely on Amazon to police itself or provide unregulated and unproven technology to the police or government agencies.” Her and Raji’s findings were later corroborated by a group of dozens of AI researchers who penned an open letter saying Rekognition was flawed and should not be in the hands of law enforcement.
The tech giant did not give a specific reason behind the moratorium. However, it said that it would continue providing the software to rights organisation dedicated to curbing child pornography and human trafficking.
This came on the backdrop of the death of George Floyd, a black man killed by former Minnesota police officers, and ongoing protests around the US and the globe against racism and systemic police brutality.
The Verge report said that It seems as if Amazon decided police cannot be trusted to use the technology responsibly, although the company has never disclosed just how many police departments do actually use the tech. As of last summer, it appeared like only two departments— one in Oregon and one in Florida — were actively using Rekognition, and Orlando has since stopped.