‘I think my blackness is interfering’: does facial recognition show racial bias?
The latest research into facial recognition technology used by police across the US has found that systems disproportionately target vulnerable minorities
Cameras are used routinely by police across the US to identify citizens, their faces cross-matched against databases of suspects and past criminals.
Yet researchers claim there is too little scrutiny of how these tools work, and have found inherent racial bias in the system. So does a sophisticated, visual analysis tool reflect human prejudice and if so, who does that effect?
Studies indicate theres racial bias in the software, said Jonathan Frankle, staff technologist at Georgetown Law School. Working with law fellow Clare Garvie, Frankle has requested public information from more than 100 police departments across the country. We want to know: is there disparate impact on vulnerable communities?
This new study, published on 7 April with a set of 10,000 pages of information from public records requests, is the most comprehensive one on modern police use of facial recognition.
The newest and potentially most intrusive way police are using facial recognition software is with their cellphones, according to Garvie and Frankle. A police officer can point a smartphone at someone during a stop-and-frisk and, using this software, attempt an identification.
The databases they use are disproportionately African American, and the software is especially bad at recognizing black faces, according to several studies.
Bias in facial recognition software has been highlighted before.
HPs MediaSmart webcam included facial recognition software so that the camera could move to follow the position of the user. In 2009, two co-workers in a retail store highlighted how the camera would pan to follow a white face, but stop as soon as her black co-worker entered the picture.
I think my blackness is interfering with the computers ability to follow me the worst part is I bought one for Christmas, said the man, who identified himself as Desi, while demonstrating the fault on video. HP said the software had difficulty identifying facial features in lower light levels, blaming standard algorithms that measure the difference in intensity of contrast between the eyes and the upper cheek and nose.
In 2010, Microsofts newly released Kinnect motion-sensing device couldnt recognize black faces either. And in a horrific coding blunder in July 2015, Google Photos, which scans photos and then suggests peoples identity, misidentified two black people as gorillas.
Theres awareness that the public would have some hesitance about the software, Garvie said. So its not something agencies are necessarily forthcoming about.
Though there has been only scant research on facial recognition, the studies that do exist are troubling to Frankle and Garvie. One group found that software made in east Asia is better at identifying east Asian faces, while software made in North America is better at identifying white faces. The lead researcher, Jonathon Phillips, said it was just a matter of what the machines had learned on.
Artificial intelligence learns from examples it was trained on, so if you dont feed it the correct variety of faces it may not be able to recognize the world you put it out in, Phillips said. If you do not include many images from one ethnic subgroup, it wont perform too well on those groups.
Phillips hopes tech teams think about how they are training their artificial intelligence: When people develop technology they think of the immediate problems theyre solving, not who the user community is. There comes a point when youre developing technology that you need to ask that question.
Garvie and Frankle will present their research at The Color of Surveillance symposium, which explores the racial bias of government monitoring, on 8 April. They dont think the software engineers or police departments are necessarily malicious, but that there does need to be more transparency.
We dont know where theyre located, how many there are, whos enrolled, what database the police use and where, Frankle said. We can sum up all the work were doing pretty easily: we dont know, and thats a problem.