Cameras and Scanning: A Case Study
Detroit shows how modern technology can lead to a virtual police state.
Yesterday, I questioned whether we should be alarmed about “FBI and ICE Using Drivers License Photos for Facial Recognition Scans.” A follow-up report from the NYT, looking at how a similar program is being used in Detroit, provides strong push-back.
Twenty-four hours a day, video from thousands of cameras stationed around Detroit, at gas stations, restaurants, mini-marts, apartment buildings, churches and schools, streams into the Police Department’s downtown headquarters.
The surveillance program, which began in 2016, is the opposite of covert. A flashing green light marks each participating location, and the point of the popular initiative, known as Project Green Light, has been for the cameras to be noticed and help deter crime. Detroit’s mayor, Mike Duggan, received applause when he promised at his State of the City address earlier this year that expanding the network to include several hundred traffic light cameras would allow the police to “track any shooter or carjacker across the city.”—-“As Cameras Track Detroit’s Residents, a Debate Ensues Over Racial Bias“
As I noted yesterday, I’m more concerned about the rise of the surveillance state, per se, than the pairing of the surveillance with ID photos. Detroit’s program strikes me as ominous: almost a literal police state.
But the report focuses on an issue that yesterday’s report in the Washington Post only touched upon: the disparate impact of the practice on racial minorities.
In Detroit, whose share of black residents is larger than in any other sizable American city, it is a racial disparity in the performance of facial recognition technology that is a primary source of consternation.
“Facial recognition software proves to be less accurate at identifying people with darker pigmentation,” George Byers II, a black software engineer, told the police board last month. “We live in a major black city. That’s a problem.”
Researchers at the Massachusetts Institute of Technology reported in January that facial recognition software marketed by Amazon misidentified darker-skinned women as men 31 percent of the time. Others have shown that algorithms used in facial recognition return false matches at a higher rate for African-Americans than white people unless explicitly recalibrated for a black population — in which case their failure rate at finding positive matches for white people climbs. That study, posted in May by computer scientists at the Florida Institute of Technology and the University of Notre Dame, suggests that a single algorithm cannot be applied to both groups with equal accuracy.
Mr. Byers and other critics spoke at a public hearing called by the Detroit Board of Police Commissioners after what the board called unprecedented public interest in two facial recognition items on its agenda. One item, specific to the new traffic light cameras, was approved last week. The other, a comprehensive “acceptable use” policy for facial recognition, has yet to be put to a vote.
Not everyone who spoke was against the use of facial recognition.
“I’m the pastor getting the call from mothers whose son was shot or their baby got snatched up,” said Maurice Hardwick, a black pastor at a nondenominational ministry who founded a group that works with high school gang members. “People want to know two things: What happened to my child, my loved one? And who did this?”
Another Detroit resident, a white woman who walked with a cane, added: “If you’re afraid of the cameras, either you’re paranoid or you’ve got something to hide.”
I’m sympathetic to Hardwick’s view and afraid of the second. The notion that we should be afraid of state surveillance only if we have “something to hide” is fascist.
Still, the debate seems to be on cost-benefit analysis moreso than liberty:
Others were more concerned with a provision that would allow the police to go beyond identifying violent crime suspects with facial recognition and allow officers to try to identify anyone for whom a “reasonable suspicion” exists that they could provide information relevant to an active criminal investigation. There was also concern that the photograph of anyone who gets a Michigan state ID or driver’s license is searchable by state and local law enforcement agencies, and the F.B.I., likely without their knowledge.
Facial recognition, the Detroit police stress, has indeed helped lead to arrests. In late May, for instance, officers ran a video image through facial recognition after survivors of a shooting directed police officers to a gas station equipped with Green Light cameras where they had met with a man now charged with three counts of first-degree murder and two counts of assault. The lead generated by the software matched the description provided by the witnesses.
Back to the race issue:
When James White, an assistant police chief in charge of the Detroit Police Department’s technology, rose to respond to critics at the public hearing, he provided unexpected backup to the charge that the software comes with baked-in bias. He himself, the assistant chief said, had been misidentified as other African-American men by the facial recognition algorithm that Facebook uses to tag photos.
“On the question of false positives — that is absolutely factual, and it’s well-documented,” he said. “So that concerns me as an African-American male.”
The solution, Chief White said, is to exercise extra care. The department’s policy specifies that facial recognition will be used only to investigate violent crimes. Although the department has the ability to implement real-time screening of anyone who passes by a camera — as detailed in a recent report by the Georgetown Law Center on Privacy and Technology — there is no plan to use it, he said, except in extraordinary circumstances.
No one in Detroit, Chief White emphasized, would be arrested solely on the basis of a facial recognition match.
“Facial recognition technology isn’t where the work stops,” he said. “It’s where the work starts.”
While that seems reasonable enough, it’s obviously a standard that won’t always be followed. And false positives are still false positives.
Civil liberties advocates say that protection isn’t enough, especially because defendants are not typically informed that facial recognition has been used in their identification. In one of the few cases to have argued that such information should be disclosed because it is potentially exonerating, a Florida appeals court ruled that a black man, Willie Allen Lynch, had no legal right to see the other matches returned by the facial recognition program that helped lead to his drug-offense conviction. Mr. Lynch had argued that he was misidentified.
A January 2018 study by two M.I.T. researchers first focused public attention on the higher misidentification rates for dark-skinned women by three leading purveyors of facial recognition algorithms. One of the co-authors, Joy Buolamwini, posted YouTube videos showing the technology misclassifying famous African-American women, like Michelle Obama, as men. The phenomenon, Ms. Buolamwini wrote in a New York Times Op-Ed, is “a reminder that artificial intelligence, often heralded for its potential to change the world, can actually reinforce bias and exclusion, even when it’s used in the most well-intended ways.”
The companies examined in the paper subsequently improved their algorithms for that particular test. But a second paper this year found that Amazon’s software had more trouble identifying the gender of female and darker-skinned faces, prompting prominent artificial-intelligence researchers to call on the company to stop selling its software to law enforcement agencies. Amazon executives have disputed the study.
It is not clear why facial recognition algorithms perform differently on different racial groups, researchers say. One reason may be that the algorithms, which learn to recognize patterns in faces by looking at large numbers of them, are not being trained on a diverse enough array of photographs.
But Kevin Bowyer, a Notre Dame computer scientist, said that was not the case for a study he recently published. Nor is it certain that skin tone is the culprit: Facial structure, hairstyles and other factors may contribute.
In Dr. Bowyer’s experiments, the recognition algorithms could achieve the same degree of accuracy for white and black Americans, but only when the algorithm was tuned to a cutoff, say, of no more than one in 10,000 false matches for the two separate groups. Given that the norm is to use the same threshold for everybody, “those programs are seeing a higher false match rate for the population of African-Americans,” Dr. Bowyer said.
A dual-threshold system would not necessarily solve the problem, he added. That would require law enforcement authorities to make a judgment about each individual’s race and apply the appropriately tweaked facial recognition software — which would in turn introduce human bias.
“Technically, it’s a very reasonable thing to say to do,” Dr. Bowyer said. “But how do you defend it, and once you put that knob out there for police to use, how do you make sure it’s not misused?”
One presumes that the technical problems can be alleviated, if not solved, with further tweaking. Still, the phenomenon is troubling.
Overall, I’m still not persuaded that the tool is inherently dangerous or a threat to our civil liberties. I remain more concerned about the ubiquity of monitoring cameras than I am with their pairing with photo databases. Alas, I suspect the demands to install more cameras—for safety, dontchaknow–will outstrip concerns over privacy.