We're in charge, as data scientist and software engineers, of software that where mistakes have a very high price. It's time to look at the impact we have, how our software will be used and how to avoid creating unfair, biased and dangerous software
50. Photo: Fadil Elmansour
AREA W. HIGH CRIME
SEND MORE POLICE
DO MORE ARRESTS
APPEARS TO HAVE MORE CRIME
SEND MORE POLICE
DO MORE ARRESTS
RUNAWAY FEEDBACK LOOP
23% no proof
Living on a certain block can label you as Crip or Hells Angel
Used to justify arrest, support maximum penalties
No right to know if you’re on it, or to challenge it
Used for background checks for jobs
Quote from a song by Tom Lehrer
The AI Gaydar paper
Wang, Y., & Kosinski, M. (2018). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images
50 clicks on facebook
Used by Cambridge analytica
81% men, 74% women
Data from dating websites
Fluidity, what determines someones preferences? Bi? Bi-curious? outspoken
Long tail, criminal in k countries, death penalty,
Criminal or Not = 90%
1856 Images of Chinese Men 18-55, No facial hair or scars – faces cut out730 -- Criminals – ID From Police Department 235 Violent Crimes,
536 Theft, Fraud, Corruption, Forgery, Racketeering1126 -- Random images from internet – “corporate photos” as Non-Criminalsnote: 90%, SAME NETWORK ARCHITECTURE WAS ONLY ABLE TO PICK UP GENDER IN 86%
Criminals
Non-criminals
Criminal or Not = 90%
Gender w. AlexNet = 86%
Harm can further be classified into five types: stereotyping, recognition, denigration, under-representation and ex-nomination.
Classification is always a product of its time
We are currently in the biggest experiment of classification in human history
Kate Crawford
Wouldn’t be able to tell one Wookie from the next
Wife: Malla, Son: Lumpy
Areas with more crime get more policing => more crimes reported => considered worse neighborhood, more policing
Scrubbing to neutral
What is neutral – gender, ethnicity, are people from lapponia counted as finnish, sexual orientation, political orientation?
What about words like pregnancy, or color blindness? Queen?
Who decides?
Everyone is a majority somewhere and a minority somewhere else