Neuron Makers · Ch. 15 · Bigotry
In 2018 Joy Buolamwini and Timnit Gebru audited three commercial face-classification systems. On lighter-skinned men all three were nearly flawless, erring on fewer than one in a hundred. On darker-skinned women the error rate climbed to as high as 34.7% — better than one wrong guess in three. The systems failed worst on the people least represented in their training data.
Across all three vendors the worst-served group is always the same. The gap between the best-served group (lighter men) and the worst (darker women) runs to more than 30 percentage points — on the same task, inside the same product.