Face Recognition Vendor Test (FRVT), Part 3: Demographic Effects

Patrick Grother, Mei Ngan and Kayee Hanaoka evaluate demographic differences in identity verification and identification. Many, but not all, tested algorithms exhibit false-positive disparities; effects vary by algorithm and image data (executive summary).^[1]

What does a candidate mean?

The report distinguishes identity recognition from gender classification. Its investigation example returns a fixed number of candidates even when the correct person is absent. Human-machine interaction and reviewer efficacy are outside its scope (printed pp.4–5).^[1]

Compare the Williams incident for the operational question of what evidence follows a lead. Benchmark findings cannot diagnose that specific arrest.

For a comparison of evaluation questions, follow Gender Shades and the researcher entry points Joy Buolamwini and Timnit Gebru. Check each study’s task before comparing its measurements.

Historical context

NISTIR 8280 is dated December 2019. Selected executive summary and background passages inspected, including rendered printed p.5. Full annexes, code, underlying datasets and the deployed police system were not audited.^[1]