Fairness Tradeoffs

A fairness claim should name its metric, outcome, population and decision use. Chouldechova’s definitions separate:

• Calibration: the same score has the same recorded outcome frequency across groups.
• Predictive parity: flagged cases have equal positive-outcome proportions.
• Error-rate balance: false-positive and false-negative rates are equal across groups.^[1]

False-positive rates condition on negative outcomes; predictive value conditions on positive predictions. Changing the denominator changes the question.

A worked example

This is original teaching arithmetic for a generic binary outcome, not criminal-justice data. Each group contains 1,000 cases. Suppose a classifier catches 80% of positives and mistakenly flags 10% of negatives:

Both groups have 10% false positives and 20% false negatives, but different proportions of true positives among flags. Equal error rates therefore do not establish predictive parity in this example. Conversely, an equal proportion of positives among flags does not tell us how frequently negatives were flagged. That requires another denominator.

If we instead flag 160 and 400 true positives with zero false positives, both predictive values become 100%, while false-negative rates remain 20%. This second hypothetical calculation shows why an impossibility claim needs its conditions.

What the theorems do and do not settle

Equation 2.6 relates the quantities at a defined operating point. Differing base rates can prevent simultaneous parity; boundary cases require separate attention.^[1]

Kleinberg and colleagues’ theorem concerns group calibration and class-conditional average scores; its exceptions include perfect prediction and equal base rates. It supplies conditional constraints, not a verdict that fair institutions are impossible.^[2]

Use the COMPAS reading path and Preference Aggregation to separate measurement from the choice of a socially legitimate objective. Social Alignment asks whose interests constrain use.