
When we talk about classification problem, we always mean that we’re trying to predict a categorical, usually binary, label, outcome or class $$Y$$ from features $$X$$ (which may or may not be categorical).

We’ll be a little more telegraphic today.

1 “Protected attributes”

• Legally mandated in some contexts
• US law generally prohibits discrimination in employment, housing, lending, education, etc., on the basis of race, ethincity, sex, religion, national origin, or age
• Obvious exceptions are in fact part of the law: don’t appeal to laws against age discrimination to try to drink underage, laws against religious discrimination don’t force a Baptist church to hire a Muslim as a priest (but might mean the church couldn’t discriminate against a Muslim janitor)
• OTOH there’s nothing in US law against (for example) discrimination by caste
• Arguably ethically mandated everywhere
• I am not going to try to tell you what to believe about ethics, but I do want you to think about whether what you are doing with data mining is ethical

2 Some notions of “fairness” for classification

1. Don’t use protected features directly
1. Sometimes called “anti-classification”
2. What about strongly-associated unprotected features?
2. Have equal measures of error across groups
1. Sometimes called “classification parity”
2. Which error measures, exactly?
3. Calibration: everyone with the same score should have the same actual probability of $$Y=1$$, regardless of group
1. Conditional independence of $$Y$$ from protected attribute given score
2. This is a very weak requirement (which some systems nonetheless manage to violate…)

2.1 Concrete example: Pretrial detention, recidivism prediction

• You don’t get arrested, your screw-up cousin Archie gets arrested
• Court decides whether or not to keep Archie in jail pending trial or let Archie go (perhaps on bail)
• Court wants Archie to show up and not do any more crimes
• $$Y=1$$: Archie will be arrested for another crime if released
• $$Y=0$$: Archie will not be arrested
• Similarly for failure to appear on trial date, arrest for violence, etc.
• We’ve worked with data like this before

2.2 Notation

• Archie’s features $$=X = (X_p, X_u)$$ where $$X_p$$ are the protected features and $$X_u$$ are the unprotected ones
• $$Y=$$ whether or not Archie will be arrested for another crime before trial
• Or: will show up for trial, will be re-arrested after being released from prison, will default on the loan, …
• Generally, today, $$Y=1$$ is the bad case
• $$\Yhat(x) =$$ prediction we make about someone with features $$x$$
• Here $$\Yhat=1$$ means “we predict re-arrest” (or recidivism), and so someone paying attention to use would presumably not release this person
• $$\Yhat(x)$$ can ignore some features in $$x$$
• $$p(x) = \Prob{Y=1|X=x}$$ is the true risk function
• Cost-minimizing $$\Yhat(x)$$ thresholds based on $$p(x)$$, and the costs of both errors
• Note that true risk function isn’t known
• $$s(x) =$$ risk score we calculate based on $$x$$ (which may or may not be an estimate of $$p(x)$$)

3 “Anti-classification”

Anti-classification means: don’t use protected categories to make these decisions/predictions. To make this useful, we’re going to need to be more precise.

3.1 Option I: No direct use of protected features

In the first formalization, the prediction our system makes must be the same for any two inputs with the same features:

Not using protected features means: if we make different predictions for cases $$x$$ and $$x^{\prime}$$, $$\Yhat(x) \neq \Yhat(x^{\prime})$$, then we must have $$x_u \neq x_u^{\prime}$$.

Equivalently, if $$x_u = x^{\prime}_u$$, then $$\Yhat(x) = \Yhat(x^{\prime})$$, we can’t make different predictions if two cases just differ in their protected features.

In the risk-prediction context, this would mean that we can’t explicitly look at race, sex, or national origin, say.

3.1.1 Inference from unprotected to protected features

The rule “don’t use the protected features” is often seen as too weak to be satisfying. Sometimes this is for (debated and debatable) ethical reasons1, but there is also the more technical concern, which is that it is often very easy to infer the values of protected features from the values of unprotected features. Thus for instance knowing what zipcode someone lives in is (in the US) a very good proxy for race (as well as for education and income, which are not, strictly speaking, legally protected features). Knowing what census tract they live in would be an even better proxy for race.

As a small example, the dot in the next figure shows the census tract where I grew up, in Bethesda, Maryland. (The dot is not on my parents’ house.) The inset table shows some summary demographic information for that tract (current for 2019) from the Census Bureau. (It’d also be easy to get information about income, housing conditions, education levels, etc.) Even if we know nothing else about someone, knowing that they’re from census tract 23224-24-031-7044.04 allows us to predict their race with 70% accuracy.