---
title: Lecture 23 --- More About Classifier Design
author: 36-462/662, Fall 2019
date: 18 November 2019
bibliography: locusts.bib
output:
  html_document:
    toc: true
---

\[
\newcommand{\Yhat}{\hat{Y}}
\newcommand{\Prob}[1]{\mathbb{P}\left( #1 \right)}
\newcommand{\indep}{\perp}
\]

## Recapitulation and notation-fixing

We're trying to predict a categorical response, label
or class $Y$, usually binary ($0$ or $1$), using a feature or vector of
features, $X$.  The prediction $\Yhat$ is a function of the feature $x$, so
we'll also write $\Yhat(x)$ for the guess at the label of a case with features
$x$.  We will be very interested in $\Prob{Y=1|X=x}$, and abbreviate it by
$p(x)$.  The over-all mis-classification error rate, or inaccuracy, is $\Prob{Y
\neq \Yhat}$.  This can be decomposed two ways.  One way is in terms of the
false positive rate, the false negative rate, and the "base rate" at
which the two classes occur:
\[
\Prob{Y \neq \Yhat} = \Prob{\Yhat=1|Y=0} \Prob{Y=0} + \Prob{\Yhat=0|Y=1}\Prob{Y=1} = FPR \times  \Prob{Y=0} + FNR \times \Prob{Y=1}
\]
The other decomposition is in terms of (one minus) the positive predictive value,
the negative predictive value, and the probability of making each prediction:
\[
\Prob{Y \neq \Yhat} = \Prob{Y=1|\Yhat=0}\Prob{\Yhat=0} + \Prob{Y=0|\Yhat=1}\Prob{\Yhat=1} = (1-PPV) \Prob{\Yhat=0} + (1-NPV) \Prob{\Yhat=1}
\]

If we want to minimize the inaccuracy, we should use the rule[^threshold] that
$\Yhat=1$ if $p(x) \geq 0.5$ and $\Yhat=0$ if $p(x) < 0.5$.  If we attach a
cost to each error, say $c_+$ to false positives and $c_-$ to false negatives,
then we should fix $\Yhat=1$ if $p(x) \geq t(c_+, c_-)$ and $\Yhat=0$ if $p(x)
< t(c_+, c_-)$.  The shape of the threshold function $t$ is such that $t=0.5$
when and only when $c_+ = c_-$.  (Can you remember, or work out, how to
find the threshold in terms of the costs $c_+$ and $c_-$?)

[^threshold]: Strictly, if $p(x)=0.5$, it doesn't matter what $\Yhat$ is, but I will always write $\geq$ for definiteness.

# Classification needs different distributions of features across classes

$p(x)$ is the distribution of the _class_ conditional on the _features_,
$\Prob{Y=1|X=x}$.  It turns out that the success of classification depends on
the "inverse" conditional probability, $\Prob{X=x|Y=y}$.  Let's introduce
the abbreviations
\begin{eqnarray}
f(x) \equiv \Prob{X=x|Y=1}\\
g(x) \equiv \Prob{X=x|Y=0}
\end{eqnarray}
(Some people would write $f_+$ and $f_-$, or $f_1$ and $f_0$, etc.)  Now
let's re-write the conditional probability of the class given the features, in
terms of the distribution of features in each class:
\begin{eqnarray}
p(x) & \equiv & \Prob{Y=1|X=x}\\
& = & \frac{\Prob{Y=1, X=x}}{\Prob{X=x}}\\
& = & \frac{\Prob{Y=1, X=x}}{\Prob{Y=1, X=x} + \Prob{Y=0, X=x}}\\
& = & \frac{\Prob{X=x|Y=1}\Prob{Y=1}}{\Prob{X=x|Y=1}\Prob{Y=1} + \Prob{X=x|Y=0}\Prob{Y=0}}
& = & \frac{f(x) \Prob{Y=1}}{f(x) \Prob{Y=1} + g(x)\Prob{Y=0}}
\end{eqnarray}

This is a bit of a mouthful, but we can get a handle on it by considering
some extreme cases.

## How distributions of features matter to classification

### Identical distributions $\Leftrightarrow$ No information $\Leftrightarrow$ No useful classifier

Suppose that $f(x) = g(x)$ for all $x$.  Then we'd get
\[
p(x) = \frac{\Prob{Y=1}}{\Prob{Y=1}+\Prob{Y=0}} = \Prob{Y=1}
\]
which is just the base rate.  So we'd have $\Prob{Y=1|X=x} = \Prob{Y=1}$.
But this is the same as saying that $Y$ and $X$ are statistically
independent, $Y \indep X$, or that the mutual information is 0, $I[X;Y] = 0$.
Whatever threshold $t$ we apply, we'd make the _same_ prediction for
everyone, regardless of their features.

### Non-overlapping distributions $\Leftrightarrow$ Easy classification

On the other hand, suppose that at some point $x$,  $f(x) > 0$ but $g(x) = 0$.   Then we have
\[
p(x) =\frac{f(x) \Prob{Y=1}}{f(x) \Prob{Y=1} + 0 \times \Prob{Y=0}} = 1
\]
and we should set $\Yhat=1$ here no matter what threshold we're using.  Similarly
if $f(x) = 0$ but $g(x) > 0$, we should set $\Yhat=0$
for any threshold.  If every point $x$ fell under one or the other of these
two cases, we'd say that the two distributions had "disjoint support", and
we could classify every point with certainty.

### In general...

Now let's consider the more general case, where we're applying a threshold
$t$ to $p(x)$.  The feature point $x$ is one where we set $\Yhat(x)=1$ when $p(x) \geq t$, or
\begin{eqnarray}
p(x) & \geq & t\\
\frac{f(x) \Prob{Y=1}}{f(x) \Prob{Y=1} + g(x)\Prob{Y=0}} & \geq & t\\
\frac{1}{1 + \frac{g(x)\Prob{Y=0}}{f(x) \Prob{Y=1}}} & \geq & t\\
1 + \frac{g(x)\Prob{Y=0}}{f(x) \Prob{Y=1}} & \leq & \frac{1}{t}\\
\frac{g(x)\Prob{Y=0}}{f(x) \Prob{Y=1}} & \leq & \frac{1-t}{t}\\
\frac{f(x) \Prob{Y=1}}{g(x)\Prob{Y=0}} & \geq & \frac{t}{1-t}\\
\frac{f(x)}{g(x)} \frac{\Prob{Y=1}}{\Prob{Y=0}} & \geq & \frac{t}{1-t}
\end{eqnarray}

Notice that this involves the base-rates ($\Prob{Y=0}$, $\Prob{Y=1}$) as well
as the distribution of features in each class.  One way to interpret this last
equation is that the ratio $f(x)/g(x)$ gauges the evidence (one way or the
other) that the features $x$ provides about the class membership of a
particular case.  How strong this evidence has to be before it pushes us into a
decision depends on the threshold ($t$), but also on what we know about the
base-rates.  If the base-rates are very lop-sided, say $\Prob{Y=0} \gg
\Prob{Y=1}$, then we'd need stronger evidence (higher ratio $f(x)/g(x)$) to
push us into saying $Y=1$, i.e., to setting $\Yhat=1$.

## Information again

Let's revert to information theory.  We've seen that we can't classify at all
when $I[X;Y] = 0$ (at least not using $X$).  More generally, $I[X;Y] =
H[Y]-H[Y|X]$, and $H[Y|X]$ is precisely the entropy of the distribution $p(X)$
(averaged over $X$).  So larger values of $I[X;Y]$ mean smaller values of
$H[Y|X]$, which in turn means that $p(x)$ is closer to 0 or 1 for more values
of $x$, which means that more and more accurate classification is possible.
When we consider classification trees, we will look directly at trying to
(greedily) minimize $H[Y|X]$.

### Feature selection

Folklore in data mining is that you should worry more about finding the right
features than about exactly what model or classifier you apply to them.  We've
now seen the rational basis for this folklore.  Informative features are ones
whose distributions differ substantially across classes.  With un-informative
features, it doesn't matter which technique we apply, there's just no
information to be had.  Once you've found informative features, it's fairly
routine to try out different standard techniques (linear classifiers, nearest
neighbors, trees, etc., etc.) on the same feature set.  In any particular
problem, some of these techniques will have an easier time extracting
information in the features.  But the information has to be there in the
features to begin with.

# Trading off false positive rates against false negative rates

Another way to tackle the problem of classifier design is to frankly admit that
we have _two_ competing objectives, and try to balance them.  One objective is
to have low false positive rates, and the other objective is to have low false
negative rates.  For historical and rhetorical reasons, it's conventional
here to consider one minus the false negative rate,
\[
\Prob{\Yhat=1|Y=1}
\]
which is also called the "power", and written $\beta$.

## The optimization problem

When we chose a classifier, we get a benefit in the form of power, $\beta$, and
pay a cost in terms of the false positive rate, FPR.  To get a handle on this,
remember that for every point $x$, either $\Yhat(x) = 1$ or $\Yhat(x) = 0$.
Call the set of points where $\Yhat(x) = 1$, $S$.  (This is called the
**acceptance region** or **decision region**, and its boundary is called the
**decision boundary** or **critical boundary**.)  Then
\[
\beta(S) \equiv \Prob{\Yhat=1|Y=1} = \sum_{x \in S}{f(x)} ~ \text{or} ~ \int_{S}{f(x) dx}
\]
depending on whether the features are discrete or continuous.  (I did the
continuous version in class, but will do the discrete version here, just to
encourage mental flexibility on your part.)  This is our over-all benefit
to using a classifier with the decision region $S$.  Against this, the cost,
in the form of false positives, is
\[
FPR(S) = \Prob{\Yhat=1|Y=0} = \sum_{x \in S}{g(x)}
\]

Every possible classifier gives us some combination of power and false positive
rate.  We can imagine plotting power against false positive rate for each
classifier.  Some classifiers are bad: they have high false positive rates and
low power.  Many other classifiers will "dominate" those bad ones, because they
have a lower false positive rate and higher power.  But then there will be
classifiers which are harder to rank: one has a strictly higher power, but the
other has a strictly lower false positive rate.  There it's ambiguous which one
we should prefer.  On our plot, we'll see a curve along which increasing the
benefit (power) can only happen if we also increase the false positive rate.
This curve is called the **possibility frontier**, or the **Pareto
frontier**[^Pareto].  To get some grasp of this, think about saying $\Yhat=1$
for every $x$, i.e., setting $S$ to be the whole feature space.  This would
give $\beta=1$, but also $FPR=1$.  Against that, setting $\Yhat=0$, or
shrinking $S$ to the empty set, would give $FPR=0$, but also $\beta=0$.  So
we'd usually[^nonoverlapping] expect the frontier to connect $(0,0)$ to
$(1,1)$.


[^Pareto]: After Vilfredo Pareto, an economist who pioneered the study of optimization under competing objectives.

[^nonoverlapping]: If the distributions $f$ and $g$ don't overlap, we can get 0 FPR with positive power, and/or power 1 without FPR 1.

Now the usual way to deal with two competing objectives is to introduce a
**price**, letting us say how much one unit of one objective is worth in
terms of the other.  Here the two objectives having high power and low
false positive rate, so we need to set a price for power in terms of
false positives.  Let's call this price $r$.  So we want to maximize
\[
\beta - r \times FPR
\]
or
\[
\sum_{x\in S}{f(x)} - r \sum_{x \in S}{g(x)} = \sum_{x \in S}{f(x) - rg(x)}
\]
Now each summand is either positive, negative or 0.  To make the _sum_ as big
as possible, we should include every $x$ where the summand is positive in the
set $S$, and exclude every $x$ where the summand is negative.  (We don't care
about points where the summand is 0.)  That is, what we want to do is,
\[
\Yhat(x) = \left\{ \begin{array}{cc} 1 & f(x) - rg(x) \geq 0\\
                                     0 & f(x) - rg(x) < 0
	               \end{array}
	        \right.
\]

## Constraining false positive rates

We might start instead by saying that we want to maximize power with a _constraint_ on the false positive rate.  That is, we set as our problem
\[
\max_{S: FPR(S) \leq \alpha}{\beta(S)}
\]
where $\alpha$ is our maximum allowed false positive rate: maybe $\alpha=0.05$,
or $\alpha=0.01$, or $\alpha=10^{-6}$ if we really don't like false positives.

(Graphically, if we go back to our plot of false positive rate versus power,
we'd be drawing a horizontal line at $FPR=\alpha$, only considering points
below that line, and then just taking the right-most point, which is the
highest power classifier that obeys the constraint.)

The usual trick for solving constrained optimization problems is to turn
them into unconstrained ones by using a Lagrange multiplier.  We had
the old **objective function**
\[
\beta(S) = \sum_{x \in S}{f(x)}
\]
subject to the constraint
\[
FPR(S) \leq \alpha
\]
or
\[
FPR(S) - \alpha \leq 0
\]
or
\[
\sum_{x\in S}{g(x)} - \alpha \leq 0
\]
The **Lagrangian** combines the objective function plus a **Lagrange multiplier** times the constraint equation:
\[
\beta(S) - r(FPR(S) - \alpha)
\]
The actual problem is now to maximize the Lagrangian over _both_ $S$ and the Lagrange multiplier,
\[
\max_{S, r}{\beta(S) - r(FPR(S) - \alpha)}
\]
Equivalently,
\[
\max_{S, r}{\sum_{x \in S}{f(x)} - r\sum_{x \in S}{g(x)} + r\alpha}
\]

Now, when we do the maximization, there will be _some_ value of the multiplier
$r$ which will enforce the constraint $\alpha$, say $r^*$.  And once we know
$r^*$ what we're doing is just
\[
\max_{S}{\sum_{x\in S}{f(x) - r^* g(x)}}
\]
and we've seen how to do that: include $x$ in $S$ if, but only if,
\[
\frac{f(x)}{g(x)} \geq r^*
\]
In other words, the Lagrange multiplier looks just like a price.  Economists
call a price which enforces a constraint a "shadow price", and so $r^*$ is the
shadow price of power.

### Summing up on the power vs. FPR trade-off

If we want to enforce a limit on the false positive rate, $FPR \leq \alpha$,
but then maximize the power, we should follow the rule
\[
\Yhat(x) = \left\{ \begin{array}{cc} 1 & f(x)/g(x) \geq r\\
                                     0 & f(x)/g(x) < r
	               \end{array}
	       \right.
\]
for some threshold $r$ which is a function of $\alpha$.

Notice that _this_ classification rule does not involve the base rates at all.
This is sensible, since neither the false positive rate nor the power involves
the base rate.


### Likelihood ratios, hypothesis tests, the Neyman-Pearson lemma

In your other statistics classes, you'll have seen a lot of hypothesis tests,
many of which involve the likelihood ratio, or (equivalently) the log of the
likelihood ratio.  The way we've analyzed classifiers is exactly parallel to a
hypothesis test.  $g(x)$ the likelihood of the features $X=x$ under the null
hypothesis that $Y=0$, while $f(x)$ is the likelihood under the alternative
hypothesis that $Y=1$.

It may seem intuitive that a test should compare the likelihoods, but why
compare them through their ratio rather than say $f(x) - g(x)$, or for that
matter $f^2(x) - g^2(x)$?  We have now seen the answer: the ratio is, uniquely,
what we need to worry about if we want to maximize power while controlling the
false positive rate.  As a result about hypothesis testing, this was first
shown by Jerzy Neyman and Egon Pearson in the 1930s, and so it's sometimes
called "the Neyman-Pearson lemma" or "the Neyman-Pearson theorem".

# Classifier design

Our analyses of classifier performance has suggested three approaches to how we
might design classifiers.

1. Estimate $p(x)=\Prob{Y=1|X=x}$ and threshold that function.
2. Estimate $\frac{f(x)}{g(x)} = \frac{\Prob{X=x|Y=1}}{\Prob{X=x|Y=0}}$, perhaps by first estimating $f(x)$ and $g(x)$, and threshold that function.
3. Pick the region $S$ where $\Yhat=1$ (or, equivalently, the boundary
of this region) to have low inaccuracy.

Strategy (1) relies on what's called the **posterior probability** of being in
each class.  (The **prior probability** of being in class 1 is just the base
rate, $\Prob{Y=1}$.)  Strategy (2) relies on the likelihood ratio, and is
sometimes called a "Neyman-Pearson" approach.  Strategy (3) is what we might
call a "direct" approach, though I don't think it has a common name.

As always, there are advantages and disadvantages to each approach.

## The posterior-probability (feature-conditional) approach

**Pro**:

- We just need to estimate the distribution
of a one-dimensional, indeed binary, variable $Y$, conditional on the features
$X$.
   + Basically, we need to have some way of defining "similar" features $x$,
and then estimate $\Prob{Y=1|X=x}$ by the proportion of $Y=1$ cases among
the similar $x$'s.
   + We will see in the next lecture some ways of dividing
up the feature space which aim to have this property, and to make it fast
and easy to find similar $x$'s.
- If we care about our _average_ or _expected_ costs, we've seen that the optimal thing to do is to threshold $p(x)$, so this is the right way to control expected costs.
- It's sensitive to the base rates, so it can achieve low over-all inaccuracy.

**Cons**:

- It relies (implicitly) on the base rates, so if the data are very un-balanced,
it will be hard for it to advance over just always predicting the more
probable class.
- It relies (implicitly) on the base rates, so there'll
be trouble if those change.
- We need to decide on the relative costs of the two kinds of errors.  (This
choice is implicit in the choice of the threshold we apply to $p(x)$.)

## The Neyman-Pearson (class-conditional) approach

**Pros**:

- It's insensitive to base rates, so it doesn't really care about un-balanced data or
to changes in the base rates.
- If we specifically need to keep a lid on the false positive rate, it's exactly the right thing to do.
  + Imagine "false positive" is something like "Keeping a harmless person in
    jail", or "Denying a loan to someone who'd repay it".
- We really only need the _ratio_ $f(x)/g(x)$, and there are tricks for estimating that without first estimating $f(x)$ and $g(x)$.

**Cons**:

- It's insensitive to the base rates, so it usually won't have as high an over-all accuracy as the posterior-probability approach.
- We have to decide on an acceptable false-positive rate $\alpha$, or equivalently a price for power in terms of false positives, and this can feel arbitrary.

## The direct (chose-a-region) approach

```{r, echo=FALSE}
set.seed(2019-11-18)
x1 <- runif(n=30)
x2 <- runif(n=length(x1))
probs <- exp(-(x1^2+x2^2))
y <- rbinom(n=length(x1), size=1, prob=probs)
plot(0, xlim=c(0,1), ylim=c(0,1), xlab=expression(x[1]),
     ylab=expression(x[2]),
     type="n")
text(x=x1, y=x2, labels=y)
```

To illustrate, imagine seeing data like the figure above, and trying to
classify it with a linear classifier.  That means trying to find a straight
line which divides the positive, $Y=1$ points from the negative, $Y=0$ points.
Typically, when people try to do this, they aim to find the line with the
minimum classification error.  You can convince yourself that there is no
single straight line which will achieve an error rate of 0 on that data, but
also that some lines are better than others.  In symbols, we have a
family[^class] $\mathcal{S}$ of possible decision regions $S$, and we want to
find
\[ \min_{S \in \mathcal{S}}{\frac{1}{n}\sum_{i=1}^{n}{Y_i \neq \mathbb{1}(X_i \in S)}} \]
where the indicator function $\mathbb{1}(X_i \in S) =1$ if $X_i \in S$ and
$=0$ otherwise.

[^class]: The more usual jargon word here is "class", which I used in lecture, but this collides with "class" for whether $Y=1$ or $Y=0$, so I'll try to avoid it

There is nothing magic about using linear classifiers.  We might, for instance,
instead try to divide the points by enclosing all the positive points inside a
rectangle, and then (in this case) we could do better than with purely linear
separators.  This amounts to changing the family of possible regions
$\mathcal{S}$.  Many families are defined through complicated functions of the
features --- a popular choice is to take a bunch of nonlinear transformations
of the features, say $\phi_1(x), \phi_2(x), \ldots \phi_p(x)$, bundle them into
a vector $\phi(x)$, and then apply linear classifiers[^circle] to $\phi$.

[^circle]: As an example, if $x$ is two dimensional, and $\phi(x) = (x_1, x_2, x_1^2, x_2^2)$, a linear classifier applied to $\phi$ can pick out points inside (or outside) a circle, which you couldn't do with the raw features.  (What would the linear classifier for $\phi$ look like?)

In general, the bigger and more flexible we make $\mathcal{S}$, the lower we
will make the in-sample error, and the lower the generalization error _can_ be.
But there are two costs to using very flexible families.

1. There is a computational cost to searching for the best $S$ over a big family.
2. There is a real danger of over-fitting, or, as we say, "memorizing the noise" in the training data.

(1) is real but task-specific; (2) is a more general statistical issue, so I'll
elaborate on it.  Suppose we could use _very_ flexible regions, say a union of
a large number of square boxes.  Then we can get a region $S$ which looks like
the next figure.

```{r, echo=FALSE}
plot(0, xlim=c(0,1), ylim=c(0,1), xlab=expression(x[1]),
     ylab=expression(x[2]),
     type="n")
# Draw little grey boxes around each 1 (a horrible idea as a real
# classifier)
positive.points <- which(y==1)
box.width <- 0.1
rect(xleft=x1[positive.points]-box.width/2,
     ybottom=x2[positive.points]-box.width/2,
     xright=x1[positive.points]+box.width/2,
     ytop=x2[positive.points]+box.width/2,
     col="lightgrey", border=NA)
text(x=x1, y=x2, labels=y)
```

If there is any noise in the labels at all, the $S$ we get will, in part,
reflect that noise.  We say that $S$ "memorizes" the noise, as well as any true
signal about where the classification boundary should be.  The next figure
shows the same classification boundary, with the same values of the features,
but with $Y$ drawn independently from the same distribution $p(x)$, and you can
see that it's not doing so well --- it now misses some positive points and
includes some negative points.

```{r, echo=FALSE}
y.prime <- rbinom(n=length(x1), size=1, prob=probs)
plot(0, xlim=c(0,1), ylim=c(0,1), xlab=expression(x[1]),
     ylab=expression(x[2]),
     type="n")
rect(xleft=x1[positive.points]-box.width/2,
     ybottom=x2[positive.points]-box.width/2,
     xright=x1[positive.points]+box.width/2,
     ytop=x2[positive.points]+box.width/2,
     col="lightgrey", border=NA)
text(x=x1, y=x2, labels=y.prime, col="red")
```

So flexibility or "capacity" has an advantage (we can fit more) but also a
disadvantage (our results are less stable and more vulnerable to noise).


One way to combat the vulnerability to noise is to impose some sort of
geometric constraint.  A common one is to require a certain minimum distance
between any point and the classification boundary --- to insist on only using
classifiers with a large "margin".  Such geometric constraints rule out shapes
with very irregular, erratic boundaries, which is (one reason) why people talk
about this as "regularizing" the problem of finding an optimal classifier.
Just as we saw above, constraining classifiers to have a large margin is the
same as penalizing classifiers based on their margin.  Either way, we would
typically use cross-validation to pick the size of the constraint or the
penalty.

(People have developed various ways of measuring model capacity, to help
quantify this trade-off.  Many of them come down to variations on seeing how
well the family could seem to classify labels which were pure noise, i.e., how
well it would seem to do when $p(x)=0.5$ for all $x$.  These are important for
theory, have helped guide the development of new classifiers, and provide
important sanity checks on how well we can hope to do: see for instance
@Bartlett-Mendelson-on-Rademacher-complexity.  But in practice, people
overwhelmingly use cross-validation to assess their classifiers.)



**Pros**:

- In the direct, region-finding approach, we don't _explicitly_ need to estimate
  any probabilities.  We are still, implicitly, guessing at where $p(x)
  \geq 0.5$ (or whatever our threshold is), but we're just trying to find that
  boundary, rather than estimating the whole function $p(x)$.
  + We can also apply the shape-finding approach in a more Neyman-Pearson way:
    we'd first find the regions in $\mathcal{S}$ with acceptably low false
    positive rates, and then among them find the region with the highest power.
- For many families, there are very efficient algorithms for quickly finding
  the optimal classification region (perhaps with regularization).

**Cons**:

- Even though we're not explicitly considering costs of different kinds of
  errors, a certain ratio between the costs is implicit in the error rate
  we decide to minimize.  We might not like that ratio if we think about
  it consciously.
- Because we're not estimating any probabilities, just finding a decision
  region leaves us uncertain about what to do if costs shift (or we change our
  mind about costs).
- Because we're not estimating any probabilities, we don't have any sense of
  _confidence_ in our classifications (are we 99% sure this is positive case,
  or just 51% sure?).

# Summing up so far

1. We can use features $X$ to predict class labels $Y$ when, and only when,
   the distribution $\Prob{X=x|Y=y}$ changes with the label $y$.  The bigger
   the difference in those distributions, the more ability we will have to
   do classification on the basis of $X$.
2. Error rates involve the ratios of distributions
   $\Prob{X=x|Y=1}/\Prob{X=x|Y=0}$ (and possibly the base rates as well).
3. Classifier design strategies can be understood as either trying to estimate
   (and threshold) the posterior probability $\Prob{Y=y|X=x}$, as trying to
   estimate (and threshold) the likelihood ratio
   $\Prob{X=x|Y=1}/\Prob{X=x|Y=0}$, or as trying to directly find a decision
   boundary with good error rates.  All three strategies have their
   advantages and disadvantages.

# Predicting demographics from web search

@Goel-Hofman-who-does-what-on-the-web used the full browsing history of about a
quarter of a million (US) Web users primarily to examine how different
demographic groups --- defined by age, sex, race, education and household
income --- used the Web differently.  If we think of each demographic category
as a label $Y$, and which websites were visited (and how often) as features
$X$, they were primarily interested in $\Prob{X=x|Y=y}$, and how this differed
across demographic categories.  For instance, people with a post-graduate
degree visited news sites about three times as often as people with only a high
school degree.  (What's $X$ and what's $Y$ in that example?)  It may or may
not surprise you to learn that they found large differences in browsing
behavior across demographic groups.  To steal an example from the paper,
men are much more likely than women to visit ESPN, and women are more likely
than men to visit Lanc&ocirc;me.

You can now see where this is going.  By point (1) in our summary above, the
fact that $\Prob{X=x|Y=y}$ differs across classes $y$ means that we can use
browsing behavior ($X$) to predict demographic classes ($Y$).  Someone
who knows what websites you browse can predict your age, sex, race, education,
and household income.  To demonstrate this, @Goel-Hofman-who-does-what-on-the-web used the 10,000 most popular websites, creating a binary feature for each site, $X_i=1$ if site $i$ was visited at all during the study and $X_i=0$ if not.
They then used a linear classifier on these features, with one of the geometric
margin constraints I mentioned.  The next figure shows how well they were
able to predict each of the five demographic variables. 

![](goel-et-al.png)

_Detail of Figure 8 from @Goel-Hofman-who-does-what-on-the-web, showing the ability of a (regularized) linear classifier to predict demographic variables based on web browsing history.  Dots show the achieved accuracy, and the $\times$ shows the frequency of the more common class._

I include this not because the precise accuracies matter ---
there's no reason to think this is the best performance attainable, even with
these features --- but rather to prove the point that this kind of prediction
can be done.  It doesn't matter _why_ different demographic groups have
different browsing habits, just _that_ those distinctions make a difference.
This lets us (or our machines) work backwards from browsing to
accurate-but-not-perfect inferences about demographic categories.

## Inference

Now imagine a recidivism prediction system which does not, officially or
explicitly, consider sex, but _does_ have access to the defendant's web
browsing history.  (No such system exists, to best of my knowledge, but there's
no intrinsic limit on its creation.)  We know, from Goel et al., that sex can
be predicted with about 80% accuracy from browsing history (at least).  A
nefarious designer who wanted to include sex as a predictor for recidivism, but
to hide doing so, could therefore use browsing history to predict sex, and then
include predicted sex in their model.  A less nefarious designer might end up
doing something equivalent without even realizing it, say by slightly
increasing the predicted risk of those who visit ESPN and slightly reducing the
prediction for those who visit Lanc&ocirc;me.  Either designer might, when
pressed, say that they're not claiming to say _why_ ESPN predicts recidivism,
but facts and facts, and are you going to argue with the math?

In fact, we can go further.  We know that younger people have a higher risk of
violence than older people, that poorer people have a higher risk than richer
people, that men have a higher risk than women, that blacks have a higher risk
than whites[^levoy-allen], that less educated people have a higher risk than more educated
people[^crime].  A system which just used Web browsing to sort people into
these five demographic categories could[^hedging], therefore, achieve
non-trivial predictive power.  You can even imagine designing such a system
innocently, where we just try to boil down a large number of features into
(say) a five-dimensional space, before using them to predict violence, without
realizing that those five dimensions correspond to age, sex, race, income and
education.

[^levoy-allen]: There are multiple reasons for this association.  One is a long-standing history (cf. @Dollard-on-Southerntown) of segregating African-Americans into neighborhoods which are under-policed (in the sense that violence often goes unpunished by the forces of the law) and over-policed (in the sense that interactions with the police are often hostile).  This sets up a dynamic where people in those neighborhoods don't trust the police, which makes the police ineffective, which makes being known for willingness to use violence a survival strategy, which etc., etc.  @Leovy-ghettoside gives a good account of this feedback loop from (mostly) the side of the police; @Allen-cuz gives a glimpse of what it looks like from the other side.

[^crime]: Cathy O'Neil would remind us that many of these would flip around if we considered risk of _financial_ crimes rather than violence.

[^hedging]: I say "could", because there's some error in all these classifications, and it's _possible_ that these errors would cancel out the ability to predict violence from demographics.

None of this really relies on the features being Web browsing history;
anything whose distribution differs across demographic groups will do.

# Some more reading


On the Neyman-Pearson approach to classifiers, see @Scott-Nowak-NP-approach,
@Rigollet-Xin-NP-classification and
@Tong-plug-in-Neyman-Pearson-classification.  The Neyman-Pearson lemma itself
goes back to @Neyman-Pearson-np-lemma (where they never call it a lemma).  The
heuristic cost-benefit derivation of it is, so far as I know, my own invention.



# References

