36-303: Sampling, Surveys and Society Exam 1 Solutions

1. [20 pts] Multiple Choice

(a) [5 pts] Which of the following statements is \underline{most correct} :

iii. It is possible to construct a representative sample without random sampling, but it is more difficult to

argue that it is really representative.

(b) [5 pts] Which one of the following is the \underline{best} justification for informed consent for survey participants, according to our textbook: ii. To give respondents meaningful control over information about themselves.

(c) [5 pts] Which of the following is \underline{not} true, for a simple random sample with replacement from the target

population?

i. You can get a more representative sample by increasing n, regardless of the response rate.

(d) [5 pts] When making a public report on a survey, which of the following is \underline{not} required?

iii. The name of the statistical package used to do the analyses.

2. [18 pts] N = 2, 505; the survey question is

Should persons taken into custody always be told their Miranda rights (‘You have the right to remain silent; anything you say can and will be used against you in a court of law [etc.]’) Yes / No”.

(a) [6 pts] Let p = P[Yes] Assuming SRS without replacement, estimate sample size n needed to estimate p within a margin of error of ±0.07, with a 95% confidence interval?

First find the SRS with replacement estimate. We need n ≥ n_{0} = \frac{z_{α/2}^{2}(SD)^{2}}{(ME)^{2}}. Since we don’t know p we take the worst-case p = 1/2, yielding SD = \sqrt{p(1 − p)} = 1/2, so we get

n ≥ n_{0} = \frac{(1.96)^{2}(0.5)^{2}}{(0.07)^{2}} = 196

Now make the SRS without replacement adjustment, which is

n ≥ \frac{Nn_{0}}{N + n_{0}} = \frac{(2505)(196)}{2505 + 196} = 181.78

so take n ≥ 182 .

Note: if you use z = 2 instead of z = 1.96, you will get n_{0} = 204.08, and n ≥ 189; this is fine.

(b) [6 pts] You call 50 officers, they all agree to talk with you, and of the 50, 20 officers say “Yes”. Make a 95% CI for p. p̂ = 20/50 = 0.40;

For sampling with replacement we would get SE_{with}= \sqrt{p̂(1 − p̂)/n} = \sqrt{(0.4)(0.6)/50}= 0.0693

For sampling without replacement we get SE_{without}= \sqrt{1 − n/N}SE_{with}= 0.0686 (not a lot of difference in this case).

So the confidence interval runs from p̂ − 1.96(SE) = 0.40 − (1.96)(0.0686) = 0.266 to p̂ + 1.96(SE) = 0.40 + (1.96)(0.0686) = 0.534, i.e. the CI is (0.27, 0.53) .

Note: if you use z = 2 instead of z = 1.96, your interval will be (0.26,0.54).

(c) [6 pts] Now suppose that you call a random sample of 200 officers from the complete list of 2,505 officers, but only 50 agree to be interviewed over the phone. Of those 50, 20 respond “yes”. You calculate the same confidence interval as in part (b).

Which of the confidence intervals (this one, or the one in part (b)) is harder to believe as evidence about the whole population of Allegheny full-time police officers? Why? In part (b) everyone who was surveyed responded: r/n = 1 = 50/50 = 1. Therefore we have no reason to think the people who responded are any different from any other random sample we might have gotten.

In part (c) only 50 of the 200 responded: r/n = 50/200 = 0.25. Even though the calculation is the same in this case, we are less sure that it represents the population, because of possible differences between the 50 who responded and the 150 who did not (indeed, one difference is that they were willing to respond!).

Therefore (c) is harder to believe.

3. [18 pts] Simple Random Sampling (3 parts).

(a) [6 pts] Carefully define Simple Random Sampling (SRS) \underline{with} replacement, and give an example. Give enough detail that it is obvious that this is a good example.

An SRS with replacement is one where each element of the sample is obtained as an independent, equally-likely draw from the whole population. In particular the same population element can appear more than once in the sample

For example consider an urn with 30 yellow balls and 70 blue ones. Mix up the balls in the bottom of the urn, tip it over enough to let one ball go up the neck, record the color of that ball, and let it fall back in the urn. Repeat 10 times for an SRS with replacement of size n = 10.

(b) [6 pts] Carefully define Simple Random Sampling (SRS) \underline{without} replacement, and give an example. Give enough detail that it is obvious that this is a good example.

An SRS without replacement is one in which, after each population element is drawn for the sample, it is not put back in the population. The remaining elements in the population are then equally likely to appear next in the sample.

For example consider an urn with 30 yellow balls and 70 blue ones. Mix up the balls in the bottom of the urn, tip it over enough to let 10 balls go up the neck. Since the 1^{st} ball up the neck is not replaced into the population when the 2^{nd} ball goes up the neck (and similarly for the first two balls when the third ball goes up the neck, etc.) this produces a SRS without replacement of size n = 10.

(c) [6 pts] Show that

\overline{y} = \frac{1}{n} ∑_{i=1}^{n} Y_{i} = \frac{1}{n} ∑_{i=1}^{N} Z_{i}y_{i}

is an unbiased estimator for

μ = \frac{1}{N} ∑_{i=1}^{N} y_{i}

E[\overline{y}] = E [\frac{1}{n} ∑_{i=1}^{N} Z_{i}y_{i}] (definition of \overline{y}  )

= \frac{1}{n} ∑_{i=1}^{N} y_{i}E[Z_{i}]  (E[aX + bY] = aE[X] + bE[Y]  )

= \frac{1}{n} ∑_{i=1}^{N} y_{i} (\frac{n}{N})  (E[Z_{i}] from formula sheet  )

= \frac{1}{n} (\frac{n}{N}) ∑_{i=1}^{N} y_{i}  (algebra  )

= \frac{1}{N} ∑_{i=1}^{N} y_{i}  ( ⋮  )
= μ  (definition of μ  )

4. [20 pts] Psych students asking 1,520 passersby in NYC variouos “help” questions.

(a) [6 pts] What, if any, ethical principles do you think this study violates?

Any of the following will be marked as correct:

iii. Respect for Persons; Confidentiality iv. Informed Consent vi. Not listed above; I think Lying to respondents (e.g. the students don’t really need help).

though I think the one really best answer is “iv. Informed Consent”.

(b) [8 pts] Do you think the results of this survey will be representative of the degree of altruistic compliance of all New Yorkers?

No, not representative. If you select this answer, use the space below to suggest one or more modifications of the survey design that would make it more representative.

Man on the street surveys are seldom representative. To make this one representative either of the following suggestions (and possibly others) would receive full credit:

• Find a list of New Yorkers and take an SRS without replacement; invite the people in the sample to come to an office for an interview, and part of the interview “experience” is to be asked for help from strangers.

• Select a specific set of locations and times to be “on the street”, so that taken together, the people you meet at these locations are likely to be repreentative of all New Yorkers^{1}. Have a “protocol” for approaching pedestrians at each location. E.g. do a coinflip for every 10th person who passes, to decide if they will be in the survey, regardless of gender, socio-econdomic statis, etc. (So you don’t inadvertently bias your results by only approaching friendly-looking people, for example.)

(c) [6 pts] Sometimes a study that is not perfect can still be useful. List one or two possible advantages of doing the study as designed, rather than modifying it in any way. Be clear and complete, not just a word or two. Full credit for either of the following answers (and possibly others):

• Since the students are asking for help in a variety of ways, we get to find out which ways of asking the questions work well or poorly for the purpose of the survey (it is like 100’s of “focus groups” for each student’s way of asking for help).

• If the responses of the passersby are carefully recorded, they may be reviewed to develop hypotheses about people’s behavior, to test in a more rigorous study later; they could also be used to try different ways of coding responses for a future (perhaps more rigorous) study.

Note that both of these reasons basically view this study as a kind of “pilot study” for a later, more scientifically rigorous study.

1This is like “stratification”, which we will discuss further later in the course.

5. [24 pts] Fixing survey questions.

(a) “Most people think that abortion is immoral. Do you agree?”

i. [3 pts] A Potential Problem: This is a leading question. Many people will answer “agree” because of the peer pressure implied by the first statement, or just to “please” the interviewer.

ii. [3 pts] Suggestion(s) For Rewrite:

One possible fix would be to ask the question directly

“Do you believe that abortion is immoral?” Another possible fix would be to give the respondent more options for a nuanced answer:

“What is your position on abortion?

• Abortion is immoral and should never be allowed.

• Abortion is immoral but certain exception should be made (e.g. if the mother’s life is in danger)

• Abortion is not immoral but it should be rare

• Abortion is not immoral and there should be no laws restricting it.”

Either answer would be fine for full credit.

(b) “Do you agree or disagree that NCLB should be abolished and replaced with merit pay for teachers?”

i. [3 pts] A Potential Problem: Two problems (either one for full credit):

• Options before substance of question

• Question depends on unfamiliar terms like “NCLB” and “merit pay”. ii. [3 pts] Suggestion(s) For Rewrite:

Put the agree/disagree options at the end; and/or define the terms as you ask the question (the following is a little wordy but on the right track):

“As you may know, NCLB (the No Child Left Behind Bill of 2001, mandating testing in schools to improve learning) is due to expire soon, and legislators are thinking of ways to improve it. One way is to replace the punishments that schools receive if their students don’t test well with “merit pay” for teachers, so teachers get paid less if their students do poory, more if their students do well. Do you favor a shift from NCLB to a merit-pay system?”

(c) “How many times have you taken a PAT bus to go shopping since the start of classes at CMU in August?”

i. [3 pts] A Potential Problem: This is asking respondents to remember how many times they did something over a very long period of time. This sort of recall exercise is difficult for people to do accurately.

ii. [3 pts] Suggestion(s) For Rewrite:

One possible fix would be to ask over a shorter more recent period of time (e.g. “the last month”)

(d) “What kind of exercise do you do?

Walking, jogging or running Exercise machines or weights”

i. [3 pts] A Potential Problem:

It is a \underline{closed-response item with too few options} (what about the swimmers? basketball players? etc.) ii. [3 pts] Suggestion(s) For Rewrite:

Add more options, and/or add an open-ended “if none of the above, write how you like to exercise here” option.