Statistics 36-303: Sampling, Surveys and Society Sample Survey Project Instructions and Rules

Outline of Groups Sample Survey Projects

Milestone  Due Date

Phase I

0. Team Requests Fri Jan 15 (*)
0. Team Assignments  Tue Jan 19
1. Propose Two Topics  Tue Jan 26
2. Target Population, Frame, Mode of Data Collection Tue Feb 2  (*)
3. Choose Topic & Write Final Proposal, Begin IRB Tue Feb 9 (*)

Sign & Turn in Team Working Agreement Phase II

4. Sampling Scheme & Question Design  Tue Feb 16
5. Pretest & Revise Questions  Tue Feb 23
6. Final IRB & Project Plan Tue Mar 2 (*)

Phase III

7. Progress Report on Data Collection & Analysis  Tue Mar 30
(Meeting of the Minds Registration  Thu Apr 1)
8. Rough Draft of Final Report  Tue Apr 6
9. Final Report Due Fri Apr 30 (*)
Oral Presentations  Apr 20–Apr 29
(Meeting of the Minds  Wed May 5)

(*) = Major item(s) to be turned in on these dates.

1 Getting Started on a Project

Milestone  Due Date

Phase I

0. Team Requests  Fri Jan 15
0. Team Assignments Tue Jan 19

1.1 Creating a Team

Create a team of four individuals. You had just until Friday January 15 to do this. Over the weekend I made teams up out of all of the people who didn’t find teams yet.

Once you know what names are on your team, you still have the task of forming those names into a team. How well one’s team works together is often one of the more satisfying or dissatisfying aspects of the project. Here are some ideas and questions that some students in similar settings have found helpful in clarifying their expectations of each other as they were forming their teams:

• Make sure that there are times in the week when everyone on the team get get together as a single group.

• What do you expect each other to contribute?

• How will you contact each other about the meetings?

• How much advance notice will you give (or do you need) before meetings?

• Be specific. Write down your agreements. The “Team Working Agreement” assignment will give you a chance to earn some points for doing this.

1.2 The Lazy Player

Inevitably, there will be a few individuals who do little or no work for their group, but yet want to claim a full share of credit for the group’s work. I consider this cheating. To try to cut down on this problem, I will occasionally ask you to evaluate the work of the other members in the group, and in particular whether one member is not doing his or her fair share.

1.3 The Question

Make sure that the question your team chooses to investigate has these properties:

A. It is of interest to you. B. This question can be answered using sample survey methods discussed in this course. C. The entire project can be done in approximately 80 hours total for the team.

1.4 Typing up Project Work

Follow these typing instructions for all work (Potential Topics and Team Working Agreements, Final Proposal, Final Report, etc.):

A. Use only 8.5”×11” paper format, and use at least 11 point type in a Times Roman or similar font. B. You are expected to do your data analyses and to prepare tables and graphs by computer (recall that

R and Minitab are readily available on campus and can be installed on your own computer).

• All relevant computer output must be labelled, e.g. “Figure 1”, “Figure 2”, etc. Similarly for tables.

• A figure or table should appear only if there are one or more sentences in the paper referring to that figure or table.

• All figures and tables must have captions explaining enough so that a reader can “get the main point” without going back to the text of the paper.

Your work will typically mix text, computer output, tables, figures, etc. It is better to place these elements near the sentences that refer to them in the text, rather than collecting them all at the back of the paper.

C. Organize your answers in numbered and labelled sections. It is critical that you do this so that

your work can be accurately graded. D. Staple your pages together. Paper-clipped, folded or loose pages run the risk of being lost or mis-

placed, and then you will not get credit for them.

E. Number your pages. This is especially a problem in MS Word, which defaults to no page numbers.

Force pages to be numbered. F. Include the names of all team members on the front page. G. It may occasionally be possible to turn in a paper electronically. In that case, turn in a single pdf file

of the paper, conforming electronically to all of the above rules.

2 Your Responsibilities

It is your responsibility to contact your other project team members and be availabel to meet with them. Students may try to avoid working (while still claiming full credit) by citing conflicts such as extra-curricular activities, work in other classes, job searches, etc. I want to give full credit only to those who contribute as full project partners, and not to those who use such excuses as a way to avoid taking on their burden in a project, but I will need your assistance.

3 Potential Topic Assignment (Maximum of three (3) pages)

Turn in a draft on Tue Jan 26. I will return with comments on Thu Jan 31. Turn in revised draft Tue Feb 2.

Milestone  Due Date

Phase I

1. Propose Two Topics  Tue Jan 26
2. Target Population, Frame, Mode of Data Collection Tue Feb 2

3.1 Potential Topics

Part of this assignment is to give you a chance to see if you can work together. Identify two survey projects that your team could actually implement over the course of the semester.

(1) An on-campus project. Examples might include:

• A survey of student attitudes toward digital rights (copyright, royalties, intellectual property, open-source and file sharing).

• A survey of academic departments to learn how they use Faculty Course Evaluations and other data to evaluate faculty teaching.

• A survey of students in HSS to see how they use, and how they like, academic advising (could become involved in official HSS effort to do this).

• A survey of staff to find out whether they feel they are adequately compensated (wages and benefits) at Carnegie Mellon.

(2) An off-campus project. Examples might include:

• A survey of Carnegie Mellon neighbors to learn about attitudes towards the University, or the golf course, or “???”…

• A telephone survey of residents of Pittsburgh to learn about their attitudes towards the Pittsburgh Public School system.

• A mail survey of subscribers to a community newspaper.

For each of projects (1) and (2), answer the follwing questions:

a. What question do you propose to study? Give me a brief answer that would be understandable to my

mother (who is not a statistician!). b. What population or populations will be sampled? This is the question many of you will miss. c. What population(s) do you wish to make inferences about? How does it (do they) differ from the

population in (b.)? d. How do you plan to carry out the survey (e.g. by telephone), and why? e. What variables do you propose to measure? f. For project (2) only, I want to see a sample (one page) questionnnaire.

You must submit this assignment as a team, and it will be graded as such. Please label your report for each project according to the individual copmonents: 2 for project (1) and 6 for project (2).

3.2 Team Working Agreement

(To be turned in with the final written project proposal below.)

At later stages of this project, I will be asking you as a group to evaluate the effort/time put in by other project team members. Your grade will depend on whether the other project members basically agree that you were a full partner.

For this team work agreement assignment, I want you to think ahead to what each of you like to do. Examples of activities might include preparing a draft questionnaire, pretesting the questionnnaire, conducting interviews, handling mail-out/mail-back, doing data analysis on the computer, writing up the results preparing a Power Point presentation, etc.

4 Written Project Proposal

Milestone  Due Date

Phase I

3. Choose Topic & Write Final Proposal, Begin IRB Tue Feb 9

Sign & Turn in Team Working Agreement

4.1 Purpose of this Assignment

The purpose of this assignment is to convince me to approve your project and approve your team as competant to do it. You can achieve both goals with a clear, complete description of all aspects of the project.

4.2 Proposal Table of Contents

Cover Page: Signed original work statement (from the box below). (Put this page on the very front of your proposal, with no blank pages on top of it.)

Pages 1–3: (A Maximum of 3 pages.) Provide information on the question(s) to be studied, the population(s) involved, and the statistical issues to be explored. A good place to begin is by answering questions A. through E. of the “Designing A Sample Survey” handout. [Make sure you first state and then answer the question so it is clear what questions you are answering.] If you are doing the project for a “client” tell me who he or she is and provide details on your assignments.

Pages 4 and on: 1. Preliminary questionnaire 2. Verbatim script for interview (if appropriate).

3. Sampling process.

4. Data collection and analysis plan.

5. Proposed project schedule including pilot testing of questionnaire.

Evaulation: Each of you must include a sealed envelope evaluating the effort of your fellow team members. I am particularly concerned to know whether someone has not been available, and if so their grade will be adjusted.

4.3 Human Subjects Clearance (IRB)

All survey research is subject to pre-approval by the University’s Institutional Review Board (IRB) . There are special forms associated with this process and they must be submitted as part of this assignment; this is a legal requirement on the university and all of its members, including faculty and students. Usually it is helpful to include the draft questionnaire as part of the submission.

Please submit completed IRB forms directly to me, not to the Carnegie Mellon IRB. Because your project is being conducted for course credit in 36-303, I—as your instructor—will act on behalf of the IRB.

Given the seriousness of this approval, I urge groups to prepare an early draft and share it with me for comments before the actual proposal is due.

4.4 Original Work

The following items may be discussed with other students, with possible clients, and with the instructor or TA:

• The selection of a project topic and various questions encountered while doing the project.

The following items must be done only by team members:

• The final choice of a project topic (with instructor approval); resolution of difficult questions; interpretations; drafting and revising survey questions and intervew protocol; design, implementation, data collection, and data analysis for the survey; and composition of project documents.

Whenever you draw upon the written work of others, that material should be properly referenced in your reports. For examples of acceptable citation styles, see http://library.osu.edu/sites/guides/apagd.php. The “reference” form should appear at the end of the paper in a list of references, alphabetized by author. The “in-text” form is used in the body of the paper, each time you call attention to the source from which you drew material.

SIGN THE FOLLOWING STATEMENT AND INCLUDE IT ON THE COVER PAGE OF YOUR PROPOSAL: (a second copy is attached to this handout for you to tear off and use)

ORIGINAL WORK STATEMENT

Project Title:

We the undersigned certify that the actual composition of this proposal was done by us and is original work.

Signatures  Typed Names  Phones  Emails

Contact Author:

5 Some Notes about Grades for Projects

Milestone  Due Date

Phase II

4. Sampling Scheme & Question Design  Tue Feb 16
5. Pretest & Revise Questions  Tue Feb 23
6. Final IRB & Project Plan  Tue Mar 2

Phase III

7. Progress Report on Data Collection & Analysis Tue Mar 30
(Meeting of the Minds Registration  Thu Apr 1)
8. Rough Draft of Final Report  Tue Apr 6
9. Final Report Due  Fri Apr 30
Oral Presentations  Apr 20–Apr 29
(Meeting of the Minds  Wed May 5)

The easiest projects typically will involve an on-campus survey of some of your fellow students, especially a subset who are easily accessible to you. The easiest mode of administration is by e-mail and/or a web service like surveymonkey.com, but followup and high response rates are often easier to achieve with other modes of data collection (especially interviews and other face-to-face methods), though they require more time and effort.

In my evaluation of your project work (both proposals and completed projects), there will be a tradeoff between originality and cleverness on the one hand, and thoroughness and difficulty on the other. If you are willing to do a more complex and difficult project this may make up for lack of originality. For example, I will specifically evaluate all proposals for thoroughness, including plans for non-response followup. On the other hand, I definitely do give credit for originality. Thus a project that is really dull and boring but does hard statistics cannot be expected to get higher marks than a project that is easier statistically but involves a clever problem or one of intrinsic scientific, social or policy interest.

5.1 Inferring Causal and Lurking Variables

Sample surveys are good at measuring things as they are, and only can be used to measure causal effects in very special and limited circumstances. Many questions of interest that are expored using sample surveys nonetheless are related to causal questions. In almost any statistical problem of comparing two or more subpopulations, or where we are studying the “effects” of some variables on others, there will be Lurking Variables, i.e. variables that—if not measured and accounted for in your analysis—will cause a distortion of the results, especially when you are looking at relationships. I will mention a few of these in class. I expect that your project will consider and measure lurking variables that are especially relevant for your question(s) of interest.

5.2 Thoroughness and Creatitivy

As suggested above, I will give a subjective evaluation of your proposal, based on two factors: First, is the idea really clever or really of scientific interest, and second, have you done a reasonably thorough job of things? I am hoping that this will induce you to think of a project that is both interseting and leaves scope for some statistical analysis.

For example, if you simply state that you plan to compare GPAs of male and female students, you are being neither creative nor thorough. It is easy to see that there are lurking variables (college and major), and there are also possible regression relationships with other variables of interest (e.g. SAT scores).

6 Other Notes Relating to the Project

• I suggest that you spend approximately 80 hours as a team on this project. You may spend more time if you are convinced that the benefits of spending more hours on this project are greater than the costs.

• Keep a diary (e.g. in the form of a lab book, or an on-line document), and update it regularly througout the project (no less than once a week!). Record your expenditures of time and other resources, as well as any notable events that may influence the interpretation of your data or data analyses. Keep track of which team members attend each meeting. Turn this diary in with your final report. (One text or pdf file per person is fine. If you handwrite your diary, it must be neat and easy to follow.) Credit will be awarded for this effort.

• You will invest a certain amount of time, energy, money and other resources, in doing this project. As is the case with all investments, the amount of return you will receive on this project, both now and later in your career, depends on the amount you invest in it and the risk you take.

7 Oral Presentation of Projects

Milestone  Due Date

Phase III

9. Final Report Due  Fri Apr 30
Oral Presentations  Apr 20–Apr 29
(Meeting of the Minds Wed May 5)

Scheduled for April 20, 22, 27 and 29. Each group will have 20 minutes for its presentation. In the past, students have found that making the presentation using PowerPoint is quite effective and

allows the group to cover more material. In additon, you may wish to prepare a handout for the class with questionnaire materials and some of your analytic results.

8 Written Final Report

Milestone  Due Date

Phase III

9. Final Report Due  Fri Apr 30
Oral Presentations  Apr 20–Apr 29
(Meeting of the Minds Wed May 5)

Due by 5:00pm on the final day of classes, Fri Apr 30. Further instructions and a proposed grading scheme will be handed out later in the course.

ORIGINAL WORK STATEMENT

Project Title: We the undersigned certify that the actual composition of this proposal was done by us and is original work.

Signatures Typed Names Phones Emails

Contact Author: