Correlational research

Correlational research refers to any non-experimental study in which correlations (associations) in data are examined. These types of studies don’t test for causality directly. Instead, we can only show causality indirectly.

The most common example of correlational research is the survey. We know people’s answers to different questions might be associated with each other. However, we can’t tell if answers to one question are causing changes in answers to other questions.

The main alternative to correlational research is experimental research. In true experimental research we can manipulate the cause, and because of this, we can establish causality.

Observational research

Observational research refers to research where the independent variables cannot be directly manipulated by the researcher, and so we cannot rule out that the independent variable was not applied with true randomness to the cases. There is always the risk of unobserved bias in observational research.

Examples of observational research include surveys, censuses, most of astronomy (studies of stars, the planets, the Sun), most of geology and biological studies of things like species and evolution.

An example of non-observational research is a randomized controlled experiment. For example a drug trial, where half the people are given placebos. A large amount of psychology and chemistry are non-observational (true experiments).

Questionnaire

Questionnaire is a fixed collection of questions used in a social survey that respondents answer. A questionnaire is basically another word for a survey.

Interview schedule

Interview schedule is a questionnaire specifically designed for an interviewer asking respondents the questions.

An interview schedule is basically a survey or questionnaire, but it is written so that the interviewer can read it out to the participants.

It often includes specific instructions to the interviewer which are not read out to the participant.

For example:

• “If Yes to Q32, please skip questions 33 to 44 and go to Q45.” Or
• “If the respondent seems unsure, prompt them with ‘You look concerned, if you would prefer, we can skip this question.’”

Screening questions

Screening questions. These are the questions at the beginning of a survey, generally before the informed consent, which ask questions which establish if the person is eligible to do the study.

For example:

• Are you currently working?
• Are you over 18 years of age?
• Are you currently married?

Open-ended question

Open-ended question format refers to survey questions that allow respondents to give any answer. In a written survey, these will generally appear as a blank line, or a blank box for the respondent to write their answer into.

For example, “You said earlier that you had problems being paid on time. Can you tell me exactly what happened when you weren’t paid on time. Why do you think it happened?”

Closed-ended question

Closed-ended question format refers to survey questions in which respondents must choose among fixed answer choices. These can me yes/no questions, they can be multiple choice, or they can be a set of checkboxes where the respondent can pick multiple items.

For example, “What is your gender? [] Male [] Female [] Other”

Mutually exclusive answer options

Mutually exclusive answer options (in a survey) are options where the answers do not overlap. A person can logically choose one answer.

For example, if you asked people their age and had the options: [] Less than 18 years; [] 18-29 years; [] 30 years or older. These are mutually exclusive.

If you asked the same question, but had the following options: [] 17 years or younger; [] 17-29 years; [] 29 years or older. If someone was 17 or 29, they would fit into two categories.

Exhaustive

Exhaustive is when all units fit into some category of a variable.

An example of a non-exhaustive question would be to ask a work what industry they are part of and only list five specific industries: [] retail, [] construction, [] hospitality, [] shipping, and [] finance. This list could be made exhaustive by adding a sixth category " [] Other (please specify) ___________" which would also have an open-ended component.

Unidimensionality

Unidimensionality is when all items of an index or scale measure the same concept or have a common dimension. Generally, an index should have unidimensionality so that it focuses on capturing one underlying concept/variable.

We will learn more about this when we discuss scales in a later lecture.

Probe

Probe is a neutral request made by an interviewer to clarify an ambiguous answer, complete an answer, or obtain a relevant response.

For closed ended survey questions, this is often just clarifying an answer so you can put the answer in the correct category.

For open ended questions, or qualitative interviews, probes can include questions like:

• Can you give me an example?
• What exactly did they say?
• How do you know that?
• So can I just clarify, did x, or y happen first?

Social desirability bias

Social desirability bias is a tendency for survey respondents to answer in a way that conforms to social expectations or makes them look good rather than to answer honestly.

For example,

• “Have you cheated on your current partner?”
• “What is your annual income?”
• “Do you support Donald Trump for President?”

These are questions which all receive under-reporting because of social desirability bias.

Order effects

Order effects are changes to respondents answers that occur because of the order survey questions are asked.

For example, imagine we are asking about attitudes to social welfare, such as unemployment benefits.

People tend to give different answers if we first ask a question about a type of person generally seen as ‘deserving’.

So if we ask a question about a widowed wife with children, then because such a person is seen as deserviing, if we follow with a question about attitudes to welfare (“Do you support the government providing more welfare for the needy?”) then we will likely get a large number of people expressing support for welfare.

If, on the other hand, we ask about a young unemployed drug user (generally seen as undeserving), and then ask a question about general attitudes to social welfare, we will get a lower number of people expressing support for welfare.

Note that the same question is asked in the second question, but very different answers given because of the ordering effect.

Double-barreled question

A double-barreled question is a confusing survey question that includes two or more ideas that really should be separated.

For example, “Are you angry and want to get justice?” This would be better rephrased as two questions.