Data collection methods

By Greg Griffin,2014-08-30 14:33
11 views 0
Data collection methods


    Data collection methods

    Handout for the Qualitative Research Module Prepared by Anna Voce, March 2005. rdDrawn from: Patton (2002) Qualitative Research and Evaluation Methods. 3 Edition.

    Sage Publications. USA.

1. Introduction

    The primary methods of data collection in qualitative research are:

    ; Observation (including document review)

    ; Interview

    ; Focus Group Discussion

Data collection method is distinguished from data collection technique. Data collection

     refers to the systematic approach to data collection. refers to the art methodTechnique

    of asking, listening, and interpreting.

2. Using multiple sources of information triangulation

    Usually multiple sources of information are used, because no single source of information can be trusted to provide a comprehensive perspective on the issue being studied. By using a combination of observation, interviews and document review, the researcher is able to use different data sources to validate and cross-check findings. Each type and source of data has strength and weaknesses. Using a combination of data types

    triangulation increases validity as the strengths of one approach can compensate for the weaknesses of another approach.

    The limitations of observations include: the possibility that the observer may affect the situation being observed in unknown ways; participants may behave atypically if they know they are being observed; the selective perception of the observer may distort the data; observations focus only on external behaviour; observational data are often constrained by the limited sample of activities observed. Observers need other data sources to find out the extent to which observed activities are typical or atypical.

    The limitations of interview data include: possibly distorted answers as a result of personal bias, emotional status, recall error, reactivity of the interviewee to the interviewer, and self-serving responses. Observations provide a check for what is reported in interviews. Interviews on the other hand, permit the observer to go beyond external behaviour to explore thoughts and feelings.

    Documents and records also have limitations. They may be incomplete or inaccurate. Document analysis however, provides a behind-the-scenes look, at the programme that may not be directly observable and about which the interviewer might not ask appropriate questions without the leads provided through documents.

    By using a variety of sources, the researcher can build on the strengths of each type of data collection while minimising the weaknesses of the single approach.


    3. Observation

    3.1. Introduction

    Scientific inquiry using observational methods requires disciplined training and rigorous preparation … Training to become a skilled observer includes:

    ; Learning to pay attention, seeing what there is to see, and hearing what there is to


    ; Practice in writing descriptively

    ; Acquiring discipline in recording field notes

    ; Knowing how to separate detail from trivia to achieve the former without being

    overwhelmed by the latter

    ; Using rigorous methods to validate and triangulate observations

    ; Reporting the strengths and limitations of one‟s own perspective, which requires

    both self-knowledge and disclosure

    … Preparation (for observation) has mental, physical, intellectual, and psychological dimensions … Evaluators and researchers (must) move their observations from the level of ordinary looking to the rigour of systematic seeing.

The purpose of direct observation is to:

    ; Describe the setting, in order to understand and capture the context ; Provide a first-hand account of the setting (as opposed to having it described


    ; Capture things that would normally be taken for granted by someone who is

    routinely exposed to the setting

    ; Observe what people may be unwilling to talk about in the interview ; Confirm the perceptions reported by interviewees

    ; Provide the researcher with first-hand knowledge of the setting during the analysis

    and interpretation stage. The impressions and the feelings of the observer become

    part of the data; the observer takes in information and forms impressions that go

    beyond what can be fully recorded in even the most detailed field notes.

    The descriptions should be factual, accurate, and thorough, without being cluttered by minute detail and trivia. The descriptions must enable the reader to enter into and understand the situation described.

3.2. Observational methods

    There are six dimensions that can describe the variations in observational methods. (Please see Patton (2002) Exhibit 6.1. Dimensions showing fieldwork variations. From rdPatton (2002) Qualitative Research and Evaluation Methods. 3 Edition. Sage

    Publications. USA.)

3.2.1. Variations in observer involvement the first and most fundamental

    distinction that differentiates observational strategies concerns the extent to

    which the observer will be a participant in the setting being studied, from full

    participant in the setting, to complete onlooker. The extent of participation can

    change over time, as the study progresses. E.g. A research can start the study

    as onlooker and then become a full participant, or vice versa. When conducting

    full participant observations, this will most likely be implemented concurrently

    with other data collection methods (i.e. document analysis, interviews, direct


    participation and observation, and introspection). As an onlooker, the process of

    observation can be separated from interviewing.

3.2.2. Variations in perspective i.e. emic vs. etic perspectives A participant

    observer shares as intimately as possible in the life and activities of the setting

    under study in order to develop an insider‟s view of what is happening (emic

    perspective), using the language and categories for classification as identified by

    people in the setting. This means that the participant observer not only sees

    what is happening but feels what it is like to be part of the setting or programme.

    When a researcher must stand far enough away from, or outside of, a particular

    setting in order to describe it and how it may differ from other settings, an etic

    (outsider) perspective is used to describe what has been observed.

3.2.3. Solo vs. Team observations, participatory vs. collaborative approaches

    Degrees of collaboration in observational methods vary along a continuum. At

    one end is the solo observer of a team of observers what characterises this end

    of the continuum is that researchers completely control the inquiry. At the other

    end are collaborations with people in the setting being observed, in a co-

    researcher role, helping to design the research, collecting data, and analyse the

    data. Along the middle of the continuum are various degrees of partial and

    periodic (as opposed to continuous) collaboration.

3.2.4. Overt vs. covert observations A traditional concern about the validity and

    reliability of observational data has been the effects of the observer on what is

    observed. People may behave quite differently when they are being observed

    versus how they behave naturally when they don‟t think they are being observed.

    Thus, the argument goes, covert observations are more likely to capture what is

    really happening than are overt observations where people in the setting are

    aware that they are being studied. Researchers have debated the ethics and

    morality of conducting covert observations/research, without the explicit and fully

    informed permission of the person to be observed.

3.2.5. Variations in the duration of observations the duration of observations will

    depend on the time and resources available, and on the information needs and

    what decisions will result from the information. On one end of the continuum are

    short-term studies that involve observations of a single segment of a programme,

    sometimes for only one hour or two. On the other end of the continuum,

    observations are conducted over months or years. Observations should last long

    enough to answer the research questions being asked and to fulfil the purpose of

    the study.

3.2.6. Variations in the observational focus The scope can be broad,

    encompassing virtually all aspects of the setting, or it can be narrow, involving

    only a small part of what is happening. The focus of the observation is provided

    by the study design and the nature of the questions being asked.

3.3. What to observe - Sensitising concepts

    Part of the value of observational studies is the opportunity to look where no-one lese has looked before and „to see what there is to see‟. However, it is not possible to observe everything. The observation must be focused. The focus is provided by the study design and the nature of the research questions being asked. In the filed, the


    observer must organise the observations so that observing becomes and remains manageable. Observers often use „sensitising concepts‟ to guide their observations.

A „sensitising concept‟ is a starting point, an initial guide to the research, and may include

    loosely operationalised concepts such as: victim; stress; stigma; group process; leadership; power. The six questions of „what, why, when, how, where, and who‟ constitute a fundamental „sensitising framework‟ based on the central elements of good


    The notion of „sensitising concepts‟ reminds us that observers do not enter the field with a completely blank slate. Highly experienced observers have internalised a „sensitising framework‟, to the point where they would not need to list these concepts in a formal

    written design. Less experienced researchers, will usually benefit from preparing a formal checklist of major „sensitising concepts‟ in the formal design and then using these concepts to help to organise the observations, and guide the fieldwork, at least initially.

3.4. What to observe sources of data

    The following may be included in a „sensitising framework‟, depending on the research questions:

    3.4.1. The physical setting: describe the physical environment, in sufficient detail to

    permit the reader to visualise the setting (e.g. in programme evaluation). The

    physical environment of a setting can be important to what happens in that

    environment. A common mistake among observers is to take the physical

    environment for granted. E.g. a researcher may report that a programme took

    place in a school. The researcher may have a mental image of a school that

    matches what was observed, but schools vary considerably in size, appearance,

    and neighbourhood setting. Even more so, the interiors of schools vary

    considerably. The same may be said of criminal justice settings, health settings,

    and any other human service activity. Nota Bene: use descriptive adjectives

    rather that interpretive adjectives. Vivid descriptive adjectives provide sufficient

    information that the reader does not have to speculate at what is meant. For

    example: simply reporting „a crowded room‟ requires interpretation. Contrast

    with this: “the meeting room had a three-person couch across one side, six

    chairs along the adjoining walls next to the couch, which included the door. With

    20 people in the room, all standing, there was very little space between people.

    Several participants were overheard to say, “this room is really overcrowded”.

    Such descriptive writing requires attention to detail and discipline to avoid vague,

    interpretive phrases.

3.4.2. The human, social environment: In describing the social environment, the

    observer looks for ways in which people organise themselves into groups and

    sub-groups. Patterns and frequency of interactions, the direction of

    communication patterns, and changes in these patterns tell us about the social

    environment. How people group together can be illuminative and important.

    Decision making patterns can be important. An observer‟s description of a social

    environment may not be the same as the perceptions of that environment

    expressed by the participants. Nor is it likely that all participants will perceive the

    social climate in the same way. At all times it is important that the observer

    record participants‟ comments in quotation marks, indicating the source – who

    said what? so at to keep perceptions of participants separate from the

    observer‟s own descriptions and evaluations.


    3.4.3. Historical information: Historical information is an important part of describing the context in which the research is taking place. The kind of questions that will be asked include: How was the programme created and funded? Who were the original people targeted for programme services, and how have the target populations changed over time? What have been staffing patterns over time? To what extent have goals and intended outcomes changed over time? If the programme is embedded in a larger organisational context, what is the history of that organisation in relation to the programme? How has the larger political and economic environment changed over time, and how have these changes affected programme development?

    3.4.4. Planned programme activities: Build observations around activities that have

    a kind of structure to them a beginning, a middle point and a closure point e.g. a class session, a counselling session, a meal time, a meeting, a home visit, etc. The following descriptive questions guide the researcher through the full sequence of observation: Who is involved? What is being said and done by all participants (e.g. staff and clients)? How do they go about what they do? Where do the activities occur? When do things happen? What are the variations in how participants engage in planned activities? How do behaviours and feelings change of the course of the activity? How is the activity ended? How do participants react to the ending of the activity? Each unit of activity is observed and treated as a self-contained even for the purpose of managing the field notes. During analysis one looks across these discrete units of activity to find patterns and themes.

    3.4.5. Informal interactions and unplanned activities: During periods of informal

    interactions and unplanned activities it may be difficult to organise observations. At these times the researcher needs to remain open to opportunity sampling. The observer watches, listens and looks for opportunities to deepen observations, recording what people do, the nature of informal interactions (e.g. what sub-groups are evident) and what people are saying to each other. The latter is particularly important because during informal interactions and unplanned activities, people have the greatest opportunity to exchange views and talk with each other about what they are really feeling. The researcher may have an opportunity to converse with participants, employing informal, conversational, interviewing. How something is said should be recorded along with what is said. It is these spontaneous interactions that often provide the most significant learning and insight. Thus it may be seen that observation is often combined with informal interviewing.

    3.4.6. “Native” language of the programme: The field notes and reports should

    include the exact language used by participants, as this provides an indication of the meaning that participants attach to something and illuminates their experiences. Observers must learn the language of participants in the setting they are observing in order to faithfully represent participants on their own terms and be true to their worldview.

    3.4.7. Nonverbal communication: While recording the language of participants, the

    observer should also attend to nonverbal forms of communication. But because nonverbal behaviours are often misinterpreted, especially cross-culturally, whenever possible and appropriate, having observed what appear to be significant non-verbal behaviours, some effort should be made to follow-up with


    those involved to find out directly from them what the non-verbal behaviours really meant.

    3.4.8. Unobtrusive observations: Being observed can make people self-conscious

    and generate anxiety, and regardless of how sensitively observations are made, the possibility always exists that people will behave differently under conditions where an observation is taking place than they would do if the observer were not present. Rather than resort to „covert‟ observations (which are associated with ethical concerns) choose „unobtrusive measures‟. Unobtrusive measures are those made without the knowledge of the people being observed and without affecting what is observed. For example: In an evaluation of the National Museum of Natural History at the Smithsonian Institute (Wolf and Tymitz 1978), the researchers looked for „wear spots‟ as indicators of use of particular exhibits.

    Dusty equipment and files may indicate things that are not used. Worn books to indicate that they have been used.

    3.4.9. Documents: Records, documents, artefacts and archives constitute a

    particularly rich source of information about many organisations and programmes. In contemporary society, all kinds of entities have a trail of paper, a kind of spoor that can be mined as part of fieldwork. E.g. Families keep photographs, letters, sentimental objects; people who commit suicide leave behind suicide notes; gangs inscribe public places with graffiti; organisations produce records; service providers keep client files. Indeed, and often intriguing form of analysis involves comparing official statements found in public documents (annual reports, policy statements) with private memos and what the observer actually hears or sees. At the very beginning of the study, access to important documents and records should be negotiated. The ideal situation would be to have access to all routine records, correspondence, financial and budget records, organisational rules, regulations, memoranda, and any other official and unofficial documents generated by or for the programme. Document review can be valuable in guiding the researcher in what needs to be pursued further in direct observation and interviewing. Confidentiality must be respected, as with all information to which the researcher has access. The extent to which actual references to, and quotations from, records and documents are included in the final reports depends on whether the documents are considered part of a public record and therefore ale to be publicised without breach of confidentiality. In some cases, with permission and proper safeguards to protect confidentiality, some information from private documents can be quoted directly and cited.

    3.4.10. Observing what does not happen: If programme goals,

    implementation design, and/or proposals suggest that certain things ought to happen or are expected to happen, and they don‟t, then it is appropriate for the

    observer to note these things that do not happen. E.g. if a community in which water is scarce shows no evidence of conflict over water rights, a researcher could be expected to report on the absence of community conflict. If a school programme is intended to provide children with opportunities to explore the community, and no such explorations occur, it is appropriate for the observer to note this. If clinic supervisors are supposed to conduct monthly supervisory visits to the clinics, and these do not happen, then it is entirely appropriate for the observer to record this.

    3.4.11. Observing oneself: The principle of reflexivity reminds the qualitative researcher to observe oneself, to be attentive to and conscious of the cultural, political, linguistic, ideological origins of his/her own perspective and voice as well


    as, and often in contrast to the perspectives and voices of those s/he is

    observing. Reflexivity calls for critical self-reflection and self-knowledge, and a

    willingness to consider how one is affected by, and how one influences, what is

    observed. The observer must ultimately deal with issues of authenticity, reactivity,

    and how the observational process may have affected what was observed as well

    as how the background and predisposition of the observer may have constrained

    what was observed and understood.

    4. Interviewing

    We interview people to find out from them those things we cannot directly observe. We cannot observe feelings, thoughts, and intentions. We cannot observe behaviours that took place at some previous point in time. We cannot observe how people have organised the world and the meanings they attach to what goes on in the world. The purpose on interviewing is to allow us to enter into the other person‟s perspective.

    Qualitative interviewing begins with the assumption that the perspective of the other is meaningful, knowable, and able to be made explicit. We interview to find out what is in and on someone else‟s mind, to gather their stories.

    The quality of the information obtained during an interview is largely dependant on the interviewer. It is important to develop disciplined an rigorous interview techniques. It is also important to have a deep and genuine interest in what people have to say about their world.

4.1. Types on interviews

    There are three basic approaches to collecting qualitative data through open-ended interviews. Each approach has strengths and weaknesses, and each serves a somewhat different purpose. Each approach involves a different type of preparation, conceptualisation and instrumentation. The three approaches differ in the extent to which interview questions are determined and standardised before the interview occurs.

4.1.1. The informal, conversational interview: relies entirely on the spontaneous

    generation of questions in the natural flow of an interaction, often as part of

    participant observer fieldwork. It is also called „unstructured interviewing‟. The

    conversational interview offers maximum flexibility to pursue information in

    whatever direction appears to be most appropriate, depending on what emerges

    from observing a particular setting or from talking with one or more individuals in

    that setting. Most of the questions will flow from the immediate context. Data

    gathered from informal conversational interview will be different for each person

    interviewed. The same person may be interviewed on several occasions, with

    questions specific to the interaction or event at hand. Previous responses can be

    revisited and deepened. This approach works particularly well where the

    researcher can stay in the setting for some period of time so as not to be

    dependant on a single interview opportunity. Being unstructured does not mean

    that conversational interviews are unfocused. Sensitising concepts and the overall

    purpose of the interview will guide the questions that are asked. The strength of

    the informal conversational method is that it offers flexibility, spontaneity, and

    responsiveness to individual differences and situational changes. Questions can

    be personalised to deepen communication with the person being interviewed. A

    weakness in the informal conversational interview is that it may require a greater

    amount of time to collect systematic information because it may take several

    conversations with different people before a similar set of questions has been

    posed to each participant in the setting. Because the approach depends on the


    conversational skills of the interviewer to a greater extent than do formal standardised formats, this go-with-the-flow style of interviewing may be susceptible to interviewer effects, leading questions, and biases, especially with novice researchers. Data obtained from informal conversational interviews may be difficult to pull together and analyse. Because different questions will generate different responses, the researcher has to spend a great deal of time sifting through response to find patterns that have emerged at different points in different interviews with different people. By contrast, interviews that are more systematised and standardised facilitate analysis but provide less flexibility and are less sensitive to individual and situational differences.

    4.1.2. The general interview guide approach: involves outlining a set of issues that

    are to be explored with each respondent before the interview starts. The guide serves as a basic checklist during the interview to make sure that all relevant topics are covered. The same basic line of questioning is pursued with each respondent. The interview guide provides topics within which the interviewer is free to explore, probe, and ask questions that will elucidate and illuminate that particular subject. The interviewer remains free to build a conversation within a particular subject area, to word questions spontaneously, and to establish a conversational style but with the focus on a particular subject that has been predetermined. The advantage of an interview guide is that the interviewer has carefully decided how to use the limited time available in an interview situation. The guide helps make interviewing a number of different people systematic and comprehensive, by delimiting in advance the issues to be explored. A guide is essential in conducting focus group discussions. Interview guides can be developed in more or less detail, depending on the extent to which the interviewer is able to specify important issues in advance and the extent to which it is important to ask questions in the same order to all respondents.

    4.1.3. The standardised open-ended interview: consists of a set of questions

    carefully worded and arranged with the intention of taking each respondent through the same sequence and asking each respondent the same questions with essentially the same words. The standardised open-ended interview is used when it is important to minimise variation in the questions posed to interviewees. In a fully structured interview instrument, the question would be completely specified, as would the probes (designed to get deeper information) as well as the transition questions (designed to introduce the next topic). An Ethics Committee may request this kind of detail, especially if the topic of the research is controversial or intrusive. In team research, standardised interviews ensure consistency across interviewers. In multi-site studies, structured interviews provide comparability across sites. Structured questions may also compensate for variability in researcher skills. Training in the interview approach will minimise variability due to differences in experience and skill. Structured open-ended interviews may also be used to collect data before, during and after a programme of intervention. This allows for answers to be compared across time periods.

    4.1.4. Combining approaches: these interview strategies are not mutually exclusive. A combined strategy offers the interviewer flexibility in probing and in determining when it is appropriate to explore certain subjects in greater depth, or even to pose questions about new areas of inquiry that were not originally anticipated in the interview instrument‟s development. A common combination strategy involves using standardised interview formats at the beginning of the interview and then leaving the interviewer free to pursue any subjects of interest during the latter parts of the interview. Another combination would include using the


    informal conversational interview early in an evaluation report, followed midway

    through by an interview guide, and then closing with a standardised open-ended

    interview to get systematic information from a sample of participants when

    concluding the study.

4.2. Types of questions

    Six kinds of questions can be asked of people. On any given topic it is possible to ask any of these questions. Distinguishing types of questions forces the interviewer to be clear about what is being asked and helps the interviewee respond appropriately.

4.2.1. Experience and behaviour questions: Questions about what a person does,

    has done, has experienced, and what activities would have been observable if the

    researcher were present. “If I followed you through a typical day, what would I

    see you doing?”

4.2.2. Opinion and value questions: Answers to these questions tell us about what

    people about a certain action or experience, as opposed to their actions and think

    behaviours. These questions help us to understand people‟s goals, intentions,

    desires, and expectations. “What do you believe?” What do you think about …?”

    What would you like to see happen?” What is your opinion of …?”

4.2.3. Feeling questions: feeling questions aim at eliciting emotions feeling

    responses to of people to their experiences and thoughts. “How do you feel

    about that?” Opinions and feelings are often confused. It is critical the

    interviewer understand the difference between the two in order to know that they

    have the kind of answer they want for the question they are asking. Sometimes

    an interviewer will ask “How do you feel about that?” and the response will be “I

    think it is the best we can do under the circumstances.” The question about

    feelings has not really been answered. The confusion sometimes occurs because

    the interviewer gives the wrong cues and does not asking the question correctly.

    When asking feeling questions, the interviewer has to ask and listen for feeling

    level responses.

4.2.4. Knowledge questions: knowledge questions inquire about the respondent‟s

    factual knowledge.

    4.2.5. Sensory questions: sensory questions ask about what is seen, heard, touched,

    tasted, and smelled. Sensory questions attempt to have the interviewees

    describe the stimuli they experience.

4.2.6. Background/demographic questions: age, education, occupation are

    standard background questions. Answers to these questions help the interviewer

    locate the respondent in relation to other people. Qualitative inquiry is

    particularly appropriate in finding out how people perceive and talk about their

    backgrounds. E.g. Tiger Woods has African, Thai, Chinese, American Indian and

    European ancestry and calls himself a “Cablinasian”.

4.3. Asking questions

4.3.1. The time frame of questions: can be in the present, past or future tense.


    4.3.2. Sequencing questions: begin an interview with non-controversial present

    behaviours, activities and experiences. Such questions ask for relatively

    straightforward descriptions. They require minimal recall and interpretation, and

    are generally easy to answer. Then opinions and feelings may be solicited,

    building on and probing for interpretations of the experience. Opinions and

    feelings are likely to be more grounded and meaningful once an experience has

    been relived. Knowledge and skill questions can be quite threatening if asked too

    abruptly. They are best asked when rapport and trust have been established.

    Questions about the present are easier than questions about the past. Then ask

    about the future. Background and demographic questions, depending how

    personal they are, may make the respondent feel uncomfortable. If asked at the

    beginning of the interview they may condition the respondent to give short

    answers. It may be best to save these questions until the end.

4.3.3. Wording questions: How a question is worded affects how the interviewee

    responds. Asking questions is an art. They should be genuinely open-ended,

    neutral, singular and clear. An genuinely open-ended questions minimises the

    possibility of imposing predetermined responses. “How do you feel about … ?” is

    open-ended whereas “How satisfied are you with …?” already presupposes an

    answer about satisfaction. Avoid questions that will lead to Yes/No answers. E.g.

    “Are you satisfied with …?” Neutral questions refer to the non-judgementalism

    adopted by the interviewer. Singular questions ensure that not more than one

    idea is contained in any given question. “How well do you know and like the staff

    in this clinic?” contains two ideas. The clarity of questions is enhanced by asking

    simple, understandable, unambiguous questions, using language and terminology

    that is familiar to the respondent.

    4.3.4. Why questions: Take care about asking why questions. They imply causal

    relationships, which may be complex to unravel, an may make respondents feel

    defensive. Think of other ways of asking what it is you want to know.

4.3.5. The final or closing question: provide th interviewee with the opportunity to

    have the final say. E.g. “That covers all I wanted to ask. Anything you would

    like to add?”

5. Focus group discussion

    Focus group is a complementary technique to individual interviews. It is a discussion on a specific topic, with a small group of people (6-10) with similar backgrounds who participate in the discussion for 1 2 hours. In a given study, a series of different focus groups will be conducted to get a variety of perspectives and increase confidence in whatever patterns emerge. In a focus group, participants get to hear each other‟s responses and make additional comments beyond their original responses as they hear what other people have to say. Participants do not need to agree with each other, and no consensus needs to be reached. Nor is it necessary for people to disagree. The objective is to get high quality data in a social context where people can consider their own views in the context of the views of others. The interviewer may rather be called the moderator highlighting the specific function of the interviewer, that of moderating or guiding the discussion. The focus group is not a collection of simultaneous individual interviews, but rather a discussion where the conversation flows because of the function of the moderator.

Report this document

For any questions or suggestions please email