RESEARCH METHODS FOR
NEEDS ASSESSMENT AND MONITORING AND EVALUATION
In the development of your telecenter you will have to deal with several project
development stages: project design, implementation, and evaluation.
Where do you start? The first step is to answer the question “Where are we
now?” It involves the collection and analysis of information on the current situation in the
community where a telecenter is being considered or already exists. This step is also
known as a needs assessment.
The four steps of project design:
Step 1: Step 2: Step 3: Step 4:
Analyze the Develop a Create a Develop a
situation framework strategy M+E plan
The next step – developing a framework – answers the question “Where do we
want to go?” In this step you will develop a clear purpose and goals for your telecenter
and define what you hope to achieve.
Next, you will need to create a strategy for your telecenter answering the
questions “How do we get there?” This step will help to identify where in a community
the telecenter should be located; what kind of hardware and software are needed; what
kind of training should be offered; what kind of and how many staff to hire; what services
to offer; how much to charge for the services; how to get volunteers from the community
involved in managing the telecenter, etc.
Finally, you must develop a Monitoring and Evaluation system (M+E) that will be
able to answer the question “How will we know when we get there and how are we
doing?” This step will help to assess how well the telecenter is functioning and whether it
is meeting the needs of the community.
1.1. What is this module about?
In this module we will focus on the first and last steps in the project cycle
described above. Keep in mind that stages 2 and 3 are mostly covered in Module 5
(Strategies for Sustainability). First, we will answer the following questions regarding
needs assessments and M+E:
? What exactly are needs assessment and M+E systems?
? Why are they important?
? Why is participation so crucial in the processes? ? How do they fit into the larger context of project development and implementation?
We will then identify and describe some of the tools that are useful for conducting
needs assessments and M+E. Various resources (books and websites) for further
exploration of these topics are provided at the end of the module.
1.2. Participation, Needs Assessment, Monitoring and Evaluation
Of all the lessons learned over the past five decades of development practice
perhaps the most important one is the need for participation of the proposed
beneficiaries in any development effort. Beneficiary communities are in the best position
to know what their needs are, what their resources are, and what direction they want to
go. When these communities are involved in the various phases of a project – from
needs assessment, to project design, to project implementation, to M+E – that project is
far more likely to address their felt needs and they are far more likely to feel “ownership”
of the project. On the other hand, if people from outside of the community control the
entire process, the project is unlikely to address the genuine needs, or attract the
attention, of the beneficiaries. As a result it will lack success and sustainability.
As you consider establishing and maintaining a telecenter in your community,
consider this need for community involvement and ownership.
A needs assessment (or, “Where are we now?”) should always be the first step
when designing a project. Taking the time to involve representatives of all potential
telecenter users in deciding on its services will result in a telecenter that is used and
supported by the community. The community should be equal partners in the needs
assessment process and therefore assume equal responsibility for analyzing, planning,
providing information, interpreting findings, and carrying out action strategies. It is very
useful to form a steering committee that is made up of community representatives who
can help you to facilitate the needs assessment process. Careful planning will help
ensure that the right information is collected and in the most efficient way.
There are two main ways to approach a needs assessment – a strength-based
approach, and a problem solving approach. Both, of course, involve stakeholders in the
processes (those people who will be affected and who will influence the telecenter:
community members; community organizations; schools; government officials; telecenter
managers; telecenter volunteers; donors, etc.).
Strength-based approaches or “appreciative inquiry” emphasizes the positive attributes of the community and looks at what has worked well in the past. It motivates
people and focuses on repeating successes. Information gathering begins by
discovering what is working at the present time, what resources are already available in
the community, and what assets exist for bringing about positive change. By thinking of
and exploring what is good and successful, communities develop commitment,
confidence, and energy for new efforts.
The problem-solving approach begins with communities analyzing their conditions and identifying problems that they want to change. In addition to uncovering
and defining problems, the community attempts to diagnose the causes of these
problems and to explore potential solutions. The approach you use – whether strength-
based or problem-focused – will help determine the kinds of questions you ask when
The three most common ways of collecting information are document reviews,
interviews, and observations. Most practitioners recommend at least two or three
sources for identifying needs. This is known as “triangulation” and helps to increase the quality of your research. Documents may include government development plans,
sector-specific documents from the various government ministries, project or annual
reports from various non-government or donor organizations, and other articles or
reports. These documents can provide extensive information on various issues, needs,
or problems in a country or region.
Interviews, observation, and various other participatory tools help to narrow the
scope of issues that are important to the community. Simply spending time with
community members and observing can provide a wealth of relevant information. For
example, observing visitors at health clinics may suggest what groups do not get health
services. Interviews with visitors may also reveal the range of health problems in the
community, and for future reference, suggest information and communication resources
that might support preventive medicine. It is important to know in advance what you want
to learn and to keep records of what you observe. Note, also, that needs of communities
will likely change over time and therefore needs assessments should be an ongoing
The tools for conducting needs assessments will be described in detail in section
2 of this module.
Monitoring and Evaluation (M+E)
Monitoring is an ongoing process that answers the question “How are we doing?”
It tells us whether we are achieving the objectives of the project and can help us to
improve the design or implementation of the project. Evaluation is generally done at a
particular point in time – perhaps in the middle or at the end of a project – and can be
done with the assistance of an external evaluator. It generally answers in greater depth
the question, “What differences can we see as a result of our efforts?” It addresses the
overall “value” and long-term impact of the project. M+E are tools – they are a means to
an end and not ends in themselves.
The advantages of involving stakeholders in the various steps of the M+E
process (planning, data collection, analysis, and using the results) are:
? providing more relevant and better quality information;
?the results more likely to be used by stakeholders; ?greater ownership of the project by stakeholders; and
?participants develop M+E skills.
Not all stakeholders will necessarily be involved with the entire M+E process and
various stakeholders may be involved in different ways. It may be helpful to establish an
M+E team from among the stakeholders. Selection of team members may depend on
their interests, skills, and availability, and can include members of the telecenter steering
committee mentioned previously. The M+E team plans and implements M+E activities.
One of the most important ways for the community to be involved in M&E is determining appropriate indicators for measuring success. Indicators are markers that
show progress and help to measure change. They are tools that help to answer M+E
questions. There are both quantitative and qualitative indicators. Quantitative or numeric
indicators show “how much” or “how many”, while qualitative indicators show how or why
people think, feel, or behave in a particular way. For example, the users of the telecenter
may decide on indicators that look at changes over time in:
? the number of people using the telecenter;
? the willingness of the community to volunteer to run the telecenter;
?the willingness of the community to pay for services.
If your telecenter is being used to disseminate information about agriculture then indicators for success may involve observing changes in agriculture practice, production,
If one of the goals is to help increase literacy, then users of the telecenter may decide that an appropriate indicator is whether literacy among users is increasing or not.
These are, of course, just a few examples. Final selection should be based on an
indicator’s relevance to answering your M+E questions; its relevance to the activities of
the telecenter; and the amount of expertise and effort needed to collect data. Ideally,
final indicators should be relevant, specific, measurable, and observable.
A more detailed step-by-step description and the necessary tools for an M+E system will be provided in section 3 of this module.
2. NEEDS ASSESSMENT
The main purpose of a telecenter is to meet the needs of a community and, more directly, its information and communication needs. In order to serve a community, it is
crucial to know their exact needs. And none is better able to assess these needs than
the community itself. A needs assessment in the respective community must therefore
be an initial step in the development and design of a telecenter.
The more participatory the needs assessment is conducted, the better it will able to give a complete picture of the needs of the community.
The following are suggested steps for a telecenter needs assessment:
1. Identifying the users of the needs assessment: In the case of a telecenter, the
users are both those who will act on the assessment – such as the manager, the
volunteers, or a telecenter committee, as well as the users of the telecenter services.
2. Identifying uses: A needs assessment for a telecenter is most likely to be used for
planning of telecenter services, which, in turn, relate to decisions about staffing, facilities
and other aspects of the telecenter's operations.
3. Describing the context: What is the physical and social environment of the
telecenter? Has the telecenter been operating for a long time? Is this an initial
assessment or are you trying to verify the appropriateness of the telecenters services?
4. Identifying needs:
Descriptions of the circumstances/problems of the stakeholders. Suggestion of possible
solutions to their needs and analysis of the likely effectiveness, feasibility, and utilization
of these solutions.
5. Meeting needs:
Recommendations for actions based on the needs, problems, and solutions identified.
6. Communicating results and recommendations: Communicating the results to the
stakeholders is an important element of the assessment.
Several techniques are suitable for a participatory needs assessment. The
methods you use will depend on the particular situation within your community, your
resources and the time frame.
The most important methods that will be described in some detail here are:
? Focus Groups
Interviews are a widely used method for needs assessments, M + E, and in
formal evaluations. Typically interviews take place in a face-to-face situation with one
interviewer and one interviewee. There are several different types of interviews to be
Open-ended or informal interviews: These interviews are free flowing and are conducted in a very conversational and informal manner. Open-ended interviews are
useful for exploring ideas and hypotheses (assumptions). An example could be when an
investigator (e.g. you as the telecenter manager) wants to learn about what is important
to a particular community in the context of needs that might be provided by the
telecenter. In this form open-ended interviews could be called “listening surveys.”
You could start an informal, open-ended interview by asking questions such as: “Please tell us about the community and its recent problems, and how you feel about
them.” You would listen carefully to the responses and follow-up with questions
requesting clarifications, explanations or more details
Semi-structured interviews: These interviews are typically more structured than open-
ended interviews. They generally consist of a series of open-ended questions asked in a
pre-determined order. If an interviewee starts to cover a new area as a response to a
question, then the interviewer keeps the flow going by asking relevant questions on his
list of topics. Each question is followed with additional probes until the answer is
explored in some depth.
A set of topics for an interview is called the “Interview Guide.”
Example of an Interview Guide
1) Introduction by the interviewer
? Who you are and what you are doing and why
? Request to tape interview or to take notes
? Assurance on confidentiality and anonymity
? Purpose and length of interview
? Reason for choice of interviewee
2) Questions about the interviewee’s prior knowledge about the telecenter
3) Question about why or why not the interviewee is using/not using the services of the telecenter
4) Question about previous experiences with telecenters, computers etc.
5) Expectations/anticipations about the telecenter
6) Interests/activities that might affect the usage of the telecenter
7) Issues that might cause problems
? Telecenter access
8) Any other issues related to the telecenter and the interviewees involvement with it
9) Demographic data such as age factors in the community, education, family size
10) Willingness to be interviewed again?
Structured interviews: This type of interview is the opposite of the free-flowing,
informal interview. Structured interviews allow little room for additional questions or
probes. These interviews can be useful when the existing data base is already
substantial and what is needed is a quick quantification of narrowly defined topics. (We
will discuss this topic in more detail when we turn to surveys in section 2.3 below.)
However, for an initial needs assessment, focus groups and semi-structured interviews
2.2. Focus groups
A focus group is a type of semi-structured interview carried out in a group setting. The person running the group session - the facilitator - can be the telecenter manager, a
team member or an outside consultant. The main role of the facilitator is to "guide" the
discussion. A focus group usually consists of 8-12 participants and a facilitator. Typically,
the focus group runs 1-2 hours.
Benefits of conducting focus groups
? Additional insights are provided through the interaction of ideas and suggestions of the
? Focus groups can be used to gather information about people's beliefs, and to collect
kinds of detailed data that are difficult to obtain through structured surveys.
? Focus groups involve more people more quickly than individual interviews.
? Focus groups do not require much training for the personnel conducting them.
Focus groups can be conducted at any point in a planning process. However, focus groups are particularly important in the evaluation process: as part of a needs
assessment, during a program (monitoring), at the end of the program, or months after
the completion of a program to gather perceptions on the outcome of that program.
For managing a telecenter, the uses of focus groups include:
? Obtaining general background information about a topic of interest (for example,
the economic and social situation of a community, the community's information
and communication needs, and other concerns that might be relevant to
? Stimulating new ideas and creative concepts for telecenter services or the
solution of existing problems (for example, low level of use of telecenter services);
? Anticipating the potential for problems with a new telecenter program, service or
Steps in conducting focus group interviews
Follow this step-to-step guide to ensure reliable results:
Step 1: Select the research team
Conducting focus groups requires a small team with at least a facilitator to guide the discussion and a note taker to write down (or record) participants' important
comments. The facilitator should be a native speaker who can put the people at ease.
The team should have substantive knowledge of the topic under discussion.
Step 2: Select the participants
First, identify the groups and institutions that should be represented in the focus group (such as community members, NGOs, government officials, partners, etc.). The
selection of the participants will be determined by the objectives of the assessment.
Separate focus groups can be held for each type of group (women, men, elders, political
leaders, farm laborers, etc.) especially if the possibility of intimidation exists). Second,
identify the most suitable people in each group. It is advisable to have a variety of people
in the selection process to minimize the biases of individual preferences.
Each focus group should consist of 8-12 people to allow the smooth flow of conversation. Participants should be relatively homogenous; for example, they might be
from similar socio-economic and cultural backgrounds.
Step 3: Decide on timing and location
Discussions usually last one to two hours and should be conducted in a convenient location with some degree of privacy. Focus groups in a small village arouse
curiosity and can result in uninvited guests. Open places are not good spots for
Step 4: Prepare n discussion/interview guide
In order to structure the focus group discussion and to replicate discussions with different groups, it is necessary to prepare an interview guide.
Whenever possible, the interview guide should be developed in collaboration with all telecenter staff and possibly with some members of the community. The guide sets
the agenda for the discussion. It should grow directly from the questions that you have
regarding the role and operations of the telecenter. These questions should be clearly
identified in the first steps of the needs assessment where you indicate the users and
the uses of the assessment.
When formulating questions for the interview guide, you should consider two principles:
? Questions should be ordered from the more general to the more specific (for
example, ask about the general situation within a community before you ask about a
particular kind of problems or a problem-solving approach).
? Questions of greater importance should be placed early, near the top of the guide,
and be asked earlier during the group session – while those of a lesser significance
should be placed near the end.
If you find there is a conflict between these two principles, start with general
questions, and move to specific questions and then back to a set of more general
questions. The number of questions should be fewer than 10, ideally around five or six.
Types of questions
Questions should be phrased carefully. Certain types of questions impede group discussions. Avoid using questions that can be easily answered with "yes" or "no"
because they do not help stimulate discussion. Be careful of why? questions that might
put participants on the defensive and cause them to take “politically correct” sides on
The questions should be unstructured and open-ended because this allows respondents to answer from a variety of dimensions, and to tell their story in their own
words. These types of questions allow the participants to add details that can result in
Some examples of general open-ended questions include:
What do you think about the healthcare situation of your community?
What are the main difficulties in living in this community?
How have you lives changed during the last few years?
Where do you get new information about government programs?
Or more specific: What do you think about the new telecenter coming to the community?
How did you feel using the telecenter services?
What do you like best about the telecenter?
Step 5: Conduct the interview:
As a facilitator you should attempt to build rapport in the group. Often participants do not know what to expect from focus group discussions. It is therefore helpful to have
the facilitator outline the purpose and format of the discussion at the beginning of the
session, and set the group at ease. Participants should be told that the discussion is
informal, everyone is expected to participate, and divergent views are welcome.
It is always a good idea to have group members introduce themselves and tell a little about themselves. This method can help "break the ice."
To keep the discussion flowing, the facilitator:
? Should practice the discipline of listening to others in group situations;
? Should memorize the questioning route;
? Should try to listen and think at the same time;
? And should be able to manage time, in particular, noting when a topic has been
exhausted and further discussion will yield little new information and in that case
pose a new question or end the session.
? Should control the discussion. In most groups a few individuals dominate the
discussion. To balance out participation:
o Direct questions to individuals who are reluctant to talk
o Give non verbal clues (look in another direction or stop taking notes when an
individual talks for an extended period)
o Intervene, politely summarize the point, then refocus the discussion
o Take advantage of a pause and say, “Thank you for that interesting idea,
perhaps we can discuss it separately. Meanwhile with your consent, I would
like to move to another item.”
Step 6: Collect data
Tape recorders are invaluable for focus group discussions; however, they are prone to pick up background noises. Set up microphones and the recorder prior to the
focus group session, and make it visible to participants. As the facilitator, you should
encourage participants to speak one at a time to avoid garbling the tape. It is a good
idea to ask members of the group to identify themselves before they speak.
The facilitator can attempt to make notes, or an assistant facilitator can try to capture exact phrases and statements made by participants. The consideration here is
that the note taking should not interfere with the discussion. Notes should be complete
and useable in the event the tape recorder stops working. It is advisable, regardless of
the method of data collection, that the moderator makes field notes after each session to
facilitate data analysis.
Generally, after each group discussion, the team should briefly summarize the information provided by the focus group's participants, the team’s impressions, and the
implication for the assessment..
Step 7: Analyze the focus group data
Analyzing the results is the most difficult part of a focus group. In particular, it is difficult to remain objective and refrain from making judgements when evaluating focus
group results. Therefore, if possible have someone available to do the analysis before
carrying out a series of focus groups. Either locate a local person with experience in
analysing focus group results, arrange for someone of your team to be trained, or hire an
However, if you have to do the analysis yourself, the first step is to transcribe the sessions. This will provide a complete record of the discussion and will facilitate analysis
of the data. The next step is to analyse the content of the discussion. Start with reading
each transcript and highlight sections that correspond with the interview guide questions.
Analyse each question separately. After reviewing all the responses to a question or
topic, write a summary statement that describes the discussion.
In analyzing the results, you and your team should consider:
? Words: Weigh the meaning of words participants used. Can a variety of
words and phrases categorize similar responses?
? Framework: Consider the circumstances in which a comment was made
(context of previous discussions, tone and intensity of the comment)
? Internal agreement: Figure out whether shifts in opinion during the discussion
were caused by group pressure.
? Precision of responses: Decide which responses were based on personal
experience and give them greater weight than those based on vague