Microsoft Word - Guidelines_IEP_unis_08_Final_update

By Shane Hill,2014-05-16 19:51
7 views 0
It is based on a SWOT analysis (assess strengths and weaknesses, identify threats and opportunities) and show how the various elements of strategic thinking

    TEMPUS Evaluation Visit

    Institutional Evaluation


    Guidelines for participating institutions


Self-evaluation is a crucial phase in the evaluation process. The self-evaluation phase has two

    aspects that are equally important: the self-evaluation process and the self-evaluation report:

    ? The self-evaluation process is a collective institutional reflection and an opportunity for

    quality improvement of any aspect of the management

    ? The self-evaluation report is one outcome of the self-evaluation process; it provides

    information to the evaluation team, with emphasis on the management's strategic and

    quality management activities.

The goal of both the process and the report is to enhance the institutional capacity for improvement

    and change through self-reflection. This is a crucial phase in which careful consideration should be

    given to maximise the engagement of the whole management.

    As soon as the management has received these guidelines it should begin the self-evaluation

    process basing its work on the checklist provided in Section 2.2 and will write the self-evaluation

    report (Section 2.3).

2.1 The self-evaluation group

    To ensure the success of the self-evaluation, the management will set up a self-evaluation group

    that represents a broad view of the management.

    The self-evaluation will result in a report submitted to the external evaluation team under the

    responsibility of the rector.

2.2 Preparing the self-evaluation report: What kind of information to collect and analyse?

As an important step in the evaluation exercise, the self-evaluation report has three major


    ? To present a succinct but analytical and comprehensive statement of the management’s

    view of quality management and strategic planning

    ? To analyse the strengths and weaknesses of the management, identify the opportunities

    and threats it faces and propose specific actions to address them

    ? To provide a framework against which the institution will be evaluated by the team

As the main vehicle for the management to present itself, the self-evaluation report is also an

    opportunity for the management to reflect critically upon the way it is managed and handles quality

    as a central process in its strategic decision making.

Therefore, the self-evaluation report should not be simply descriptive, but analytical, evaluative and

    synthetic. It is based on a SWOT analysis (assess strengths and weaknesses, identify threats and

    opportunities) and show how the various elements of strategic thinking and quality management

    are interconnected.


Four central questions structure this SWOT analysis:

    ? What is the management trying to do? What are its norms and values, the mission and


    ? How is the management trying to do it? What are the organisational characteristics of the

    managements and its key activities and to what extent these are in line with the norms and


    ? How does the management know it works? To what extent does the management know

    whether its activities and organisational structures meet the management’s objectives?

    ? How does the institution change in order to improve?

2.3 The Checklist

The following checklist will guide the data collection and analysis in the preparation of the

    self-evaluation report. It is important that all the bullet points on the list are addressed by the

    self-evaluation group but, since each management operates within its own specific context, the

    self-evaluation group may want to tailor the checklist before starting its work. If some questions are

    not relevant or if specific pieces of information are impossible to provide, this should be noted in

    relation to the questions.

    The checklist is structured into four major sections that reflect the four central questions mentioned

    above. A fifth optional section considers a strategic priority for the evaluation, which may be selected by the management for more in-depth consideration.

I. Norms and values: What is the management trying to do?

This section discusses institutional norms and values. It analyses the mission and goals of the

    management. The evaluation team will be particularly interested in the strategic choices the

    management has made with regard to its scope and profile.

    ? Governance and management

    ? What is the degree of centralisation and decentralisation that the management aims


    ? Does the management have human resources and gender policies in place?

    ? Academically-related activities: What are the management’s goals for its relationship to

    society (external partners, local and regional government) and its involvement in public


    ? Funding: What should be the management’s relationship to its funding agencies (public

    and others, such as research contractors)?

    ? What balance is the management aiming to achieve in terms of its local, regional, national,


    and international positioning?

    ? Any other institutional goals?

II. Organisation and activities: How is the management trying to do it?

In practice, the institution manages its activities (teaching, research, and service to society) so as to

    realise its mission and goals, while taking account of the specific opportunities and constraints it

    faces. The inevitable discrepancy between what ought to be (norms and values) and what actually

    exists (organisation and activities) is an indicator of the management’s strengths and weaknesses.

    It is the analysis of strengths and weaknesses that constitutes the next phase of the self-evaluation.

The issues addressed in Section I should be re-visited, but rather than stating objectives, Section II

    will reflect upon the management’s strategy in terms of each of these issues and how they are

    achieved, and will analyse the extent to which the management takes full advantage of its

    autonomy. Moreover, each subheading in this section should also contain concrete proposals on

    how identified weaknesses could be remedied and strengths could be further enhanced.

    ? Governance and management: Re-visit questions in Section I by taking the following

    issues into account:

    o Analysis of management practice: what are the respective roles of central-level

    administrators, offices and faculties/institutes? Does co-ordination among

    faculties/institutes take place, and if so how? What does the institutional leadership 1control and decide? What do the deans of facultycontrol and decide with respect


    ? Academic activities and policies (teaching and learning, research)

    ? Funding issues

    ? The selection and promotion of academic and administrative staff

    ? The selection of students

    ? Development of entrepreneurial activities

    o How does the management involve students and external stakeholders in

    institutional governance?

    o How adequate are the management’s human resources, human resource policy

    and practice to current and future needs (e.g., age profile, recruitment, promotion,

    redeployment and staff development);

    o Does the management have a gender policy? To what extent is it successfully


     1The term faculty is used in this text in a generic sense to denote the main structural sub-units of an institution


These questions should be discussed during the site visit:

    ? Academically-related activities: Re-visit questions in Section I by taking the following issues

    into account:

    o Analysis of research and technology transfer, continuing education, regional and

    community service, etc. This can be brief, unless some programmes deserve specific


    o Analysis of student support services: How do the various academically-related

    activities reflect the management’s mission and goals, and how could discrepancies

    between the goals and reality addressed and strengths reinforced?


    ? Funding: Revisit questions in Section I by taking the following issues into account:

    o What is the total budget of the management, including salaries, contracts, etc.?

    o What percentage is allotted by the state or other public authorities, by student fees, by

    private sources (research contracts, foundations, etc.)?

    o Is the state allocation a lump sum, or, if not, what percentage of this allocation is ear-


    o What part of the budget is controlled centrally?

    o What are the amounts allotted to faculties and departments, and according to which

    criteria are they distributed? Are these amounts decided by the management?

    o What are the allocation procedures within the management? Who decides what and


    o What percentage of the budget could be used by the institutional leadership to

    implement new initiatives?

    o Is the management able to calculate the full costs of research and teaching activities?

    o What does the management perceive as strengths and weaknesses in terms of its

    funding, and how could weaknesses be remedied and strengths be further enhanced?

    Site visit questions

    III. Quality assessment practices: What internal quality assessment procedures do you have?

The question “How does the management know it works?” refers to the internal quality processes

    and practices available and operative in the management.

These quality processes include data gathering and an evaluative judgement concerning the

    management’s activities. Processes related to teaching and learning are enshrined in the

    “Standards and Guidelines for Quality Assurance in the European Higher Education Area” (ESG),

     which were adopted by ministers in Bergen (2005)2and are provided below.



    ? Are internal quality processes based on explicit and publicly available quality strategy and

    policy? Are these quality policies widely known and accepted in the institution? Is there a

    shared quality culture?

    ? What is the role of students and stakeholders?

    ? Does the institution have formal mechanisms for approval, periodic review and monitoring

    of its programmes?

    ? Are students examinations based on published criteria, regulations and procedures that are

    applied consistently?

    ? Does the institution have procedures to ensure that teaching staff is competent and


    ? Are the available resources to support student learning adequate for each programme


    ? Does institution have regular quality procedures to monitor other activities besides


    o Research activities

    o Administrative processes

    o External relations (local, regional, national and international)

    ? How does the institution ensure that the data collected via quality procedures is extensive,

    analysed and used for effective management of the activities?

    ? To what extent are the outcomes of internal quality processes used in decision making and

    strategic development? For instance, if student course evaluations are carried out, how

    does the unit concerned (programme, department, etc.) handle the results from these

    evaluations if they fall outside norms set by the unit? Or what are the consequences, for

    example, of external evaluations of research units?

    ? How will the management react to any strengths or weaknesses revealed by this analysis?

    ? Does the management publish regularly up to date, impartial and objective information

    about questions its activities?

    IV Strategic priority

2.4 The structure of the self-evaluation report

After the self-evaluation group has collected and analysed the data as outlined above, it will

    synthesise all the information gathered and present its findings in the self-evaluation report. The

    following proposes a structure for this report. The report should be fairly short, analytical, reflective

    and critical.


    Brief analysis of the self-evaluation process:

    ? Who are the self-evaluation team members?

    ? With whom did they collaborate?

    ? To what extent was the report discussed across the management?

    ? What were the positive aspects, as well as the difficulties, encountered in the


    self-evaluation process?

    Institutional context (The information given during the kickoff meeting should be sufficient but may be extended as


    Brief presentation of the management in its context:

    ? Brief historical overview

    ? Geographical position of the management (e.g., in a capital city, major regional centre,

    concentrated on one campus, dispersed across a city)

    ? A brief analysis of the current regional and national labour-market situation

    ? Number of faculties, research institutes/laboratories, academic and administrative staff and


    ? Autonomy with respect to:

    o Human and financial resources

    o Capacity to set its own profile for teaching, research and innovation

    o Capacity to set its own governing structures


    The conclusion summarises the strengths, weaknesses, opportunities and threats and offers a

    specific action plan to remedy weaknesses and to develop strengths further.

    A useful conclusion has the following characteristics:


    ? Since the goal of the evaluation is to promote ongoing quality and strategic development,

    the report should be honest and self-reflective. Therefore, strengths and weaknesses need

    to be stated explicitly; specifically, it is best to avoid playing down or hiding weaknesses.

    ? Strengths and weaknesses that are not discussed in the body of the report should not

    appear in the conclusion since they would be unsubstantiated.

    ? Strengths and weaknesses that are discussed in the main part of the report are

    summarised again in the conclusion.

    ? Plans to remedy weaknesses are offered in the conclusion in the form of a specific action



    Annexes will include the following:

    ? An organisational chart of the management’s faculties (or any other relevant units of


    ? An organisational chart of the central administration and support services (rector’s office

    staff, campus maintenance, libraries etc.)

    ? An organisational chart of the management structure (rector, council/senate, faculty deans


    and councils, major committees, etc.)

Practical aspects

    ? The university will provide minute-takers.

    ? The maximum length of the self-evaluation report is 20 - 30 pages, excluding the


    ? The self-evaluation report should be read and signed by the rector before being sent to the

    evaluation team. This ensures that the institutional leadership is informed appropriate.

    ? The self-evaluation report should be made available to all institutional members who will

    meet the evaluation team during the site visits.

    ? The report should be sent to the Programme Contractor and to each individual team

    member at least six weeks prior to the site visit.


    3.1 Preparing for the site visits

    The evaluation process is intended to act as a support to develop further the universities’ capacity to change.

    3.2 First visit: agenda and suggested schedule For the participating management, the first visit serves the following purposes:

    ? To contribute to greater awareness by the management at large of the evaluation process

    and its main purpose: to enhance the management’s strategic development and change

    management through an examination of its internal quality arrangements

For the evaluation team, the visit will contribute to develop their understanding of:

    ? the national higher education context

    ? institutional operations in terms of students, staff, finance, facilities and location

    ? the structures and processes of strategic decision making (planning, teaching and research,

    financial flows and personnel policy)

    ? the important local issues with respect to strategic management

    ? the existing procedures for quality assurance

The visit should result in a validation of the self-evaluation report, and the evaluation team should

    get a broad impression of how the management operates (decentralisation, co-ordination, etc.).

    Therefore, the choice of persons the evaluation team meets is highly important.

    An indicative list of persons and bodies that the evaluation team should meet includes:

    ? The rector as well as other members of the rector’s team

    ? The self-evaluation group, including any sub-group

    ? Representatives of the central staff: mainly from the quality office, international relations



The management is responsible for proposing the schedule for the visit, which will need to be

    validated by the evaluation team. A sample schedule for the visit is presented below, but other

    options are also possible in consultation with the evaluation team secretary.

Please adapt the schedule to the characteristics of your management and keep in mind that the

    team will only be present for one visit.

    Time schedule for visit

    Day 0 6 weeks Submission of self-evaluation

    Day 0 Arrival of external team

     First meeting with (vice-)rector

    and self-evaluation team

    Day 1 Meeting with self evaluation


     Meeting with members of the

    administration involved in the


     Meeting of external team first


     Discussion of results with

    self-evaluation team and


    First action lines

     Departure of external team

    Day 2 + 4 weeks Written report submitted


Report this document

For any questions or suggestions please email