By Beatrice Bennett,2014-10-17 11:52
9 views 0


    Language Policy Division

    Division des Politiques linguistiques Evaluation and assessment within the domain of

    Language(s) of Education

Waldemar Martyniuk (ed.), Mike Fleming, José Noijons

Languages of Education

    Language Policy Division, Strasbourg

    Evaluation and assessment within the domain of Language(s) of Education

    Waldemar Martyniuk (ed.), Jagiellonian University, Poland Mike Fleming, University of Durham, United Kingdom José Noijons, CITO, The Netherlands

Intergovernmental Conference

    Languages of schooling within a European framework for Languages of Education: learning, teaching, assessment

Prague 8-10 November 2007

Organised by the

    Language Policy Division, Council of Europe, Strasbourg in co-operation with the

    Ministry of Education, Youth and Sports, Czech Republic

The opinions expressed in this work are those of the authors and do not

    necessarily reflect the official policy of the Council of Europe.

All correspondence concerning this publication or the reproduction or translation of

    all or part of the document should be addressed to the Director of School, Out of

    School and Higher Education of the Council of Europe (F-67075 Strasbourg


    The reproduction of extracts is authorised, except for commercial purposes, on condition that the source is quoted.

Language Policy Division

    DG IV Directorate of School, Out-of-School and Higher Education ? Council of Europe


The Challenge of Assessment within Language(s) of Education

    Mike Fleming .............................................................................................9

    The Relevance of International Assessment for the development of a Framework for the Languages of Education

    José Noijons ............................................................................................ 17

Assessing competences at the end of compulsory schooling the Polish case

    Waldemar Martyniuk .................................................................................. 23



    Evaluation and assessment within the domain of Language(s) of Education


    Evaluation and assessment of competences related to language(s) of education is an area of special interest within the Council of Europe‟s project examining the feasibility of

    developing a common European reference document for the LE domain. The discussions 1carried out so far within the Working Group Evaluation and Assessment can be

    summarised as follows:

    1. The fundamental question to be answered when making decisions regarding

    assessment is WHY do we want to have an assessment scheme?‟ The most

    problematic area is usually not the assessment itself but the use made of the

    results, the claims made on the basis for the assessment, and its impact.

    2. Suitable assessment solutions and approaches can be offered only once attainment

    targets (competence standards) have been clearly identified.

    3. Assessment designers should „negotiate‟ with curriculum developers when

    attainment targets (competence standards) are being turned into assessment tasks

    and items.

    4. Only those attainment targets (competence standards) that may be turned into

    observable behaviour can become the subject of assessment procedures (testing


    5. Issues such as validity and reliability can be first addressed at the level of items and


    6. Item level testing may not be capable of/suitable for handling all of the complexity

    of LE competences.

    7. Sampling may be the only feasible option if educational systems (not only individual

    learners) need to be evaluated.

    8. Introducing standards and assessment schemes should be a bottom-up process

    (reflecting the needs of learners and teachers) rather than a top-down procedure

    (reflecting only the needs of decision makers).

    9. Standards and assessment schemes should be promoted as assistance tools rather

    than control measures.

    10. The envisaged LE framework document should serve as an awareness-raising and

    reflective tool, broadening the notion of evaluation and assessment beyond testing

    and levels.

    Below we present a set of studies dealing with some of these issues. The opening paper by Mike Fleming is an overview article listing and addressing challenges related to evaluation and assessment within the domain of LE that may serve as a basis for a future chapter in the envisaged LE framework document. The paper concludes with a proposal called „Languages of Education Portfolio‟ – an integrated approach to evaluation and assessment

    within the domain of LE.

    The two case studies that follow illustrate how the issues raised in the overview article are being turned into practice on an international and a national level. The point made by José Noijons when presenting the two international assessment frameworks PISA and PIRLS is

    that even if people, specifically teachers, may not be in favour of internationally produced

     1 Group members: Christoph Arnold, Mike Fleming, Waldemar Martyniuk (co-ordinator), José Noijons


    standards, they are already with us and national policy makers wish to adhere to these standards. His suggestion for the work on an LE framework document is to acknowledge the already established conceptual frameworks of international surveys and the standards that go with them.

    The presentation of a national-level system of compulsory external examinations where competences in a language of schooling Polish is assessed, is an attempt to examine

    how the challenges to standardise and assess competence in LS and LAC are being met in practice. Various forms of external assessment schemes are in use in most of the member states of the Council of Europe. 29 out of the 44 national respondents (including 8 from the German Länder and 5 from the United Kingdom) to a Preliminary survey on curricula for teaching national/official/school languages in compulsory education, carried out by the Language Policy Division in April 2005, claimed to have external examinations in place. A new survey, targeting specifically the use, the scope and the significance of these examinations was sent out by the Language Policy Division in mid July 2007. The results of the new survey will certainly serve as very valuable input for the work on an LE framework document.

    Waldemar Martyniuk


The Challenge of Assessment within Language(s) of Education

    Mike Fleming, University of Durham, United Kingdom


    The purpose of this paper is to provide an overview of some of the key issues and challenges related to evaluation and assessment within the domain of language(s) of education. It incorporates elements of the previous paper distributed at the Strasbourg 2conference in 2006 but is also addresses new more practical issues related to (a) the possibility of situating assessment for language as school subject (LS) within the wider construct of language(s) of education (LE); (b) the need to reconcile portfolio approaches with more formal testing. Assessment is important but it is not without controversy and it can easily lead to polarised views and unhelpful tensions. This is particularly the case in the context of language as school subject (LS) because of the diversity and complexity of its aims. Constructive debate around differences of opinion is always helpful, but too often disagreements about assessment become entrenched and unproductive. This happens for a number of reasons, including:

    ; a failure to recognise that assessment needs to fulfil a wide range of legitimate


    ; an assumption that a single assessment tool will be able to serve all needs;

    ; a lack of awareness that it is the use made of assessment, not necessarily the

    assessment process itself, that will largely determine its impact;

    ; a tendency to search for universal solutions to assessment issues and neglect the

    significance of context.

    Teachers of language as school subject are sometimes hostile to the idea of large-scale or formal testing on the grounds that it diminishes the subject and ignores the significance of context. This view needs to be considered.

    Section One will consider a range of different purposes of assessment based on different potential audiences. An ideal assessment strategy would meet the needs of all those interest groups. Section Two will examine different approaches to assessment in relation to language as school subject (LS), including the value of portfolio assessment. Section Three will consider the implications of situating the assessment of LS within a broad strategy of assessment within languages of education (LE). Section Four will develop further the concept of an integrated approach to assessment which seeks to reconcile its different purposes within an assessment strategy. It will consider the view that a portfolio approach is not necessarily in conflict with more formal testing, as is often assumed. 1. Purposes of assessment

    In its simplest formulation, assessment provides information on whether teaching/learning has been successful. However the information it provides has a number of potential different audiences whose precise requirements may vary. Classroom teachers need regular information on how pupils‟ knowledge, skills and understanding are developing, both to inform how they should adjust their teaching and to determine what kind of feedback is needed to improve pupils‟ learning. On the other hand, school principals and policy makers need additional, broader information on the quality of education in a school or country. The sort of comparative data required for this purpose needs a high level of reliability and uniformity. In the case of language as school subject this requirement is

     2 Intergovernmental Conference on “Languages of Schooling: towards a Framework for Europe”, Strasbourg,

    Council of Europe, 16-18 October 2006. See the Report on:


    challenging because it is difficult to create tests which are manageable but at the same time faithful to the aims of the subject. Employers and society at large also need reliable information which can help certify achievement and provide a basis for selection. Parents too require information which can help them understand their children‟s achievements and

    limitations. Learners themselves need to know how they are progressing and how to improve their performance but they may need to be protected from the potentially demotivating effects of negative assessment.

    The concept of „accountability‟ when used in relation to assessment usually refers to the imposition of systems of assessment external to the learning process as a form of „policing‟ of standards to ensure that the education system is functioning effectively. But the term may be employed more broadly and more positively than this, referring to the different obligations that are relevant to all. Teachers have a responsibility to the learner but also to the needs of the wider society. Policy makers clearly have a duty to the public and need to ensure that the education system is delivering results but they also have responsibilities to the individual learners and need to consider consequences of policies in those terms. The concept of accountability interpreted in this way will take people outside of vested interests in order to see the larger context. Accountability needs to be linked with a process of sharing perceptions and fostering understanding. It is important therefore not to exaggerate differences between different potential „stakeholders‟; what all parties have in

    common is a fundamental concern that assessment should help raise achievement and improve learning. A starting point for resolving tensions related to matters of assessment is to develop understanding of other points of view. A key challenge is to develop a system of assessment that acknowledges the different functions of assessment and it helps to see these as complementary rather than being in opposition to each other.

    2. Approaches to assessment

    The different purposes of assessment lead to different approaches to assessment. Traditionally assessment of language as school subject took a very simple form: pupils were given a narrow written task which was then awarded a grade or mark. This allowed them (and the teacher) to make a judgement of how they ranked in relation the rest of the group (normative assessment). However the absence of clear criteria meant that the information rarely gave an indication of how they could make progress in their learning. Also the test itself often embodied a very narrow conception of what competence in language entailed; it often centred on knowledge of language form and structures (syntax and grammar) and a narrow range of language uses (often only a written form of essay). The implicit understanding of what reading literacy involved was also very narrow, often involving just decoding and literal understanding. In traditional approaches of that kind there was unlikely to be any attention to oral work, to a range of writing purposes, to a wide range of reading and response to reading. On the other hand, the advantage of a fairly narrow approach to assessment was that is was easier to provide reliable outcomes; the more complex the system of assessment becomes, the more difficult it is to ensure that the award of grades or marks for particular outcomes are consistent. In approaches to assessment, two central tendencies emerge which are relevant to language as subject. One places emphasis on the assessment of learning where reliable,

    objective measures are a high priority. The focus here is on making summative judgements which in practice is likely to involve more formal examinations and tests with marks schemes to ensure that the process is sound. An alternative approach is to change the emphasis from assessment of learning to assessment for learning, implying a more

    formative approach where there is much more emphasis on feedback to improve performance. The approach here might be through course work and portfolio assessment in which diverse information can be gathered which reflects the true broad nature of the subject.


Report this document

For any questions or suggestions please email