Marking Criteria for Assessing Portfolios submitted for the

By Sylvia Perkins,2014-04-24 16:31
11 views 0
Marking Criteria for Assessing Portfolios submitted for the

    Marking Criteria for Assessing Portfolios submitted for the Qualification in Forensic Psychology (Stage 2)

    Background & Guiding Principles

    In the process of developing from an in-training to a chartered forensic psychologist, a key issue is the assessment of competence through both practice examples and integration of theory.

    A portfolio is a collection of evidence, usually in written form, of both the products and processes of learning. It attests to achievement and personal and professional development by providing critical analysis of its contents”

    (Webb, Endacott, Gray, McMullan, Miller & Scholes, 2002; McMullan, 2008). The checklist approach for assessing portfolios is inadequate and does not sufficiently allow for the complexity and individuality of each portfolio. The problems in assessing portfolios include:

    ; Consistency & reliability

    ; Inter rater moderation

    ; Subjectivity vs. objectivity

    ; The use of evidence

    ; The use of reflective writing and commentaries

    ; The use of marking criteria

    ; Agreement on what is being assessed

    The definitions of competence and therefore how this can be assessed differ greatly amongst individuals.

    Published marking criteria are often inadequate to encompass the complexities of demonstrating practice outcomes or the application of theory to practice within a specific context.

    The key stages common to most practice and demonstration of competence in applied psychology are:

    Assessing planning implementing evaluating

    This cycle is demonstrating within the competencies of each of the Core Roles of the Qualification. It is this cycle or process as well as the individual elements within it which is of interest.

    The Design of the Portfolio

    Endacott, Gray, Jasper, McMullan, Scholes, & Webb (2004) identify 4 „models‟ of

    portfolios within nursing education which seem to generalise to other health settings.

    1. Shopping Trolley: This portfolio contains anything and everything that has

    been used or produced during the period of supervised practice. The file is

    simply a set of documents with little analytical content or cohesion.

    2. Toast Rack: This portfolio has a number of discrete elements that assess

    different aspects of practice and/or theory. The folder itself acts as a useful

    method of organising the submission into assessment headings but there is

    no real cohesion overall.

    3. Spinal Column: This portfolio is structured around the competencies, in a

    similar way to the toast rack, with evidence slotted in behind each

    competency „vertebra‟ to demonstrate how each competency is met.

    Analytical accounts tend to be included to show how the evidence meets the

    competency. The evidence is there to support the narrative account.

    4. Cake Mix: This portfolio is where the sum is more than the parts in what is

    said about the candidate as a practitioner. The emphasis is on integration

    with an overarching narrative. Only evidence included in that reflective

    narrative is presented.

    The portfolio should aim to fall somewhere between the spinal column and cake mix approach to be demonstrating the standard required.

    The basic components of portfolios are:

    A statement of the learning outcomes

    Reflective reviews/commentary discussing achievement

    Evidence to support the claims made (primary/secondary etc)

    Assessment Criteria

    The operating assumption is that it is the totality of the portfolio that needs to fulfil the

    marking criteria.

    The presentation of the work and its organisation is critical in enabling an assessor to determine whether it is of sufficient standard. The structure and intention of the portfolio along with the outcome need to be clearly stated at the beginning of the work.

    The portfolio must contain a reflective review in which the candidate reviews the

    portfolio content and makes a case for competence. It is this account which is

    assessed as primary evidence alongside additional evidence (primary/secondary) submitted to support the claims.

    The evidence presented needs to be logically organised and referenced within this narrative account.

    Each criterion, where it applies, is presented as a separate statement. The criteria are generic and present a standard against which the specific module outcomes can be compared.

    Standards of Presentation

    ; The portfolio must have a coherent and obvious structure, be indexed and

    cross referenced.

    ; The portfolio should be typographically correct. Errors will result in a

    maximum award of a conditional pass.

    ; It should be written in standard English.

    ; APA referencing system must be employed.

    ; The portfolio should contain a signed copy (by Chief Supervisor) of the agreed

    exemplar plan. Where this is not possible, clear explanation is required or

    alternative submission such as an initial letter of acknowledgement from the

    previous Chief Supervisor.

    ; For submission, comb-binding is preferable to lever arch as it less likely to

    split and mix the papers.

    ; Standard margins should be used for the reflective report.

    ; A font size no smaller than 12 point should be used for the reflective report.

    ; The portfolio should be checked for confidentiality/anonymity of third parties. rdAny 3 party information will result in a maximum award of a conditional pass

    and can be a sign of poor ethical practice, as well as breaching internal

    organisational information sharing protocols.

    ; A reflective narrative report is required to a maximum of 1000 words per

    exemplar. This can be combined into one maximum 2000 word narrative

    across a Core Role. Any submissions 10% above this will be returned


    Presentation of evidence is considered extremely important in demonstrating the ability to organise work to show that learning outcomes have been achieved. In the past the Board of Assessors has been relatively lenient in reviewing poorly presented and disorganised submissions to the benefit of some candidates. This is not a practice which is expected to continue.

    The nature of evidence

    All those involved in the process are sometimes confused about what evidence is required to support the statements made in the narrative. The Exemplar Plans (EPs) provide a way of agreeing the type of evidence which will be submitted as a form of

    planning for the trainee and supervisor. In many cases, the evidence originally listed may not in fact be the best evidence to submit once the candidate develops the narrative. Where this is the case, advice should be sought from the Chief Supervisor. Assessors are expected to review the evidence submitted against the EP. Where minor discrepancies exist, they are able to exercise their judgement if an explanation is provided. Where no explanation and/or large discrepancies exist, the exemplar/Core Role will not be assessed until clarification is obtained. Both the spinal column and cake mix assume that only the evidence which is referenced in the narrative, which is necessary to support statements needs to be submitted as part of the portfolio evidence. In practice, many candidates for the Qualificationhave understandably started with a more „shopping trolley‟ evidence

    presentation and have moved towards the toast rack. It is safe to say poor portfolios have remained in the toast rack area of development and do not have sufficient coherence or integration. A candidate should therefore consider what is useful evidence to support the claims to competency in a particular area; for example

    what does a risk assessment report show what elements of it are specifically

    meeting the different competencies within Core Role 1? This should be highlighted within the reflective report.

    Case studies are often used as a means to assist candidates in achieving a cake mix approach. An overall reflective report is still required to map aspects of the case studies to the Qualificationguidance.

    Assessing the Portfolio

    Guiding Principles:

    ; Each portfolio is unique to the individual and thus not amenable to


    ; The criteria can be used for both assessment and self assessment.

    ; A grade is to be awarded of „distinction‟ - competence demonstrated

    conditional pass competence not yet demonstrated.

    ; The candidate and supervisor should be able to join these grades and

    feedback together to identify strengths and weaknesses for the practitioner for

    future professional practice and development.

    ; The purpose of the Qualificationis to evidence the practice of those in training

    to demonstrate that they are sufficiently competent to practice without

    supervision. It is not expected that trainees will have no weaknesses, nor

    developmental areas. Assessors must decide whether work is of an

    acceptable standard which would benefit from feedback, advice and guidance.

    In other words, the benchmark is “good enough” rather than “perfect” or “near

    perfect”: it is recognised that individuals will continue to develop. This

    judgement must be made across the Core Role, with strengths in certain

    areas able to compensate for weaknesses in others.

    ; A set of „criteria‟ do exist. These are listed below:

    o Each Core Role must demonstrate around 90-110 days worth of

    practice diaries. Where this is not the case, the assessor must flag it in

    the assessment report.

    o Each Core Role must meet the majority of competencies. It is not

    possible to say that a certain % must be met as it is the quality of the

    overall submission, not merely meeting each competency, which is

    relevant. The emphasis, it will be recalled, is on the individual making

    an effective overall case for having demonstrated sufficient experience

    and mastery of the Core Role.

     Where a competency is not attempted within an exemplar, this must o

    have been agreed in advance and be evidenced by the exemplar plan.

    o The indicators within each competency in the guidance provide

    examples of the sort of content expected within the competency. They

    are not an exhaustive list, nor does a candidate have to meet a

    particular proportion of the indicators. Again it is the quality of the

    submission overall, not the quantity of criteria hit. Substantial omissions

    may weaken the case for completion, but this principle also means that

    an arid, exhaustive “tick- box” approach is neither required nor


    o Each submission must meet clerical and other presentational guidance.

     Each submission must meet the standards of ethical practice. o

    Concerns identified must be noted on the assessment report.

    Candidates may be asked to attend to such issues. In some cases,

    reference will be made to the supervisor regarding unethical practice.

    o Guidance states that the supervision log entries should be signed by

    the supervisor. If this is not the case, at the least an explanation as to

    why not, and a letter from the supervisor confirming the entries as

    accurate is required.

Marking Criteria

    There are four levels of marks recognised by the British Psychological Society‟s

    Statutes. As these are defined in the Society‟s generic regulations, they must be

    adhered to by the Division of Forensic Psychology Board of Assessors. Distinction:

    Excellent work demonstrating the ability to work independently and competently across all of the competencies required for this Core Role. The exemplars (including the Practice Diary) demonstrate excellent organisation and provide a comprehensive illustration of how the candidate meets all of the requirements. Evidence that the candidate has employed a systematic approach to work undertaken and considered ethical issues. The Supervision Log, Practice Diary and summary statement all provide substantial evidence of reflective practice. Excellent presentation in accordance with the standards set.

    Competence Demonstrated:

    Satisfactory work demonstrating an adequate level of potential to work competently across the majority of competencies required for this Core Role. Work represented in the exemplar is clearly of a forensic nature and provides evidence of depth and breadth of experience. Evidence of reflective practice is provided through the