DOC

Marking Criteria for Assessing Portfolios submitted for the

By Sylvia Perkins,2014-04-24 16:31
6 views 0
Marking Criteria for Assessing Portfolios submitted for the

    Marking Criteria for Assessing Portfolios submitted for the Qualification in Forensic Psychology (Stage 2)

    Background & Guiding Principles

    In the process of developing from an in-training to a chartered forensic psychologist, a key issue is the assessment of competence through both practice examples and integration of theory.

    A portfolio is a collection of evidence, usually in written form, of both the products and processes of learning. It attests to achievement and personal and professional development by providing critical analysis of its contents”

    (Webb, Endacott, Gray, McMullan, Miller & Scholes, 2002; McMullan, 2008). The checklist approach for assessing portfolios is inadequate and does not sufficiently allow for the complexity and individuality of each portfolio. The problems in assessing portfolios include:

    ; Consistency & reliability

    ; Inter rater moderation

    ; Subjectivity vs. objectivity

    ; The use of evidence

    ; The use of reflective writing and commentaries

    ; The use of marking criteria

    ; Agreement on what is being assessed

    The definitions of competence and therefore how this can be assessed differ greatly amongst individuals.

    Published marking criteria are often inadequate to encompass the complexities of demonstrating practice outcomes or the application of theory to practice within a specific context.

    The key stages common to most practice and demonstration of competence in applied psychology are:

    Assessing planning implementing evaluating

    This cycle is demonstrating within the competencies of each of the Core Roles of the Qualification. It is this cycle or process as well as the individual elements within it which is of interest.

    The Design of the Portfolio

    Endacott, Gray, Jasper, McMullan, Scholes, & Webb (2004) identify 4 „models‟ of

    portfolios within nursing education which seem to generalise to other health settings.

    1. Shopping Trolley: This portfolio contains anything and everything that has

    been used or produced during the period of supervised practice. The file is

    simply a set of documents with little analytical content or cohesion.

    2. Toast Rack: This portfolio has a number of discrete elements that assess

    different aspects of practice and/or theory. The folder itself acts as a useful

    method of organising the submission into assessment headings but there is

    no real cohesion overall.

    3. Spinal Column: This portfolio is structured around the competencies, in a

    similar way to the toast rack, with evidence slotted in behind each

    competency „vertebra‟ to demonstrate how each competency is met.

    Analytical accounts tend to be included to show how the evidence meets the

    competency. The evidence is there to support the narrative account.

    4. Cake Mix: This portfolio is where the sum is more than the parts in what is

    said about the candidate as a practitioner. The emphasis is on integration

    with an overarching narrative. Only evidence included in that reflective

    narrative is presented.

    The portfolio should aim to fall somewhere between the spinal column and cake mix approach to be demonstrating the standard required.

    The basic components of portfolios are:

    A statement of the learning outcomes

    Reflective reviews/commentary discussing achievement

    Evidence to support the claims made (primary/secondary etc)

    Assessment Criteria

    The operating assumption is that it is the totality of the portfolio that needs to fulfil the

    marking criteria.

    The presentation of the work and its organisation is critical in enabling an assessor to determine whether it is of sufficient standard. The structure and intention of the portfolio along with the outcome need to be clearly stated at the beginning of the work.

    The portfolio must contain a reflective review in which the candidate reviews the

    portfolio content and makes a case for competence. It is this account which is

    assessed as primary evidence alongside additional evidence (primary/secondary) submitted to support the claims.

    The evidence presented needs to be logically organised and referenced within this narrative account.

    Each criterion, where it applies, is presented as a separate statement. The criteria are generic and present a standard against which the specific module outcomes can be compared.

    Standards of Presentation

    ; The portfolio must have a coherent and obvious structure, be indexed and

    cross referenced.

    ; The portfolio should be typographically correct. Errors will result in a

    maximum award of a conditional pass.

    ; It should be written in standard English.

    ; APA referencing system must be employed.

    ; The portfolio should contain a signed copy (by Chief Supervisor) of the agreed

    exemplar plan. Where this is not possible, clear explanation is required or

    alternative submission such as an initial letter of acknowledgement from the

    previous Chief Supervisor.

    ; For submission, comb-binding is preferable to lever arch as it less likely to

    split and mix the papers.

    ; Standard margins should be used for the reflective report.

    ; A font size no smaller than 12 point should be used for the reflective report.

    ; The portfolio should be checked for confidentiality/anonymity of third parties. rdAny 3 party information will result in a maximum award of a conditional pass

    and can be a sign of poor ethical practice, as well as breaching internal

    organisational information sharing protocols.

    ; A reflective narrative report is required to a maximum of 1000 words per

    exemplar. This can be combined into one maximum 2000 word narrative

    across a Core Role. Any submissions 10% above this will be returned

    unmarked.

    Presentation of evidence is considered extremely important in demonstrating the ability to organise work to show that learning outcomes have been achieved. In the past the Board of Assessors has been relatively lenient in reviewing poorly presented and disorganised submissions to the benefit of some candidates. This is not a practice which is expected to continue.

    The nature of evidence

    All those involved in the process are sometimes confused about what evidence is required to support the statements made in the narrative. The Exemplar Plans (EPs) provide a way of agreeing the type of evidence which will be submitted as a form of

    planning for the trainee and supervisor. In many cases, the evidence originally listed may not in fact be the best evidence to submit once the candidate develops the narrative. Where this is the case, advice should be sought from the Chief Supervisor. Assessors are expected to review the evidence submitted against the EP. Where minor discrepancies exist, they are able to exercise their judgement if an explanation is provided. Where no explanation and/or large discrepancies exist, the exemplar/Core Role will not be assessed until clarification is obtained. Both the spinal column and cake mix assume that only the evidence which is referenced in the narrative, which is necessary to support statements needs to be submitted as part of the portfolio evidence. In practice, many candidates for the Qualificationhave understandably started with a more „shopping trolley‟ evidence

    presentation and have moved towards the toast rack. It is safe to say poor portfolios have remained in the toast rack area of development and do not have sufficient coherence or integration. A candidate should therefore consider what is useful evidence to support the claims to competency in a particular area; for example

    what does a risk assessment report show what elements of it are specifically

    meeting the different competencies within Core Role 1? This should be highlighted within the reflective report.

    Case studies are often used as a means to assist candidates in achieving a cake mix approach. An overall reflective report is still required to map aspects of the case studies to the Qualificationguidance.

    Assessing the Portfolio

    Guiding Principles:

    ; Each portfolio is unique to the individual and thus not amenable to

    standardization.

    ; The criteria can be used for both assessment and self assessment.

    ; A grade is to be awarded of „distinction‟ - competence demonstrated

    conditional pass competence not yet demonstrated.

    ; The candidate and supervisor should be able to join these grades and

    feedback together to identify strengths and weaknesses for the practitioner for

    future professional practice and development.

    ; The purpose of the Qualificationis to evidence the practice of those in training

    to demonstrate that they are sufficiently competent to practice without

    supervision. It is not expected that trainees will have no weaknesses, nor

    developmental areas. Assessors must decide whether work is of an

    acceptable standard which would benefit from feedback, advice and guidance.

    In other words, the benchmark is “good enough” rather than “perfect” or “near

    perfect”: it is recognised that individuals will continue to develop. This

    judgement must be made across the Core Role, with strengths in certain

    areas able to compensate for weaknesses in others.

    ; A set of „criteria‟ do exist. These are listed below:

    o Each Core Role must demonstrate around 90-110 days worth of

    practice diaries. Where this is not the case, the assessor must flag it in

    the assessment report.

    o Each Core Role must meet the majority of competencies. It is not

    possible to say that a certain % must be met as it is the quality of the

    overall submission, not merely meeting each competency, which is

    relevant. The emphasis, it will be recalled, is on the individual making

    an effective overall case for having demonstrated sufficient experience

    and mastery of the Core Role.

     Where a competency is not attempted within an exemplar, this must o

    have been agreed in advance and be evidenced by the exemplar plan.

    o The indicators within each competency in the guidance provide

    examples of the sort of content expected within the competency. They

    are not an exhaustive list, nor does a candidate have to meet a

    particular proportion of the indicators. Again it is the quality of the

    submission overall, not the quantity of criteria hit. Substantial omissions

    may weaken the case for completion, but this principle also means that

    an arid, exhaustive “tick- box” approach is neither required nor

    appropriate.

    o Each submission must meet clerical and other presentational guidance.

     Each submission must meet the standards of ethical practice. o

    Concerns identified must be noted on the assessment report.

    Candidates may be asked to attend to such issues. In some cases,

    reference will be made to the supervisor regarding unethical practice.

    o Guidance states that the supervision log entries should be signed by

    the supervisor. If this is not the case, at the least an explanation as to

    why not, and a letter from the supervisor confirming the entries as

    accurate is required.

Marking Criteria

    There are four levels of marks recognised by the British Psychological Society‟s

    Statutes. As these are defined in the Society‟s generic regulations, they must be

    adhered to by the Division of Forensic Psychology Board of Assessors. Distinction:

    Excellent work demonstrating the ability to work independently and competently across all of the competencies required for this Core Role. The exemplars (including the Practice Diary) demonstrate excellent organisation and provide a comprehensive illustration of how the candidate meets all of the requirements. Evidence that the candidate has employed a systematic approach to work undertaken and considered ethical issues. The Supervision Log, Practice Diary and summary statement all provide substantial evidence of reflective practice. Excellent presentation in accordance with the standards set.

    Competence Demonstrated:

    Satisfactory work demonstrating an adequate level of potential to work competently across the majority of competencies required for this Core Role. Work represented in the exemplar is clearly of a forensic nature and provides evidence of depth and breadth of experience. Evidence of reflective practice is provided through the supervision log and practice diary or summary statement. Ethical guidelines have been observed. Satisfactory presentation overall. Assessors may allow weaknesses in some competencies to be compensated for by other areas of strength.

    Conditional Pass:

    This mark is available only where clerical errors are identified, including issues such as signatures required on the supervision log; editing or clarification of tables; grammatical and typographical errors; referencing errors; any failure to anonymise third parties . The conditional pass work must be re-submitted within one month: hence recommendations for Conditional passes must be for clerical issues, and rectifiable within one month. Conditional passes are not designed for additional evidence collation/collection. Where this is wanted, the assessor should consider is it really needed? If yes, then a fail must be awarded. If it is not essential, it can be provided as a piece of advice and guidance as part of a pass report. Competence Not Yet Demonstrated:

    The work described in the exemplar is significantly flawed and demonstrates an inadequate level of competence. The following issues may be grounds for failure:

    ; Ignoring key areas for assessment and intervention (where such work is not to

    be carried out, it should at least be recognised as a need/limitation)

    ; Significant failure to observe ethical guidelines

    ; Significantly flawed research or evaluation designs

    ; Little or no evidence of reflective practice

    ; Significant deviations from the exemplar plan (without sufficient explanation)

    ; A lack of primary evidence (which can include the narrative account)

    ; Insufficient time over the Core Role (whilst some slippage exists for the

    amount of „days‟ required for each exemplar within a Core Role, the rough

    guide of 3 months per exemplar remains)

    ; Presentation is poor and disorganised so that assessment is difficult. The following tables provide a framework for considering Core Role submissions. This is not intended to provide a marking scheme, but to assist in structuring the thinking of candidates, supervisors and assessors of the process of Core Role and practice completion.

    Some of these „criteria‟ will be more relevant than others for certain competencies. For example, a candidate may well not demonstrate much „rationale‟ in evidencing their relationship management (1.3) although they may well show an awareness of the goals and consequences of such relationship maintenance/importance. The grids aim to assist in holistic decision making about the Core Role submission to enable assessors to make an overall judgement about standard.

CORE ROLE ONE Establishing requirement Planning of Establish/develop Implement app/intervention Direct implementation of Evaluating results of for/benefit of maintain working rltnshps app/intervention app/interventions app/intervent. app/intervent Rationale: clarity, logic and relevance of reasoning and decisions including awareness of goals, options, consequences Sensitivity to contextual issues: awareness/incorporation of perspectives/concerns of clients/others. Situational and other influences and constraints. Ethical considerations Technical understanding & use of sources: accurate and appropriate identification and use of concepts, comparisons and procedures drawn from a range of well chosen primary & secondary sources/other expert knowledge Critical thinking: balanced & rigorous evaluation/synthesis of theoretical aspects, methodological issues and research evidence Creativity & independence of thought: original thinking in identifying, examining & integrating key issues and generating new insights Writing & presentation: in accordance with set out requirements, coherent structure, development of arguments, precise and lucid expression, professional standards of grammar, spelling etc

Reflection: indications of learning incl awareness of whether aspects could have been handled better, making sense of experience, deriving implications for future practice

CORE ROLE TWO Designing psychological research Conducting psychological research Evaluating and analysing psychological activities activities research data Rationale: clarity, logic and relevance of reasoning and decisions including awareness of goals, options, consequences Sensitivity to contextual issues: awareness/incorporation of perspectives/concerns of clients/others. Situational and other influences and constraints. Ethical considerations Technical understanding & use of sources: accurate and appropriate identification and use of concepts, comparisons and procedures drawn from a range of well chosen primary & secondary sources/other expert knowledge Critical thinking: balanced & rigorous evaluation/synthesis of theoretical aspects, methodological issues and research evidence Creativity & independence of thought: original thinking in identifying, examining & integrating key issues and generating new insights Writing & presentation: in accordance with set out requirements, coherent structure, development of arguments, precise and lucid expression, professional standards of grammar, spelling etc Reflection: indications of learning incl awareness of whether aspects could have been handled better, making sense of experience, deriving implications for future practice

CORE ROLE THREE Promoting awareness of the Providing psychological Providing psychological Preparing and Responding to informal Providing feedback to

    actual and potential advice to assist and advice to aid the presenting evidence in requests for psychological clients

    contribution s of applied inform problem solving formulation and formal settings information

    psychological services and decision making implementation of policy

    Rationale: clarity, logic and relevance of reasoning and

    decisions including awareness of

    goals, options, consequences

    Sensitivity to contextual issues: awareness/incorporation of

    perspectives/concerns of

    clients/others. Situational and

    other influences and constraints.

    Ethical considerations

    Technical understanding & use of sources: accurate and appropriate

    identification and use of concepts,

    comparisons and procedures

    drawn from a range of well chosen

    primary & secondary sources/other

    expert knowledge

    Critical thinking: balanced & rigorous evaluation/synthesis of

    theoretical aspects,

    methodological issues and

    research evidence

    Creativity & independence of thought: original thinking in

    identifying, examining & integrating

    key issues and generating new

    insights

    Writing & presentation: in accordance with set out

    requirements, coherent structure,

    development of arguments,

    precise and lucid expression,

    professional standards of

    grammar, spelling etc

    Reflection: indications of learning incl awareness of whether aspects

Report this document

For any questions or suggestions please email
cust-service@docsford.com