DOC

Course Assessment Handbook

By Phyllis Rogers,2014-10-17 11:11
8 views 0
Course Assessment Handbook

    Course Assessment Handbook

    By

    William Peirce, revised by Mike Gavin

    Coordinator, Academic Outcomes Assessment

    Revised February 2005

    Prince George's Community College

    Vera Zdravkovich, Vice President for Instruction Verna Teasdale, Academic Assistant to the Vice President

    ?Prince George's Community College

Table of Contents, Course Assessment Handbook

    I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 3 II. How To Determine That All Instructors Are Addressing All the

    Outcomes on the Course Master Syllabus . . . . . . . . . . . . . . . . . . . . . 8 III. Two Taxonomies of Higher Order Learning: Bloom and Dimensions

    of Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 IV. How to Plan and Map the Course Assessment . . . . . . . . . . . . . . . . . 13 V. How to Construct Objective Test Questions . . . . . . . . . . . . . . . . . . . 16 VI. How to Perform an Item Response Analysis . . . . . . . . . . . . . . . . . . 19 VII. How to Write Essay Questions for Tests and Assignments . . . . . . 21

    VIII. How to Design Rubrics for Scoring Essays, Projects, and

    Performances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 IX. How to Use the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 X. Resources and Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Appendix. Course Assessment Planning Form . . . . . . . . . . . . . . . . . . . . 31 Appendix. Matrix Connecting Course Outcomes with Gen. Ed……34

    Appendix. Summary of Departments’ Use of Results…………………35

     2

I. Introduction

    This handbook is for faculty who are involved with course assessment at Prince George‟s

    Community College. Course assessment is one element in the college‟s assessment program. The college‟s assessment program encompasses:

     General education (assessed by the Academic Profile test)

     Professional programs assessed by accrediting organizations

     Graduates‟ satisfaction with their education at the college

     Disciplines

     Courses

This handbook is intended to guide course assessment only.

Course Assessment Controlled by Faculty

    Because you as a faculty member teach the courses, you are in the best position to know what the course content should be, what students should learn, and how best to determine if they have learned. When you design a course assessment, the information that you receive from analyzing the results can provide valuable insight into how the course can be strengthened to improve student learning.

“Course” versus “Class”

    Before getting into the nitty-gritty of course assessment design, a distinction needs to be made between a course and a class; and an explanation is in order about why course grades do not qualify as a course assessment. A course consists of all the classes (sections) being taught; for example, EGL 101, MAT 112. A class is one section of a course. This handbook is about course assessment, not class assessment.

Course Grades versus Course Assessment

    Course grades do not provide the same insight that a course assessment does.

     Grades give a global evaluation but do not provide sufficiently detailed

    information about which course outcomes students are mastering well and which

    are giving them trouble.

     Course grades alone don‟t stimulate faculty discussions about how to improve

    student learning of particular course outcomes.

     Grades sometimes are based on more than mastery of course content; for example,

    participation, attendance, bonus points.

     Grading standards often vary widely among different instructors and do not

    indicate the same degree of mastery of course outcomes.

     Grade inflation (easy tests, generous grading, extra-credit bonuses) sometimes

    presents a misleading indicator of student mastery of course outcomes.

     3

Grades as Part of Course Assessment

    Grades on individual tests, assignments, and projects may be incorporated into the assessment process if they contribute to the focus of course assessment. The focus of course assessment is to determine how well students enrolled in a particular course are learning the content that faculty who teach the course agree that students should learn.

Benefits of Course Assessment

    Course assessment benefits students when it leads to improved learning of the course content; the faculty benefit as well. The course assessment process provides one of the few opportunities for faculty to discuss course content with each other and, based on the results of an assessment, determine how they can improve student learning in the course. Using assessment results as evidence, instructors might decide to

     Revise the course outcomes to include more higher-order thinking and greater

    intellectual rigor

     Obtain more consistency in large multi-section courses

     Reduce grade inflation by linking test and course grades to mastery of all

    outcomes

     Increase contact with adjunct faculty

     Explore active learning strategies and other teaching methods

     Explore other ways of assessing outcomes

     Explore technological enhancements (labs, equipment, CD tutorial, etc.), using

    the assessment evidence to support a request for increased funding

     Conduct a retreat or workshop for instructors

    (See Appendix B for detailed examples of how departments used results to benefit student learning).

     4

    Overview of the Course Assessment Process

1. Responsibility for Course Assessment.

    Department chairs have a leadership role in the process. They initiate and monitor the assessment of a course from the planning stage to analyzing the results. Faculty teaching subsequent courses should participate in the planning of foundational (101) courses.

2. Course syllabi.

    Before the course is assessed, the course objectives on the course master syllabus need to be rewritten as assessable outcomes (“Upon successfully completing the course, students

    will be able to . . .”) and receive the approval of the Academic Outcomes Assessment Committee (AOAC). Check with your department chair to find out if the course outcomes have received approval from the AOAC. If the course assessment process inspires your department to revise the course outcomes on the master syllabus, you do not need to submit them again to the AOAC.

Beginning spring 2005, three new policies were adopted by the AOAC:

    1) If a submitted syllabus seems not to require higher-order thinking, the AOAC will ask the instructor(s) proposing the course to contextualize the outcome s within the discipline. The AOAC will certify new courses only when the outcomes, based on contextualization, promise to require critical thinking skills of students that can be measured.

    2) If a master syllabus has to be amended to reflect the new directive of requiring higher-order tasks, a department may submit the revised syllabus and the plan simultaneously. Submission of revised syllabi is required only when assessment for the corresponding course is taking place.

    The form for submitting plans has been amended so that departments will identify which questions on the assessment tools measure the outcomes requiring college-level, higher-order thinking tasks. The AOAC will begin to ensure that those outcomes are being measured.

    3) The AOAC and General Education committees will begin to examine all courses submitted for approval to the Curriculum Committee. The committees will review the proposed courses‟ outcomes for their measurability, emphasis on higher-order thinking,

    and, if they are proposed as general education courses, for their correlation with PGCC‟s Core Learning Outcomes. The approved CLOs appear below and on the AOAC website.

    (A rubric that serves as a model for how courses will be measured with regard to those CLOs appears in Appendix A of this booklet.)

     5

    Core Learning/General Education Outcomes

    These core educational outcomes are intended to provide graduates with the ability to:

     Communicate effectively in standard oral and written English.

     Comprehend, analyze, and interpret written materials.

     Use appropriate methods of quantitative reasoning to understand, interpret, and

    manipulate numerical data.

     Understand and apply the scientific method.

     Reason abstractly and think critically.

     Recognize core values of various cultures and display sensitivity in a global

    community.

     Understand the nature and value of the fine, literary, and performing arts.

     Demonstrate informational literacy, and apply technological competencies to

    access, evaluate, and communicate information.

     Apply ethical standards and value judgments to decision making.

3. Schedule.

    Your department has designed a schedule of courses to assess. Departments with separate disciplinary faculty (for example, art and music) assess courses separately and simultaneously.

    If the department falls off schedule, a re-scheduling form is available on the AOAC website and should be submitted the semester that the departure occurs.

4. Process.

    Departments are assessing one or more courses each semester. The course assessment loop typically takes three semesters:

     First semester: planning the course assessment and obtaining approval by the

    AOAC

     Second semester: implementing the assessment

     Third semester: analyzing the assessment results and discussing how to improve

    student learning in the course

Planning the Course Assessment

    All the instructors teaching the courseor a committee of instructors if it is a large multi-section courseshould participate in planning the course assessment. The Course

    Assessment Planning Form is where you record your plan for assessing a course. It is in the Appendix at the end of this handbook and is also available as a Word document at the AOAC web site at http://academic.pgcc.edu/assessment/index.htm. Just download it into your computer and fill it out in Word. (The AOAC web site has other useful resources for

    faculty planning a course assessment.) A course assessment plan has three requirements; they all appear on the Course Assessment Planning Form:

    1. Determine that all instructors are addressing all the outcomes on the course master

    syllabus.

     6

2. Conduct an overall course assessment of at least 60-70 % of the outcomes.

    3. Use the results to improve student learning.

    1. Determine that all instructors are addressing all the outcomes on the course master syllabus.

    This is usually done by each instructor indicating which test items or assignments address specific outcomes and then sharing this information with a committee of other instructors in the course or with the chair or course coordinator. The committee, chair, or course coordinator, determines that everyone is addressing all the outcomes.

    See Section II, How To Determine That All Instructors Are Addressing All the Outcomes on the Course Master Syllabus.

2. Plan and conduct an overall course assessment by selecting which outcomes to

    assess and which course-embedded tests, assignments, and projects to use to assess them. Conduct an overall course assessment of at least 60-70 % of the outcomes.

    Faculty use students‟ responses on tests or written or oral assignments in the following ways:

    ; Analyzing test answers for all students or for a sample

    ; Using a rubric for scoring essay assignments, performances, or projects for all

    students or for a representative sample

The relevant sections in the Table of Contents are:

    III. Two Taxonomies of Higher Order Learning: Bloom and Dimensions of Learning IV. How to Plan and Map the Assessment

    V. How to Construct Objective Test Questions.

    VI. How to Perform an Item response Analysis

    VII. How to Write Essay Questions for Tests and Assignments

    VIII. How to Design Rubrics for Scoring Essays, Projects, and Performances.

3. Use the results to improve student learning. See above, Benefits of Course

     Assessment and Section IX, How to Use the Results.

     7

    II. How To Determine That All Instructors Are Addressing

    All the Outcomes on the Course Master Syllabus

    (Item II on the Course Assessment Planning Form)

    Follow These Steps

    1. Find out how instructors are assessing all

    course outcomes.

    2. Evaluate the tests and assignments.

    3. Consider revising the tests and assignments

    or revising the course outcomes.

1. Find out how instructors are assessing all course outcomes.

    All instructors should provide copies of their tests and assignments to the department chair or to the committee assessing the course. On their tests they should indicate which test questions address which outcomes. On assignment and project instructions, they should indicate which outcomes are addressed.

    If only a few instructors teach the course, simply share with each other your tests and assignments. If the course has many sections, especially if they are taught by adjunct faculty who can‟t come to meetings, design a survey in which instructors explain how they assess the outcomes on the course master syllabus. Include a list of the outcomes and ask for details about how they are assessed. A sample generic survey is provided at the end of this section. The sample survey is also available as a Word document at the AOAC web site at http://academic.pgcc.edu/assessment/index.htm. Download it into your computer and revise it in Word.

    If a small number of faculty are teaching the course, share your tests and assignments with each other at a single meeting and check to see that everyone is addressing all the outcomes. In a large multi-section course, the department chair, course coordinator, or a committee of faculty reviews the surveys, tests, and assignments to ensure that everyone is addressing all the outcomes.

2. Evaluate the tests and assignments.

    Check to see that test questions, assignments, and projects match all the outcomes and require higher order thinking when appropriate. Be guided by Section III, Two Taxonomies of Higher Order Learning: Bloom and Dimensions of Learning.

    3. Consider revising the tests and assignments or revising the course outcomes.

     8

    Some departments are finding this component of the assessment process very useful. Some valuable exchanges of test questions among full-time and adjunct faculty have taken place in some departments. The result is better test questions in everyone‟s sections. The passage of time and changes in textbooks may have caused mismatches between outcomes and test questions. The value of sharing tests and assignments is (a) perhaps your department faculty will ascertain that all outcomes are indeed being addressed by all instructors in a satisfactory way, or (b) some instructors might discover that their tests and assignments emphasize some outcomes more than they deserve and that other course outcomes are receiving less attention. This component of the

    course assessment process might even inspire a revision of the course outcomes.

    Another benefit of looking at how outcomes are addressed is discovering whether test questions intended to assess a higher order thinking skill actually do that. When many instructors look closely at their test questions, they realize that they are assessing recall of textbook chapters and lecturesnot the higher order thinking skills they intended to

    address. Section III of this handbook presents Bloom‟s taxonomy of higher order

    thinking skills and dimensions of learninganother popular taxonomy of higher order

    learning. The faculty‟s discussion of whether all outcomes are suitably addressed might raise the conflict of classroom autonomy versus course integrity. On the one hand, the principle of classroom autonomy supports the right of instructors to determine the content and emphasis of their course. On the other hand, the principle of course integrity supports the right of students to expect that the course they‟re getting is the course

    described in the master syllabus; and transfer institutions expect that the course master syllabus accurately describes the course the receiving institution is accepting. Healthy, well-intentioned departments can resolve this conflict in ways that preserve both classroom autonomy and course integrity and encourage innovation without resorting to the extremes of self-indulgence or rigid conformity.

    On the next page is a sample generic survey that a department might want to modify to obtain information from all instructors about how they are assessing the course outcomes on the master syllabus. The sample survey is also available as a Word document at the AOAC web site at http://academic.pgcc.edu/assessment/index.htm. Just download it into your computer and revise it in Word.

    Review of Item II on Course Assessment Planning Form

    1. Are all instructors in all sections addressing all the

    outcomes on the course master syllabus?

    2. Is higher order thinking assessed?

    3. Should course outcomes, test questions, or assignments

    be revised?

     9

    Sample Survey to Discover How Course Outcomes Are Assessed by

    Faculty

Instructions to the faculty member:

    As part of the assessment of ___________, we are conducting a survey to find out how the outcomes on the course master syllabus are being addressed in all sections. Listed below are the outcomes listed in the master syllabus for ____________. Please indicate which of your tests, assignments, and projects assess these outcomes. Provide copies of your tests and indicate for each question which outcome is being addressed. Provide copies of your assignments and project instructions and indicate which outcome is being addressed. If you do not give tests, or assign essays or projects, please indicate that on the form below. Return the completed survey and your tests and assignments to me by [date]. [Or, if everyone will be bringing their tests and assignments to a meeting of all course instructors, specify the date and place of the meeting.]

Signed: _____________________ Department Chair

    [Outcome 1 copied from course master syllabus.] How do you assess this outcome?

    [Outcome 2 copied from course master syllabus.] How do you assess this outcome?

    [Outcome 3 copied from course master syllabus.] How do you assess this outcome?

    [Outcome 4 copied from course master syllabus.] How do you assess this outcome? [etc.]

    Please attach copies of all your tests and assignments indicating which outcomes are addressed by the test questions and assignment instructions. [or bring the tests and assignments to the meeting on date]

    If you do not give tests or assign essays or projects, please indicate that below.

Your signature______________________________

    Extension _________________

     10

Report this document

For any questions or suggestions please email
cust-service@docsford.com