Associate of Science in Nursing (10)
Dr. Ann Schlumberger
UNIVERSITY OF ARKANSAS AT LITTLE ROCK Plan No. 10
Assessment Progress Report Form - Calendar Year 2004
I. USE OF ASSESSMENT FOR PROGRAM BUILDING AND
As a component of the nursing department’s formal written assessment plan, nursing students are tested at the end of the first year of the program (Custom) and prior to graduation (Exit) using a national nursing examination developed by Health Education Science, Incorporated (HESI). Data provide students with individual feedback on knowledge in key areas. This is used to assist students in working on areas of weakness and provides group data specific to course content and program outcomes that is used by faculty to strengthen the curriculum. In 2004, score calculation converted from percentage to the HESI Predictability Model (HPM), a proprietary mathematical model. Research studies have found the HESI HPM to be highly accurate in predicting success on the nursing licensure exam, NCLEX-RN. Based on HESI research data, a score of 900 is identified as the recommended level of performance, with 850 identified as acceptable. A score of 850 has been established as the benchmark that has been incorporated into program outcomes. The class mean is established as the benchmark for consistency in reporting data correlated with NCLEX-RN data.
Students are required to score 850 on the Custom HESI and two attempts are allowed. The exam is administered at the completion of the first year of the program. Students scoring below 850 on the second take are encouraged to use the tutorial resources available in the Learning Resource Center, but progression is not interrupted. Individual score reports indicate areas of strengths/weaknesses which is used to guide students in remediation. Custom HESI data have proven to be an indicator of success in the second
year of the program. In 2004, a team faculty review was conducted to verify correlation of the HESI Custom exam content to curricular concepts. Eleven revisions were made and will be reflected on the Spring 2005 exam. As a result of data analysis, a score of 850 on the HESI Exit Exam is required for successful completion of Nursing 2910. The exam is administered at the end of the course. Students are allowed three attempts to achieve the required score, with remediation between attempts. To date, all students have achieved the required score and NCLEX-RN pass rates continue to exceed state and national first-write pass rates. The 1999-2004 retake pass rate is 100%.
Analysis of May 2004 HESI Exit Examination data indicated established levels of achievement were met or exceeded in all categories of Program Outcomes/Goals. The performance on the Exit Examination shows improvement in all areas.
The current method of assessment provides the program with trended, aggregated data for use in program improvement. Annually the faculty review data as a component of the systematic Plan for Evaluation. The use of assessment findings for program improvement contributed to the 2001 UALR Department of Excellence Award and the maximum 8-year
accreditation by the National League for Nursing Accreditation Commission (NLNAC Fall 2002 Accreditation Visit). Data indicate continued improvement in graduate performance on the HESI Exit Exam and on NCLEX-RN. Curricular revisions related to admission and progression were implemented in 2004. No curricular changes will be made for 2005. Assessment data are also obtained from annual evaluations. First and second year students complete Program Evaluations. Student response rate is 78%--100%. Graduates and Employers complete a Graduate/Employer Evaluation Survey 6 months after graduation. This survey has a Content Validity Index of 1.0. Response rate ranges from 24%-33%. In an effort to increase response rates, the department contracted with the Institute of Government Survey Research Center (SRC) to conduct a telephone survey of 2004 Graduates and Employers. Response rate increased to 54% and 93% respectively.
During 2004, the department assessed each Program Outcomes/Goals and two Program Objectives.
Program Outcome (Goal 1)
Benchmark: First and second year students will have a mean score of 850 or higher on the overall HESI examination and on subscores in the areas of critical thinking, communication abilities, and therapeutic nursing interventions.
Click to see HESI Test results
*Prior to the score reporting change, the benchmark for HESI testing was 85%.
Benchmark: 90% or more of first and second year students who respond to the Program Evaluation will rate program preparation in the areas of critical thinking, communication abilities, and therapeutic nursing
interventions as satisfactory.
Click to see Program Evaluation
Program Outcome (Goal) 2
Benchmark: 95% or more of RN graduates who actively seek employment will be employed as RNs. Benchmark not met. 83% of RN graduates who actively sought employment are employed as RNs. A component of the curriculum includes a capstone course (Nursing 2910) with a preceptorship in which students work with selected registered nurses in a practice setting. Data indicate that most graduates are employed by an institution that supports the preceptorship component of the curriculum. In Spring 2004, preceptor sites were expanded to include facilities in adjacent counties where increasing numbers of nursing students reside. This action was to meet student and area healthcare workforce demands. This number reflects employment secured prior to graduation. The remaining students were pursuing employment leads or indicated they would seek employment following graduation.
Program Outcome (Goal) 3
Benchmark: 95% or more of RN graduates will initially practice in the State of Arkansas. Benchmark not met. 89% of RN graduates of 2004 are initially practicing in Arkansas. The remaining reported plans for initial licensure in other states, mainly due to spousal employment, military duty, or family
location. States listed included Texas, Maryland, New York, Alaska, and Pennsylvania.
Program Outcome (Goal) 4
Benchmark: Graduates will meet or exceed the average Arkansas NCLEX-RN pass rates on first writes. Benchmark exceeded since 1999. The May 2004 class pass rate is 98%. The Arkansas pass rate is 86%. Of note, UALR nursing graduates exceeded both the Arkansas and national pass rate (85%) and have consistently done so for the past 6 years.
Program Outcome (Goal) 5
Benchmark: 90% or more of graduates who respond to the Graduate Survey will report overall satisfaction with preparation provided by the program. Benchmark exceeded. The Graduate Survey is distributed (per NLNAC guidelines) 6 months following graduation. 98% of respondents of the reported overall satisfaction with preparation provided by the program. Benchmark: 90% of more of first and second year students who respond to the Program Evaluation will report overall satisfaction with preparation provided by the program. Benchmark exceeded; Program evaluations completed in May 2004 by first year students indicated 96% satisfaction and second year students, 92% satisfaction.
Program Outcome (Goal) 6
Benchmark: 90% or more of employers who respond to the Employer Survey will report overall satisfaction with the preparation provided by the program.
Benchmark exceeded; 100% satisfaction reported by employers. Program Outcome (Goal) 7
Benchmark: Annual graduation rate will be 50% or higher. Benchmark not met. The graduation rate for the class of 2004 was 49%. This increased from 2003 following development and implementation of a retention plan. Retention efforts included the restructuring of the first semester course to 3 teaching teams to provide continuity between classroom and clinical instructors, free tutoring in the department, and the addition tutorial
resources for the Learning Resource Center. Admission criteria were revised and Anatomy & Physiology I is required for program admission and support courses (Anatomy & Physiology II, Chemistry 1400 and Biology 2401) must be completed during the program, beginning Fall 2005. Data are being collected to evaluate the effectiveness of an RN Pre-Admission Examination as an admission requirement. This study will be completed in 2006 to determine the reliability of this exam in predicting program success for students entering the program. In 2004, an 8-hour “Survivor Course” was
implemented to introduce new students to the program, facilities, available resources and support, as well as providing information relating to study, time-management, and test-taking skills required for program success. Enrollment was limited to fifty students and 47 completed the course. The course will be offered again this fall and the enrollment limit will be doubled. Faculty volunteer to serve as “guides” for the course. In addition to retention efforts, data are collected via chairperson interview as students exit the program to determine factors that contributed to course failure/withdrawal. For the Class of 2005, 18% cited poor study habits and testing skills as the reason for course failure; 11% cited family or health problems and work interference. 7% withdrew prior to completing the course and the majority of these students cited social issues as interfering with success. While attrition and retention assessment is required by the NLNAC, it should be noted that our retention rate is consistent with the national average (NLNAC, 2004). This year the faculty also assessed two Program Learning Objectives: 1. Use critical thinking while applying the nursing process to provide effective nursing care to human beings with unmet biophysical and psychosocial needs who require promotion, maintenance, and/or restoration of health.
2. Practice accountability and responsibility by adhering to legal and ethical standards of practice.
Program/Learning Objective – Nursing Process
The nursing process includes assessment, analysis, planning, implementation, and evaluation. Benchmark: First and second year students will have a mean HESI score of 850 or higher on nursing process subscores (assessment, analysis, planning, implementation, evaluation).
Click to see HESI Results
Benchmark: 90% or more of first and second year students who respond to the Program Evaluation will rate program preparation in nursing process as satisfactory.
Click to see Program Evaluation results
Program/Learning Objective – Legal/Ethical
Practice accountability and responsibility by adhering to legal and ethical standards of nursing.
Benchmark: First and second year students will have a mean score of 850 and higher on legal/ethical subscores.
Click to see HESI Testing Data
Benchmark: 90% or more of first and second year students who respond to the Program Evaluation will rate program preparation in legal/ethical standards as satisfactory.
Click to see Program Evaluation Data
Benchmark: 90% or more of graduates and employers who respond to the Graduate/Employer Survey will report preparation in legal/ethical standards as satisfactory.
Click to see Follow Up Graduate/Employer Survey
II. FACULTY AND STAKEHOLDER INVOLVEMENT
The department has a systematic method for program assessment and evaluation that was implemented in 1998. Under this governance model, faculty serve on department committees. A revision in 2003 resulted in committee assignments based on individual faculty strengths and preferences as well as department needs. This structure allows for continued systematic program assessment. Bi-annual Systematic Evaluation Meetings (December and May) serve as a forum for presenting assessment data, reviewing program progress, and soliciting faculty input. Each committee chairperson completes a written report using a designated report format. Faculty are provided with copies of all reports in advance of the meeting.
Decisions regarding program maintenance, revision, and/or development are based on trended data analysis and are approved by faculty at these meetings. The department underwent an accreditation visit by the NLNAC in September 2002. We were cited as having a “fully implemented Systematic Plan of Evaluation that serves as a model for assessment activities in nursing education!” Patterns of strength that were cited in the official reporting
document included, “the outstanding contributions of the faculty members within the department in all areas;” “the UALR DON is truly a Department of excellence, not only at UALR but also within the broader nursing education community;” “the full achievement of each of the 22 NLNAC
criterion for accreditation.” Additionally, the NLNAC National Review Panel cited the program as a national model and encouraged the program to publish the Self-Study Report as an example of an exemplary report. The department consistently earns high ratings (3.7—4.0) in college
assessment evaluations. For several years we have submitted a poster at the University Assessment Expo outlining our assessment process. In 2001, a Student Advisory Committee was developed consisting of student representatives from the first and second year classes. The committee replaced a system that required students to attend numerous departmental committee meetings in which student participation was limited. Student participation in the Student Advisory Committee has resulted in monthly meetings in which student/faculty dialogue regarding concerns and suggestions for departmental improvement.
Additional stakeholders involved in the ongoing assessment process include the institutions and community agencies in which nursing students participate in clinical laboratories as students and subsequently as registered nurses. Employers and alumni alike provide formal assessment feedback to the program each January via the Graduate/Employer Follow up Survey. In 2004, we hosted the first annual “Day of Dialogue,” in which representatives from area healthcare agencies were invited to meet with faculty to discuss issues relating to program assessment and improvement. Feedback from the meeting was positive and resulted in several nursing student policy revisions. The community of health care consumers in the metropolitan area of central Arkansas participates in the program’s assessment process. These are accomplished by evaluation of direct care received by students/graduates as
part of extensive service learning projects and nursing care provided by the nursing students/graduates.
Data from all stakeholders is used as a part of the formal systematic plan for evaluation. It is used to evaluate and modify teaching/learning methods, content, and objectives, as trended data indicate. Beginning Fall 2003, the first year of the program was offered at the UALR Benton site and the second year is in progress. This was in response to the community’s request for nursing course availability in Benton and the awarding of a $826,000 Workforce Grant. Public and agency support for the course offerings has been generous and continuation is planned.
Program Outcomes (Goals)
Program Outcomes (Goals) are specified by the National League for Nursing Accreditation Commission (NLNAC). Benchmarks are established by the department.
1. Critical Thinking, Communication Abilities, Therapeutic Nursing Interventions
? First and second year students will have a mean score of 850 or higher on the overall HESI examination score and on subscores in the areas of critical thinking, communication abilities, and therapeutic nursing interventions. ? 90% or more of first and second year students who respond to the Program Evaluation will rate program preparation in the areas of critical thinking, communication abilities, and therapeutic nursing interventions as satisfactory.
2. Graduation Rates
? Annual graduation rate will be 50% or higher.
3. Job Placement Rates
? 95% or more of registered nurse graduates who actively seek employment will employed as registered nurses.
4. Patterns of Employment
? 95% or more of registered nurse graduates will initially practice in Arkansas.
5. NCLEX-RN Pass Rates
? UALR Department of Nursing graduates will meet or exceed the average Arkansas NCLEX-RN pass rate on the first write.
6. Program Satisfaction (students/graduates)
? 90% or more of graduates who respond to the Graduate Survey will report overall satisfaction with the preparation provided by the program. ? 90% or more of first and second year students who respond to the Program Evaluations will report overall satisfaction with the preparation provided by the program.
7. Employer Satisfaction
? 90% or more of employers who respond to the Employer Survey will report overall satisfaction with the preparation provided by the program. Central Student Learning Objectives (Program Objectives)
Program Objectives for the nursing program are required to be linked to concepts in the philosophy (i.e. nursing process, communication, etc.) At the conclusion of the Associate Degree Nursing Program, the student is prepared to:
1. Use the nursing process to deliver effective nursing care for human beings with unmet biophysical and psychosocial needs who require promotion, maintenance, and/or restoration of health.
2. Use effective communication to assist individuals, families, significant others, and health care providers.
3. Practice accountability and responsibility by adhering to legal and ethical standards of practice.
4. Manage nursing care for a small group of individuals by establishing priorities for care and collaborating with other health care providers. 5. Pursue self-established goals for continued self-development and maintenance of clinical competencies.
6. Prepare to contribute to society through participation in community and/or professional activity.
The department uses the following data collection tools that are rated consistent and valid by established content validity indices: ? Program Evaluations: These are completed each April by both first and second year students. In 2003 the evaluations were done electronically via WebCT. Student response was favorable and this method will continue. In addition to Likert scale items, students are asked to provide suggestions for improvement for any item not rated as satisfactory. Students are also provided with the opportunity to document general comments regarding the program. The Program Evaluation tool was developed using NLNAC guidelines and is assessed annually. Data obtained from Program Evaluations are analyzed by the Assessment Committee and distributed to faculty at the August Faculty Meeting. Data are reported as counts and percentages.
? Graduate and Employer Surveys: Surveys are conducted six months following graduation to each graduate and direct supervisor of the graduate. In an effort to increase the response rate to this survey, they are being conducted by the ISR via telephone. Surveys were developed using NLNAC guidelines and are evaluated annually by the Assessment Committee. Data from the surveys are presented at the May SPE faculty meeting. Data are reported as counts and percentages. Faculty implement developments and/or revisions resulting from data analysis the following fall semester. ? Heath Education Science, Incorporated (HESI) testing: This is a proven external prediction model for NCLEX-RN pass rates and a valid measurement tool for use in evaluating nursing curricula. Reliability is calculated using the Kuder-Richardson Formula 20 (KR-20). The KR-20 has indicated high reliability for our students since we began using the HESI exam.