DOC

School Panel Review Report Springfield Academy

By Diana Martinez,2014-08-12 17:04
5 views 0
School Panel Review Report Springfield Academy ...

Panel Review Report Springfield Academy Page 1

    Fall/Winter 2004-05

    School Panel Review Report

    Springfield Academy

    Springfield Public Schools

Introduction

    The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students‟ MCAS performance is not at a level that reaches the school‟s Adequate Yearly Progress targets in English language arts or mathematics or both. The Springfield Academy met these criteria and was one of 15 schools selected for panel review in Fall/Winter of 2004-05. The panel review was conducted on November 2-3, 2004.

    The review panel‟s charge was to analyze data and written information on the school‟s performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

    1. Does the school have a sound plan for improving student performance?

    2. Are the conditions in place for the successful implementation of the school's

    improvement plan(s)?

    The panel‟s responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

    The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether Springfield Academy is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Springfield Academy Profile

    Enrollment

    Springfield Academy is one of the 47 public schools located in Springfield. The school serves students from grades K-12, and contains three separate programs. The Emergency Services Program (ESP) provides extended evaluation and assessment services for students suspected of having emotional/behavioral disabilities. The Assessment Center (AC) provides intervention services for district-wide students under long-term suspension for serious violations of the discipline code. Finally, the Long Term Program provides an academic setting with therapeutic support for students with emotional/behavioral disabilities. The Long-Term Program was the primary focus of this panel review.

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 2

    Fall/Winter 2004-05

    Over the last four years, enrollment at this school has tripled, from 66 in 2001 to 193 as of October 1 of this school year. There have also been small yet noticeable changes in student demographics.

    Between 2001 and 2004, the proportion of White students attending Springfield Academy declined from 36 to 24 percent, while the percentage of Black students rose from 20 to 30 percent. This year 46 percent of students are reported as Hispanics, a slight increase over the 42 percent represented in 2001. Asian students, who accounted for 2 percent in 2001, this year account for 1 percent. In 2004 90 percent of Springfield Academy‟s students were from low-

    income families. Eight percent of the students in 2004 are reported as having a primary language other than English, but none of the school‟s students are reported as being Limited English Proficient. This 2004 school year 97 percent of students are reported to be receiving special education services, which is an increase of 30 percent over last year.

    In 2004, Springfield Academy registered an attendance rate of 84.8 percent, with students absent 20.4 days on average. The school‟s retention rate was 8 percent in 2003, the last year for which these data are available. No out-of-school or in-school suspensions were reported; the dropout rate for grades 9-12 in 2003 was 18 percent.

Staffing

    The 2004-2005 Springfield Academy staffing report indicates that the school is composed of 3 administrators, 18 guidance staff members, including 2 school psychologists, and 41 teachers. Approximately 30 educators at this school have taught there for three or fewer years. All but 6 staff members are certified in the subject that they teach.

MCAS Overview

    Students at Springfield Academy are assessed in grades 3, 4, 7, and 10 in English language arts (ELA) and in grades 4, 6, 8, and 10 in mathematics. Since 2001, the school has not been found to make Adequate Yearly Progress (AYP) in either ELA or mathematics. In the school‟s Cycle III

    End-of-Cycle (2003-2004) and Mid-Cycle (2003) AYP Reports, the school failed to make AYP in ELA and mathematics in the aggregate and for the two reported subgroups: Special Education 1and Free/Reduced Price Lunch.

    Due to the transitory and expanding nature of Springfield Academy‟s population, MCAS results are not available for all grade levels and subject areas for the past five years. Further, due to the small populations tested at all grade levels, a degree of caution is required in examining the school‟s MCAS results. The data available is described below.

     1 In accordance with the federal No Child Left Behind Act passed in 2001, student performance is disaggregated by the following subgroups: Limited English Proficient, Special Education, Free/Reduced Price Lunch, African-American/Black, Asian or Pacific Islander, Hispanic, Native American, and White. A minimum of 40 students (or 5% of the total number of students assessed, whichever is greater) per subgroup is required to issue a statistically sound rating or determination of Adequate Yearly Progress (AYP). The subgroups meeting the minimum sample size at Springfield Academy in 2004 were Special Education and Free/Reduced Price Lunch (for both subject areas).

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 3

    Fall/Winter 2004-05

Student Performance in English Language Arts

GRADE 3

All Students

    The performance of regular education students in grade 3 Reading has improved during the two years of data available. In 2003, 36 percent of students performed at the Needs Improvement level and 55 percent at Warning. In 2004, the percentage of Needs Improvement students increased to 67 percent while the percentage of Warning students decreased to 33 percent. In each of the two years, however, only 11 students were assessed.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 4

All Students

    At the grade 4 level in ELA, only one year of data is available. In 2004, 19 students were assessed. Of these, 37 percent scored in the Needs Improvement category, and 63 percent scored in the Warning category.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 7

All Students

    At the grade 7 level in ELA, four years of data are available; these data show a trend toward improvement and away from Warning/Failing. In 2004, 8 percent of students scored in the Proficient range, 46 percent in Needs Improvement, and 46 percent in Warning. In 2001, only 12 percent of students scored in the Needs Improvement range, and 88 percent scored in Warning range. Between 20 and 24 students were assessed in each of these four years.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 4

    Fall/Winter 2004-05

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 10

All Students

    At the grade 10 level in ELA, only two years of data are available; these data reflect no substantial change. In 2004, 36 percent of students scored in Needs Improvement, and 64 percent in Failing. In 2003, only 33 percent of students scored in the Needs Improvement range, and 67 percent scored in the Failing range. Only 14~15 students were assessed in each of these two years.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

Student Performance in Mathematics

GRADE 4

All Students

    At the grade 4 level in mathematics, only two years of data is available. In 2004, 20 students were assessed; in 2003, 9 students were assessed. In 2004, 10 percent scored in the Needs Improvement category, and 90 percent scored in the Warning category. These percentages were similar in 2003.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 6

All Students

    At the grade 6 level in mathematics, four years of data are available; these data show a slight improvement trend, but are skewed by the small sample sizes. In 2004, 0 percent of students scored in the Proficient range, 22 percent in Needs Improvement, and 78 percent in Warning. In 2003, 9 percent of students scored in the Proficient range, 0 percent in the Needs Improvement

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 5

    Fall/Winter 2004-05

    range, and 91 percent in Warning. In 2001, 100 percent of students scored in the Warning range. Between 11 and 17 students were assessed in each of these four years.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 8

All Students

    At the grade 8 level in mathematics, five years of data are available; these data show slight variation, but are skewed by the small sample sizes. In 2004, 14 percent of students scored in Needs Improvement, and 86 percent in Warning/Failing. In 2003, 100 percent scored in the Warning/Failing range. In 2002, 26 percent of students scored in Needs Improvement, and 74 percent in Warning/Failing. Between 19 and 29 students were assessed in each of these five years.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

GRADE 10

All Students

    At the grade 10 level in mathematics, three years of data are available; these data reflect a slight improvement trend, but are skewed by the small sample sizes. In 2004, 21 percent of students scored in the Proficient range, 29 percent of students scored in Needs Improvement, and 50 percent in Failing. In 2003, 11 percent of students scored in the Needs Improvement range, and 89 percent scored in the Failing range. Between 11 and 18 students were assessed in each of these three years.

Special Education

    As described above, 97% of the Springfield Academy student body currently receives Special Education services; as such, MCAS results for this subgroup reflect those for all students.

Limited English Proficient

    Fewer than 10 Limited English Proficient students were assessed in each of the two years for which data is available (2003 & 2004).

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 6

    Fall/Winter 2004-05

    PANEL RESPONSES TO THE KEY QUESTIONS

KEY QUESTION 1: DOES THE SCHOOL HAVE A SOUND PLAN FOR IMPROVING

    STUDENT PERFORMANCE?

    Springfield Academy has constructed a detailed improvement plan that contains many of the elements of a sound plan, but as currently written, the plan does not adequately identify the specific gaps in knowledge and skills for its unique student population nor does it provide a logical connection between the problems, causes and solutions that will be required to remedy academic weaknesses.

    A. Has the school analyzed appropriate data and program information to accurately

    identify the gaps in student performance and determined why those gaps exist? The Springfield Academy School Improvement Plan (SIP) represents the first steps toward an effective strategy for renewed focus on the academic program, but it is limited in the accuracy of its definition of the specific learning gaps for its students. The school‟s plan was developed from a limited source of data, broadly defines learning needs within ELA and math, and does not target the specific academic weaknesses of its unique population. The plan identifies causes that, while plausible, are not supported by evidence as the root causes of poor student achievement specific to the Springfield Academy student population.

    As an alternative school, Springfield Academy enrolls a unique student population. The children assigned to the Academy bring with them significant social/emotional disabilities that limit their ability to attend traditional schools. Of the three programs located at the Chestnut Street facility, only one the „long term‟ program – was the focus of the panel‟s inquiry, as the student

    performance information for these students is that which is reported for this school. Ninety seven percent of its students are classified as needing special education services that are defined in the student‟s Individualized Education Program (IEP). In recognition of its special population, the school identifies behavior as its primary focus, with low student reading levels as the second most important problem. As part of the planning process with the district, the school included an improvement objective for mathematics as well. However, while academic and behavioral issues are represented in the school‟s plan, the academic problems are derived from an inadequate source of information and, as a result, may or may not meet the needs of Springfield Academy students.

    The SIP developed by Springfield Academy with district support does define specific student performance objectives, citing modest gains in the Composite Performance Index (CPI) for each grade level as its targets. The plan closely follows the planning process guidelines in listing its student learning objectives, using language and terminology consistent with the MA Curriculum Frameworks. While these elements help make the plan a good starting point, the goals and objectives originate from a definition of learning gaps that may not be accurate because of the weaknesses in the sources of student performance information used to reveal those gaps. The school‟s data analysis suffers from two primary limitations: (1) lack of a variety of student

    performance information sources, and (2) small numbers of students at tested grade levels. In the first steps of the district planning process, the school references MCAS results and district mid-term and final exams as the source of data used in identifying core student learning needs. In

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 7

    Fall/Winter 2004-05

    interviews during the Panel visit, school leaders and staff members mentioned DRA (Diagnostic Reading Assessment), Read 180 assessments and other measures as additional sources of information to identify student learning needs. While these other sources were discussed, none of these data were analyzed in a way that would make them useful in the planning process. As such, their review was omitted from the written description of the school‟s analysis process.

    Complicating the problem of using few data sources for diagnosing student learning needs is the limited sample of students represented by each year‟s MCAS results. Grades 3, 4, 6 and 10 had fewer than 20 students tested in each of the previous two years, and grades 7 and 8 had less than 30. The item analysis described in the school‟s improvement plan is based on this small sample of students and may not be an accurate reflection of the generalizable trends in academic strength or weakness. In addition, the overwhelming majority of students in each tested group have special educational needs, predominantly social/emotional. Relying on MCAS as the primary source of information about student learning needs in this setting is unreliable. Changing student population is another factor limiting the quality of the data analysis underlying the Springfield Academy School Improvement Plan. The school reported to the Panelists that there is a high rate of transiency among its students, with an average enrollment of 165 days. It was not clear whether the MCAS item analysis used in the school‟s planning process included only those students who remain at the school or whether it included all students enrolled. Based on responses to questions about tracking students throughout the system, there appears to be limited use of the student information database to disaggregate MCAS information to explore the needs of continuing students or to examine whether learning gaps common to Springfield Academy mirror district-wide trends. Without a deeper exploration of the limited data, the school‟s identification of core student learning needs may not be an accurate assessment.

    During interviews, school leaders, staff members and special education supervisors did not refer to the learning needs in reading/literacy and mathematics described in the Individualized Education Programs (IEPs) of the 97 percent of Academy students classified with special educational needs. The SIP referred to the learning accommodations required by the IEP but not the defined content or skill gap for the students as the focus of instructional support. Since almost all students at this school have clearly defined educational program requirements that include individual learning goals, the panel was concerned that the improvement plan did not integrate specific learning needs identified on the IEPs with its generic goals and objectives. According to interviews with SIP team members, school leaders and district support personnel, this additional source of data on student strengths and weaknesses was not considered when developing the plan.

    In exploring the causes for the identified student weaknesses, the school had little (if any) information about classroom practices on which to base its conclusions about the reasons for student learning gaps. School leaders generally rely on informal visits to classrooms to learn about instructional practices. Formal evaluations are conducted periodically, but the standard form in use district-wide does not provide specific instructional information suitable to an analysis of the impact of instruction on student learning. Periodic walk-throughs with district personnel add somewhat to an understanding of instructional practice, but are too infrequent to offer sufficient insight school-wide.

    Overall, the Springfield Academy School Improvement Plan represents the beginning of a planning process that incorporates a new focus on student achievement for this alternative school.

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 8

    Fall/Winter 2004-05

    At this point, however, the analysis of student performance data and program quality information are limited for a variety of reasons. Until the school and its district support personnel probe more deeply and broaden the sources of data they examine, it is uncertain that they have accurately identified the specific learning gaps among their students and the reasons for poor student performance.

    B. Does the plan set out specific improvement objectives that are grounded in the

    school’s analysis of the reasons for poor student performance?

    The Springfield Academy School Improvement Plan sets out a collection of improvement objectives that are specific and aligned with the Massachusetts Curriculum Frameworks and that reference broad areas of weakness in language and mathematics. The improvement goals relate to the general need across the student population for support in reading, writing and problem solving. The goals vary in specificity and may not address specific needs of Springfield Academy students for the reasons cited in response to Question A, above.

    The Springfield Academy goals were developed from the template provided by the Massachusetts Department of Education (Goal Worksheet), which calculates improvement targets for the school. A sample student performance objective for grade 4 math demonstrates the specificity of these targets: “Increase the percent of students scoring in the proficient category from 0% (0 students out of 20 in 2004) to 5% (1 student out of 20) in 2005 and to 10% (2 students out of 20) in 2006.” The performance goals defined in the Springfield Academy plan

    are similarly specific and modest, requiring changes in only a few students per year to make progress toward the school‟s improvement targets.

    The student learning objectives are likewise specific and make use of the language of the Massachusetts Curriculum Frameworks: “All students will write with a clear focus, coherent organization and sufficient detail (Standard 19).” As a result of student performance weaknesses in open response and constructed response answers on the MCAS, the school‟s SIP team defined

    this learning objective as an important step toward meeting student academic needs. While the performance and learning objectives are clear and specific, they may not address the specific learning needs of Springfield Academy students because of the limitations of the variety of information sources used in the analysis of reasons for poor student performance and due to the reported transiency of the student population. For instance, writing appears as an important need for the recently tested students according to a review of MCAS items, but it may not correlate with the skills identified in IEPs specific to the students currently enrolled. These factors transiency and small tested population make it difficult to state with confidence that

    the improvement targets and activities are an accurate assessment of student needs. C. In order to accomplish each improvement objective, does the plan specify strategies

    that appear likely to lead to improved student results?

    If the strategies described in the school‟s “Action Plan” sections of its improvement plan are implemented as described, there is likely to be improved student learning. Because of the transiency of the student population, however, current measures of student achievement (the MCAS) are not likely to accurately describe student learning gains.

    The Springfield Academy School Improvement Plan Action Plan (Appendix 12) describes in

    detail the steps needed to implement the instructional changes defined in other sections of the

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 9

    Fall/Winter 2004-05

    plan. For example, to achieve the student learning objective cited above (writing), the action plan lists the overarching activity, “Full implementation of Collins Writing Program.” Beneath that heading, there are a number of subordinate steps that define what teachers need to do to fully implement the program drawn from the Collins instructional guide. (“Determine appropriate Focus Correction Areas (FCAs) for classes of students.”)

    In keeping with its stated mission and unique student population, the school defines an objective related to behavior: “All students will actively participate in a Unified Behavior Modification Plan (UBMP)/Level System to support them with engaging in daily academic lessons.” As for the academic objectives in ELA and mathematics, a detailed action plan provides a list of specific tasks teachers and staff members must undertake to achieve the objective and implement the UBMP.

    In each Action Plan in its improvement plan, Springfield Academy lists strategies that are built around recognized research-based programs (Collins Writing; Springfield District Reading Program) or that appear likely to lead to improved student results (UBMP). The key to predicating the likelihood of success, though, is the level of quality of implementation. In the Action Plan, the school defined “measures of implementation” that are vague and do not define a standard or criteria for quality. For example, full implementation of the Collins program will be monitored through “Lesson plans, observations, student portfolios.” These are useful tools to identify if teachers are following the implementation guidelines, but it is not clear from the plan or any other documentation provided what the content of the lesson plans should be or what specific indicators are expected to be seen during observations. Professional development to support implementation of Collins Writing is another worthy strategy, but the only measure of its effectiveness is the attendance list from professional development sessions and classroom observations. According to interviews, school leaders have not instituted any systematic, regular process for conducting the focused classroom observations implied by this measure to track the quality of implementation of the strategy. The plan includes tools for effective monitoring of classroom practice, but the Benchmarks Assessment Checklist has not been used by school leaders as designed.

    D. Are the school’s written improvement planning document (s) clear and specific

    enough to guide their implementation of planned improvement initiatives?

    Many of the structural components of a useful improvement plan are present in the Springfield Academy document, but a key component a clear description of how the plan will be

    implemented at the classroom level and how these changes will address the specific needs of Academy students remains unclear. The plan lacks a clear articulation of the logic involved in determining the academic needs of their students and the connection to the strategies selected to address student needs. Responses from school leaders, faculty and staff members varied widely in defining the core academic needs of students. Members of the school community indicated widespread belief that transiency and behavior limited the ability of the school to have a positive impact on student achievement.

    As indicated in Sections B and C, the components of the Springfield Academy School Improvement Plan are generally clear and specific. Student performance goals are stated in terms of measurable gains on MCAS. Student learning objectives define specific standards from the Massachusetts Curriculum Frameworks that are to be the focus of instruction. Instructional

     December 6, 2004 Massachusetts Department of Education

Panel Review Report Springfield Academy Page 10

    Fall/Winter 2004-05

    change objectives and action plans describe the steps to be taken by the faculty to enhance the educational program to better address learning gaps.

    Interviews with 38 staff members, including teachers and adjustment counselors, revealed varying degrees of understanding of their roles in the implementation of the improvement plan. Some identified attendance at professional development as their responsibility and others suggested specific academic program adoption as their primary contribution. When probed, teachers and counselors were unable to speak specifically about what actions or strategies they are expected to carry out that would represent the implementation of the plan‟s initiatives. In several focus groups, staff members echoed the principal‟s claim that implementing an academic improvement plan would be a “challenge” because of the severe disabilities exhibited by

    Springfield Academy students. An additional challenge will be identifying key priorities within the substantial document whose structure is unwieldy and whose organization is awkward. In almost all interview settings, staff members cited as their primary responsibility to first address the behavior problems that “prevent students from accessing the curriculum.” Concern and attention to the academic program was secondary to concern and attention to behavior issues. As defined by school leaders, staff members and district personnel, the goal of the school is to stabilize students so that they can be transferred to a less restrictive setting, ideally in their local school.

    E. Was the School Improvement Plan developed through a process that will support its

    successful implementation?

    The process through which the Springfield Academy Improvement Plan was produced should be sufficient to support its implementation, but there remain significant gaps in the process that limit the likelihood of its successful implementation. As indicated in the previous section of this report, teachers do not understand their roles in the execution of the plan and indicate disbelief that the plan can be implemented or that it will be effective even if implemented. Benchmarks are defined but there is no indication from teachers or administrators that they are being tracked, and there is no plan for regular periodic checks on the progress of the plan. At this point in time, communication of the plan‟s goals across the school community is spotty and implementation of the plan‟s strategies is nominal.

    The process for developing the Springfield Academy School Improvement Plan included gathering faculty in small groups to suggest areas of concern. Under the guidance of the district curriculum coordinators, members of the SIP team reviewed MCAS item analyses to identify student strengths and weaknesses. Drafts of the plan were shared with teachers at the beginning of the 04-05 school year at an extended day meeting, and teachers confirmed they were given opportunities to provide input. This process should be suitable for promoting widespread understanding and acceptance of the plan‟s goals, but several factors have lessened the success of the process. First, a large number of teachers left the school at the end of the last school year, so some of those involved early in the process are no longer at the school. The new staff members have limited context for the plan and are less familiar with the urgency of the academic problems identified through the school‟s analysis. Second, as noted above, faculty building-wide

    identify behavior rather than academics as their primary focus. For many of the new personnel,

     December 6, 2004 Massachusetts Department of Education

Report this document

For any questions or suggestions please email
cust-service@docsford.com