Collecting Postschool Outcome Data on Youth with Disabilities
Community of Practice (CoP) Teleconference Call
Date: Tuesday, September 27, 20005
Presenters: Eric Fox (efox@DAWNINC.ORG), New Jersey
Doris Jamison (djamison@MAIL.NYSED.GOV), New York
Bob Shepherd (shephers@POTSDAM.EDU), New York
Cinda Johnson (cinda@SEATTLEU.EDU), Washington Title: ―Using Postschool Outcome Data for Improvement‖ Participants:
Donna Aksamid Lynn Holdheidi Marianne Moore
Chun Allen Patty Johnson Amy Pleet
Ginger Blalock Mary Kampa Laura Rothenberger
Cindy Chen Millicent Kellner Gerry Teevens
Jane Falls Susan Loving David Test
Denny Hasko Steve Mishlove Deanne Unruh
NEW JERSEY (Eric Fox)
? Eric Fox, Transition Specialist for the New Jersey State Department of Education, presented
? The New Jersey Department of Education, Office of Special Programs, has a 3 year grant through the NJ
Centers for Independent Living (part of a statewide improvement grant). New Jersey is TAKING ACTION
―Promoting Self-Advocacy During Transition‖
? The project developed in part by looking at previous work, such as Wehmeyer & Palmer (Adult Outcomes
for students with Cognitive Disabilities Three-Years After High School: The Impact of Self Determination
[Michael Wehmeyer and Susan Palmer] University of Kansas Education and Training in Developmental
Disabilities 2003) and the NLTS.
? For example, the NLTS1 outcomes were weak, students were not thriving.
? Important components of the grant include: empowerment, self-advocacy, and overall personal confidence.
? There are over 400 Centers for Independent Living across the United States and 12 within New Jersey.
? The 4 core services are information & referral, advocacy, peer support, and individual life skills training.
? Two points to our grant: (1) information dissemination (e.g., for each county a Disability Resource Guide
has been created); and (2) self-advocacy training provided to students ages 14-21.
? There are about 100 self-advocacy curricula out there….we looked for commonalities and found a few
recurrent themes: (1) accessing community resources, (2) understanding personal learning styles, (3)
establishing and maintaining positive relationships, (4) identifying personal values & personal wellness, (5)
realistic goal setting, and (6) knowing one’s rights
? Goal is to go from reactive ―red-time‖ to a proactive mind-set ―green time.‖ What can I do for myself?
20% of families report that they are the primary person in developing their IEPs nearly 50% state that the
school has it already prepared
? What are these students advocating for? (1) Recreational opportunities, (2) Learning new skills, and (3)
? In turn, this has driven schools to pursue Community Based Instruction to a greater degree…with agency
collaboration or by utilizing internal personnel.
? The greatest challenge is that we cover students with all disabilities (multiple disabilities to mild learning
disability) who are college bound. Two separate tracks – very different needs.
? Fruits of our labor (success stories)
- creation of community based individualized employment program at high schools
- creation of self-advocacy clubs within schools
- creation of links systems for self-contained classes to meet other students
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 2
- creation of mentoring programs between LD and DD groups
- www.thinkcollege.net post-secondary programs for students with intellectual disabilities granting
certificates in self-determination Bellevue Community College in Bellevue, WA pioneer offering an
AA in essential studies
? Drawbacks – not enough capacity within social services
- Systems advocacy is needed
- PARENT INVOLVEMENT
- Alternative ways of thinking: Real Life Choices
- DDD – keeping young adults at home but providing funding
NEW YORK (Bob Shepherd and Doris Jamison)
? Bob Shepherd from SUNY-Potsdam presented; Doris Jamison from the NY State Education Department
could not be present on the call, but she provided written notes (see below)
? verview of NY’s postschool follow-up system (see below)
? The partners wanted the information to make local and state decisions.
? SUNY-Potsdam is working on a contract, so information is reported to Doris Jamison at the State
? Results are provided at statewide, aggregate levels as well as broken down by specific subpopulations.
? The information is reported internally to voc rehab and special ed (once they have examined the data, they
may ask the university for additional information / data runs)
? The data are presented to the NY Board of Regents, other state agencies (such as DD, mental health), and
the technical assistance networks, as well as being presented at state, regional, and national conferences.
? If a district is large enough, a district will also receive a report about their former students (and they will help
them work with the data).
? They have 9 years of data. In Bob’s opinion, without the data, transition would have faded from the picture
(e.g., as part of the transition network, several projects have come out of looking at the data, such as
CareerLink, an IEP guidance document from the state, etc.).
? 2 or 3 key performance indicators in NY relate to transition goals—can use the follow-up data to look at
successful outcomes for kids with disabilities and kids without disabilities
[Note: The following information was compiled from notes taken during the teleconference call and the
New York state methodology handout located on the Western Regional Resource Center Web site at
http://interact.uoregon.edu/wrrc/NWpass/stateinfo.htm#top and was previously reported in the Postschool Outcomes CoP call notes from 4/19/2005.]
Bob Shepherd Doris Jamison Statewide Coordinator Manager, Special Education St. Lawrence Lewis BOCES Policy Implementation 315-267-2718 NYSED VESID firstname.lastname@example.org 518-474-7566 email@example.com firstname.lastname@example.org
Which Students are Included?
? Have two sets of students: completers from 2000 and 2001. The class of 2000 serves as the beta-testing
? Did not include dropouts: a matter of research focus. For this purpose, why use up resources tracking
down dropouts when it is known that their outcomes will not be as good. This is a research vs. program
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 3
evaluation question; that is, the hypothesis is “Is completion of school with a certain credential a
sufficient level to transition successfully?” As we move to a compliance reason to gather this data, then we will need to include all students.
? Have a general education and a special education sample from each school.
? New York state has large range in district size: a couple of hundred (small rural) to over one million (New York City)
? Approximately 25,000 special education students exit each year in New York; however, no list of each student is currently kept, so NY used the Federal PD5 to estimate the proportion of exiters by year to the building level in all districts throughout the state
? Stratified random sample within each of 10 regions; 2000 sample from 74 school districts, 2001 sample from 141 school districts (see details below).
? Stratified cluster sampling was used to target 25% of students with disabilities who would be graduating from secondary education in June 2000. Computer analysis projected that a random sample of 166 schools from 10 geographic regions would yield a representative statewide sample of 4500 students with disabilities.
? Five of the regions are the Big Five Cities (New York City, Buffalo, Rochester, Syracuse, and Yonkers). The other regions are Western New York, Central New York, Northern New York, Capital District\Upper Hudson Valley, and the Lower Hudson Valley/Long Island.
? From these participating schools, 1 of every 35 general education seniors, sampled by randomly drawn last name initial, would yield 1500 general education students.
? For the beta group, recruitment with schools, their students and families initially yielded 89 schools willing to participate and identified 4300 possible seniors with disabilities and 1200 without disabilities. Following the process of obtaining parent and guardian consents for these minors to participate (which was required the first year, but not the second), the sample was formed from 74 school districts—1,019
seniors with disabilities and 352 seniors without disabilities. Providing some financial reimbursement for local districts to help cover survey related costs aided recruitment.
? To form the sample for the class of 2001, recruitment of schools from the sampling list continued through the 2000-2001 school year. This added additional school districts to the sample. Students from 141 districts completed 2001 Senior Surveys, including 2,386 seniors with disabilities (2,234 special education; 152 with 504 plans) and 956 seniors without disabilities from general education.
Strategies to Increase Response Rate
? Ask for relative’s address in Senior Exit Survey
? Use a phone interview rather than a mailed interview. (Abbreviated paper interviews are available. They are sent if requested, or where no phone survey is completed.)
? More often than every 18 months, send a letter stating "Return address service requested" which costs $.75. (Postal Service forwards mail up to 18 months. If mail cannot be delivered as addressed, the Post Office will provide address correction service, which allows the sender to find out the addressee's new address.)
? Have used Private Investigator/Bill Collector software to locate addresses and telephone numbers ? White page searches / DMV records
? Send a shorter, paper version of the survey if unable to reach student by telephone for interview (send this to a relative if there's no response from the student)
? Have 800 toll-free phone numbers for students to use in calling back for interview and a Web site ? Offered $15 to complete interview (offered $20 to complete it on the web, since there's no interviewer.) This increased participation for special education participants but not for general education.
When is Data Collected?
Seniors graduating in 2000 and 2001 were surveyed during their last month in high school. The senior survey
return rate in 2001 was the best at 53%. Interviews are being conducted at one, three and five years out of high
school. Milestones to date include:
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 4
1. Class of 2000 seniors from 74 schools surveyed at school exit included 1,019 special education and 352
general education students.
2. Class of 2001 seniors from 141 schools surveyed when they graduated included 2,234 special education
students, 152 students with 504 Plans, and 956 general education students.
3. Class of 2000 interviews one year after leaving high school included 920 special ed students and 234
general education students.
4. Class of 2001 interviews one year after leaving high school included 2267 special ed students and 1252
general education students.
Who Collects Data?
The State University of New York at Potsdam (SUNY Potsdam) with personnel from the local education
agency, St. Lawrence Lewis Board of Cooperative Educational Services (BOCES), is contracted to coordinate
all aspects of the project. This includes conducting the interviews through the on-campus calling center or
regional coordination teams. Regional interviewers use laptop computers containing the survey protocols and
enter data with each interview. Using outside interviewers provides the opportunity for more objective data;
however, it lends to a lower response rate than using local district staff to conduct interviews.
How are Costs Covered?
Approximately $75 per student for each follow-up cycle ($2.75 million 7-year follow up study), using IDEA
discretionary funds, Part B.
How is Data Used?
Both the school district and the SEA receive the reports. Data is used for monitoring, improvement, and
technical assistance/training. Survey data is also reported to other appropriate groups in the state.
MESSAGE FROM DORIS JAMISON, NEW YORK (who could not be on the call)
We have used the data to show that transition makes a difference in the lives of kids. This type of message has gotten attention of the powers-that-be, and gotten a priority for using resources for: maintaining the transition
technical assistance network; demonstration projects to engage ILCs, community rehabilitation providers and
Career Link projects; and to get policies approved that further transition (e.g., earlier onset for transition planning,
inclusion of transition needs IN the IEP PLP section and a Skills Achievement Profile for IEP students who
complete Career and Technical Education courses) among others. We have gotten support from the vocational
rehabilitation program in the state and see steady progress in numbers of youth made eligible and served, and have
signficantly increased postsecondary education participation.
We use a multiple-message dissemination process. Since every group sees the picture differently, our messages are styled for different audiences. We have done formal papers for Board of Regents discussion but
developed simple brochures and factsheets to educate the local level. Our Transition Coordination Sites, who lead
regional level stakeholder planning teams, use the data in planning sessions to help increase awareness of
schools/families/agencies to needs for change and help them develop local strategic plans for improvement.
An example of a fact sheet is attached, which is geared to local school administrators. Some links below give
Introduction to our current special education APR section on post-school transition. (The new one is being developed per the new IDEA.) http://www.vesid.nysed.gov/sedcar/apr/apr0304data/cluster5.htm
Body of the Transition section of the APR, outlining history of how we got here and what we see in the data. It also shows an integration of transition Key Performance Indicators in our State's strategic plan affecting both
special education and vocational rehabilitation. We'll need to tweak a couple of the KPI's to fit with new IDEA
Career and Technical Education Skills Achievement Profile, which grew out of using data to increase the awareness of the general education system regarding the career learning needs of our students aiming for IEP
diplomas and getting their support.
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 5
Local Level dissemination examples are in our transition web page toolbox.
Powerpoint slides overviewing the NYS Post School Outcomes Longitudinal Study and summarizing major
ways NYS has used data from the longitudinal study at the National PSO site.
Field guidance regarding sample IEP, including expectations for transition (needs to be updated re: IDEA
2004) to be incorporated IN the IEP rather than as an attachment.
Powerpoint presented summer, 2005 at national RSA transition conference re: NYS integration of VR and
special education. http://www.dtiassociates.com/RSAtransitions/presentations/RSA_final_wnotes.ppt
WASHINGTON (Cinda Johnson)
Using Post-school Data for Program Improvement
Cinda Johnson, Ed.D.
Center for Change in Transition Services
Washington State Post-School Data Research
? Long history of data collection
? First study in 1987
? Consistent statewide since 1998
? 2004 Study
? 80% of the School Districts in Washington State
? 78% of youth contacted
Washington State Post-school outcome data is connected to the anticipated outcome on the Individualized
Education Program (IEP)
? Post-secondary training as a goal
? Employment as a goal
? Agency connection as a goal
? Rate of Employment, post-school training and education, agency linkages.
? Executive Summary and Complete Report are provided to OSPI.
? OSPI and CCTS post report on websites.
? District reports, in comparison to state data, are provided to individual districts.
? Special education directors write goals from these outcomes for their LEA funds.
Focus groups, surveys, documents and follow up site visits were conducted with districts to determine what
positively impacts the use of post-school data for program improvement.
What formats are most useful for reporting data?
? Charts with summaries.
? Personalize the data.
―We need the stories‖
―We need the heart part.‖
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 6
? Examination of the data with colleagues.
―I can’t made a connection, not at all, without this group.‖
What personnel need to examine these data for program improvement?
? Special education personnel
? General education teachers
? Vocational education teachers
? Guidance counselors
? School board members
What program decisions were made?
? Disseminate the post-school data to community.
? Policy and practice
? Adult agency connections
? Graduation policy
? Transition continuum
? Curricular planning and development.
What have we learned?
? Buy in from districts is difficult.
? Entering data is time consuming, expensive and tedious.
? Last minute data connection is difficult.
? Involve students early and the graduates in the spring of their graduating year.
? Share the data with other agencies.
? Value the support of the OSPI.
? Buy in from districts can be developed.
? Center for Change in Transition Services
? Cinda Johnson, Ed.D.
? Amy Pleet asked Cinda Johnson about what locals need to help them use the data. Cinda responded that it
has taken them 10 years to do this (providing training, TA, and just talking to people about collecting the
data and looking at the data). Cinda said that they’ve found that in the districts that use the information,
teachers make the phone calls. In many districts, once the graduates are identified, each teacher only needs
to make 2-3 calls (and some districts offer release time to do this). In the districts that don’t use the data as
much, secretaries tend to make the calls. This year, WA identified the 10 best districts in the state in terms
of contact rate and how they use the data. They also identified the 10 worst districts and then offered them
time and money to help them figure out how this system/data can work for them.
Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 7
? Bob Shepherd said that in NY when staff were doing the surveys, they had several requests to make the
process more ―scientific‖ so they use outside interviewers. He thinks this may be useful for SPP reporting
(to have more objective data).
? NY has a lower contact rate, but a more objective system since teachers don’t make the calls; WA has a
higher contact rate, but a less objective system since teachers do make the calls.
? In NY, they’ve identified the ―transition gap‖ (from the outcome data). Are kids in special ed and gen ed
doing things after leaving/exiting school at about the same rates? To answer this, they needed to compare
special ed and gen ed. They found, in general, that more general ed kids were doing things than special ed
kids. The people who were doing very little/nothing after school tended to be primarily female, minority,
inner city, and had more significant disabilities. The data pointed out that they need to be provided better
help in making connections with agencies.
? NY asked, what is the transition gap, how do we reduce the gap, who’s in the gap?
? NY has developed 2-4 page fact sheets that help people use the data. NY tries to bring a variety of groups
together around the data, such as VR, special ed, Workforce Investment Board, etc.
? The engagement rate in WA is similar to the transition gap in NY. ? Dave Test asked Cinda whether they have published anything about the comparison of student IEP goals as
stated on the IEP and student outcomes (as determined by the WA follow-up information). Cinda said it’s
on their Web site at www.seattleu.edu/ccts. ? Millicent Kellner from NJ explained that she has been doing outcome studies for schools for people with
severe disabilities. She believes that very disabled students are ignored by large studies like the NLTS2. She
wondered whether other state postschool outcomes systems are missing some of the schools for kids with
severe disabilities (she said there are 12,000 kids in private approved schools in NJ alone). She encouraged
states not to forget about getting outcome data from schools for very disabled kids.
? Dave Test mentioned that NC has asked him to help them with their SPP. He would like to wait until a
national template on what data to collect is available from the National Post-School Outcomes Center. Jane
Falls said a draft version is available on their Web site at www.psocenter.org.
? Gerry Teevens asked about whether states obtain releases of information from students/parents to collect
postschool data. States collecting this data said they do not for a variety of reasons. Note that Gerry also
sent a note to the entire listserv hoping to receive responses to this issue from other states.