CoP call notes from 92705

By Shane Wagner,2014-05-07 20:56
11 views 0
CoP call notes from 92705

    Collecting Postschool Outcome Data on Youth with Disabilities

    Community of Practice (CoP) Teleconference Call

Date: Tuesday, September 27, 20005

    Presenters: Eric Fox (efox@DAWNINC.ORG), New Jersey

    Doris Jamison (djamison@MAIL.NYSED.GOV), New York

    Bob Shepherd (shephers@POTSDAM.EDU), New York

    Cinda Johnson (cinda@SEATTLEU.EDU), Washington Title: ―Using Postschool Outcome Data for Improvement‖ Participants:

    Donna Aksamid Lynn Holdheidi Marianne Moore

    Chun Allen Patty Johnson Amy Pleet

    Ginger Blalock Mary Kampa Laura Rothenberger

    Cindy Chen Millicent Kellner Gerry Teevens

    Jane Falls Susan Loving David Test

    Denny Hasko Steve Mishlove Deanne Unruh


    ? Eric Fox, Transition Specialist for the New Jersey State Department of Education, presented

    ? The New Jersey Department of Education, Office of Special Programs, has a 3 year grant through the NJ

    Centers for Independent Living (part of a statewide improvement grant). New Jersey is TAKING ACTION

    ―Promoting Self-Advocacy During Transition‖

    ? The project developed in part by looking at previous work, such as Wehmeyer & Palmer (Adult Outcomes

    for students with Cognitive Disabilities Three-Years After High School: The Impact of Self Determination

    [Michael Wehmeyer and Susan Palmer] University of Kansas Education and Training in Developmental

    Disabilities 2003) and the NLTS.

    ? For example, the NLTS1 outcomes were weak, students were not thriving.

    ? Important components of the grant include: empowerment, self-advocacy, and overall personal confidence.

    ? There are over 400 Centers for Independent Living across the United States and 12 within New Jersey.

    ? The 4 core services are information & referral, advocacy, peer support, and individual life skills training.

    ? Two points to our grant: (1) information dissemination (e.g., for each county a Disability Resource Guide

    has been created); and (2) self-advocacy training provided to students ages 14-21.

    ? There are about 100 self-advocacy curricula out there….we looked for commonalities and found a few

    recurrent themes: (1) accessing community resources, (2) understanding personal learning styles, (3)

    establishing and maintaining positive relationships, (4) identifying personal values & personal wellness, (5)

    realistic goal setting, and (6) knowing one’s rights

    ? Goal is to go from reactive ―red-time‖ to a proactive mind-set ―green time.‖ What can I do for myself?

    20% of families report that they are the primary person in developing their IEPs nearly 50% state that the

    school has it already prepared

    ? What are these students advocating for? (1) Recreational opportunities, (2) Learning new skills, and (3)

    Employment opportunities

    ? In turn, this has driven schools to pursue Community Based Instruction to a greater degree…with agency

    collaboration or by utilizing internal personnel.

    ? The greatest challenge is that we cover students with all disabilities (multiple disabilities to mild learning

    disability) who are college bound. Two separate tracks very different needs.

    ? Fruits of our labor (success stories)

    - creation of community based individualized employment program at high schools

    - creation of self-advocacy clubs within schools

    - creation of links systems for self-contained classes to meet other students

Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 2

    - creation of mentoring programs between LD and DD groups

    - post-secondary programs for students with intellectual disabilities granting

    certificates in self-determination Bellevue Community College in Bellevue, WA pioneer offering an

    AA in essential studies

    ? Drawbacks not enough capacity within social services

    - Systems advocacy is needed



    - Alternative ways of thinking: Real Life Choices

    - DDD keeping young adults at home but providing funding

    NEW YORK (Bob Shepherd and Doris Jamison)

    ? Bob Shepherd from SUNY-Potsdam presented; Doris Jamison from the NY State Education Department

    could not be present on the call, but she provided written notes (see below)

    ? verview of NY’s postschool follow-up system (see below)

    ? The partners wanted the information to make local and state decisions.

    ? SUNY-Potsdam is working on a contract, so information is reported to Doris Jamison at the State

    Education Department

    ? Results are provided at statewide, aggregate levels as well as broken down by specific subpopulations.

    ? The information is reported internally to voc rehab and special ed (once they have examined the data, they

    may ask the university for additional information / data runs)

    ? The data are presented to the NY Board of Regents, other state agencies (such as DD, mental health), and

    the technical assistance networks, as well as being presented at state, regional, and national conferences.

    ? If a district is large enough, a district will also receive a report about their former students (and they will help

    them work with the data).

    ? They have 9 years of data. In Bob’s opinion, without the data, transition would have faded from the picture

    (e.g., as part of the transition network, several projects have come out of looking at the data, such as

    CareerLink, an IEP guidance document from the state, etc.).

    ? 2 or 3 key performance indicators in NY relate to transition goalscan use the follow-up data to look at

    successful outcomes for kids with disabilities and kids without disabilities

[Note: The following information was compiled from notes taken during the teleconference call and the

    New York state methodology handout located on the Western Regional Resource Center Web site at and was previously reported in the Postschool Outcomes CoP call notes from 4/19/2005.]

Contact Information

    Bob Shepherd Doris Jamison Statewide Coordinator Manager, Special Education St. Lawrence Lewis BOCES Policy Implementation 315-267-2718 NYSED VESID 518-474-7566

Which Students are Included?

    ? Have two sets of students: completers from 2000 and 2001. The class of 2000 serves as the beta-testing


    ? Did not include dropouts: a matter of research focus. For this purpose, why use up resources tracking

    down dropouts when it is known that their outcomes will not be as good. This is a research vs. program

    Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 3

evaluation question; that is, the hypothesis is “Is completion of school with a certain credential a

    sufficient level to transition successfully?” As we move to a compliance reason to gather this data, then we will need to include all students.

    ? Have a general education and a special education sample from each school.

    Sampling Strategy

    ? New York state has large range in district size: a couple of hundred (small rural) to over one million (New York City)

    ? Approximately 25,000 special education students exit each year in New York; however, no list of each student is currently kept, so NY used the Federal PD5 to estimate the proportion of exiters by year to the building level in all districts throughout the state

    ? Stratified random sample within each of 10 regions; 2000 sample from 74 school districts, 2001 sample from 141 school districts (see details below).

    ? Stratified cluster sampling was used to target 25% of students with disabilities who would be graduating from secondary education in June 2000. Computer analysis projected that a random sample of 166 schools from 10 geographic regions would yield a representative statewide sample of 4500 students with disabilities.

    ? Five of the regions are the Big Five Cities (New York City, Buffalo, Rochester, Syracuse, and Yonkers). The other regions are Western New York, Central New York, Northern New York, Capital District\Upper Hudson Valley, and the Lower Hudson Valley/Long Island.

    ? From these participating schools, 1 of every 35 general education seniors, sampled by randomly drawn last name initial, would yield 1500 general education students.

    ? For the beta group, recruitment with schools, their students and families initially yielded 89 schools willing to participate and identified 4300 possible seniors with disabilities and 1200 without disabilities. Following the process of obtaining parent and guardian consents for these minors to participate (which was required the first year, but not the second), the sample was formed from 74 school districts1,019

    seniors with disabilities and 352 seniors without disabilities. Providing some financial reimbursement for local districts to help cover survey related costs aided recruitment.

    ? To form the sample for the class of 2001, recruitment of schools from the sampling list continued through the 2000-2001 school year. This added additional school districts to the sample. Students from 141 districts completed 2001 Senior Surveys, including 2,386 seniors with disabilities (2,234 special education; 152 with 504 plans) and 956 seniors without disabilities from general education.

    Strategies to Increase Response Rate

    ? Ask for relative’s address in Senior Exit Survey

    ? Use a phone interview rather than a mailed interview. (Abbreviated paper interviews are available. They are sent if requested, or where no phone survey is completed.)

    ? More often than every 18 months, send a letter stating "Return address service requested" which costs $.75. (Postal Service forwards mail up to 18 months. If mail cannot be delivered as addressed, the Post Office will provide address correction service, which allows the sender to find out the addressee's new address.)

    ? Have used Private Investigator/Bill Collector software to locate addresses and telephone numbers ? White page searches / DMV records

    ? Send a shorter, paper version of the survey if unable to reach student by telephone for interview (send this to a relative if there's no response from the student)

    ? Have 800 toll-free phone numbers for students to use in calling back for interview and a Web site ? Offered $15 to complete interview (offered $20 to complete it on the web, since there's no interviewer.) This increased participation for special education participants but not for general education.

    When is Data Collected?

    Seniors graduating in 2000 and 2001 were surveyed during their last month in high school. The senior survey

    return rate in 2001 was the best at 53%. Interviews are being conducted at one, three and five years out of high

    school. Milestones to date include:

Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 4

    1. Class of 2000 seniors from 74 schools surveyed at school exit included 1,019 special education and 352

    general education students.

    2. Class of 2001 seniors from 141 schools surveyed when they graduated included 2,234 special education

    students, 152 students with 504 Plans, and 956 general education students.

    3. Class of 2000 interviews one year after leaving high school included 920 special ed students and 234

    general education students.

    4. Class of 2001 interviews one year after leaving high school included 2267 special ed students and 1252

    general education students.

    Who Collects Data?

    The State University of New York at Potsdam (SUNY Potsdam) with personnel from the local education

    agency, St. Lawrence Lewis Board of Cooperative Educational Services (BOCES), is contracted to coordinate

    all aspects of the project. This includes conducting the interviews through the on-campus calling center or

    regional coordination teams. Regional interviewers use laptop computers containing the survey protocols and

    enter data with each interview. Using outside interviewers provides the opportunity for more objective data;

    however, it lends to a lower response rate than using local district staff to conduct interviews.

    How are Costs Covered?

    Approximately $75 per student for each follow-up cycle ($2.75 million 7-year follow up study), using IDEA

    discretionary funds, Part B.

    How is Data Used?

    Both the school district and the SEA receive the reports. Data is used for monitoring, improvement, and

    technical assistance/training. Survey data is also reported to other appropriate groups in the state.

MESSAGE FROM DORIS JAMISON, NEW YORK (who could not be on the call)

    We have used the data to show that transition makes a difference in the lives of kids. This type of message has gotten attention of the powers-that-be, and gotten a priority for using resources for: maintaining the transition

    technical assistance network; demonstration projects to engage ILCs, community rehabilitation providers and

    Career Link projects; and to get policies approved that further transition (e.g., earlier onset for transition planning,

    inclusion of transition needs IN the IEP PLP section and a Skills Achievement Profile for IEP students who

    complete Career and Technical Education courses) among others. We have gotten support from the vocational

    rehabilitation program in the state and see steady progress in numbers of youth made eligible and served, and have

    signficantly increased postsecondary education participation.

    We use a multiple-message dissemination process. Since every group sees the picture differently, our messages are styled for different audiences. We have done formal papers for Board of Regents discussion but

    developed simple brochures and factsheets to educate the local level. Our Transition Coordination Sites, who lead

    regional level stakeholder planning teams, use the data in planning sessions to help increase awareness of

    schools/families/agencies to needs for change and help them develop local strategic plans for improvement.

     An example of a fact sheet is attached, which is geared to local school administrators. Some links below give

    further information.

    Introduction to our current special education APR section on post-school transition. (The new one is being developed per the new IDEA.)

    Body of the Transition section of the APR, outlining history of how we got here and what we see in the data. It also shows an integration of transition Key Performance Indicators in our State's strategic plan affecting both

    special education and vocational rehabilitation. We'll need to tweak a couple of the KPI's to fit with new IDEA


    Career and Technical Education Skills Achievement Profile, which grew out of using data to increase the awareness of the general education system regarding the career learning needs of our students aiming for IEP

    diplomas and getting their support.

Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 5

    Local Level dissemination examples are in our transition web page toolbox.

    Powerpoint slides overviewing the NYS Post School Outcomes Longitudinal Study and summarizing major

    ways NYS has used data from the longitudinal study at the National PSO site.

    Field guidance regarding sample IEP, including expectations for transition (needs to be updated re: IDEA

    2004) to be incorporated IN the IEP rather than as an attachment.

    Powerpoint presented summer, 2005 at national RSA transition conference re: NYS integration of VR and

    special education.[1]_final_wnotes.ppt

    WASHINGTON (Cinda Johnson)

Using Post-school Data for Program Improvement

    Cinda Johnson, Ed.D.

    Center for Change in Transition Services

    Seattle University

    Seattle, Washington


    Washington State Post-School Data Research

    ? Long history of data collection

    ? First study in 1987

    ? Consistent statewide since 1998

    ? 2004 Study

    ? 80% of the School Districts in Washington State

    ? 78% of youth contacted


    Data Collected

    Washington State Post-school outcome data is connected to the anticipated outcome on the Individualized

    Education Program (IEP)

    ? Post-secondary training as a goal

    ? Employment as a goal

    ? Agency connection as a goal

    ? Rate of Employment, post-school training and education, agency linkages.


    Data Reported

    ? Executive Summary and Complete Report are provided to OSPI.

    ? OSPI and CCTS post report on websites.

    ? District reports, in comparison to state data, are provided to individual districts.

    ? Special education directors write goals from these outcomes for their LEA funds.



    Focus groups, surveys, documents and follow up site visits were conducted with districts to determine what

    positively impacts the use of post-school data for program improvement.


    What formats are most useful for reporting data?

    ? Charts with summaries.

    ? Personalize the data.

    ―We need the stories‖

    ―We need the heart part.‖

Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 6

    ? Examination of the data with colleagues.

    ―I can’t made a connection, not at all, without this group.‖


    What personnel need to examine these data for program improvement?

    ? Special education personnel

    ? Principals

    ? Vice-principals

    ? General education teachers

    ? Vocational education teachers

    ? Guidance counselors

    ? School board members


    What program decisions were made?

    ? Disseminate the post-school data to community.

    ? Policy and practice

    ? Adult agency connections

    ? Graduation policy

    ? Transition continuum

    ? Curricular planning and development.


    What have we learned?

    ? Buy in from districts is difficult.

    ? Entering data is time consuming, expensive and tedious.

    ? Last minute data connection is difficult.

    ? Involve students early and the graduates in the spring of their graduating year.

    ? Share the data with other agencies.

    ? Value the support of the OSPI.

    ? Buy in from districts can be developed.


    Contact Information

    ? Center for Change in Transition Services


    ? Cinda Johnson, Ed.D.



    ? Amy Pleet asked Cinda Johnson about what locals need to help them use the data. Cinda responded that it

    has taken them 10 years to do this (providing training, TA, and just talking to people about collecting the

    data and looking at the data). Cinda said that they’ve found that in the districts that use the information,

    teachers make the phone calls. In many districts, once the graduates are identified, each teacher only needs

    to make 2-3 calls (and some districts offer release time to do this). In the districts that don’t use the data as

    much, secretaries tend to make the calls. This year, WA identified the 10 best districts in the state in terms

    of contact rate and how they use the data. They also identified the 10 worst districts and then offered them

    time and money to help them figure out how this system/data can work for them.

    Postschool Outcomes CoP September27, 2005, Teleconference Call Notes p. 7

? Bob Shepherd said that in NY when staff were doing the surveys, they had several requests to make the

    process more ―scientific‖ so they use outside interviewers. He thinks this may be useful for SPP reporting

    (to have more objective data).

    ? NY has a lower contact rate, but a more objective system since teachers don’t make the calls; WA has a

    higher contact rate, but a less objective system since teachers do make the calls.

    ? In NY, they’ve identified the ―transition gap (from the outcome data). Are kids in special ed and gen ed

    doing things after leaving/exiting school at about the same rates? To answer this, they needed to compare

    special ed and gen ed. They found, in general, that more general ed kids were doing things than special ed

    kids. The people who were doing very little/nothing after school tended to be primarily female, minority,

    inner city, and had more significant disabilities. The data pointed out that they need to be provided better

    help in making connections with agencies.

    ? NY asked, what is the transition gap, how do we reduce the gap, who’s in the gap?

    ? NY has developed 2-4 page fact sheets that help people use the data. NY tries to bring a variety of groups

    together around the data, such as VR, special ed, Workforce Investment Board, etc.

    ? The engagement rate in WA is similar to the transition gap in NY. ? Dave Test asked Cinda whether they have published anything about the comparison of student IEP goals as

    stated on the IEP and student outcomes (as determined by the WA follow-up information). Cinda said it’s

    on their Web site at ? Millicent Kellner from NJ explained that she has been doing outcome studies for schools for people with

    severe disabilities. She believes that very disabled students are ignored by large studies like the NLTS2. She

    wondered whether other state postschool outcomes systems are missing some of the schools for kids with

    severe disabilities (she said there are 12,000 kids in private approved schools in NJ alone). She encouraged

    states not to forget about getting outcome data from schools for very disabled kids.

    ? Dave Test mentioned that NC has asked him to help them with their SPP. He would like to wait until a

    national template on what data to collect is available from the National Post-School Outcomes Center. Jane

    Falls said a draft version is available on their Web site at

    ? Gerry Teevens asked about whether states obtain releases of information from students/parents to collect

    postschool data. States collecting this data said they do not for a variety of reasons. Note that Gerry also

    sent a note to the entire listserv hoping to receive responses to this issue from other states.

Report this document

For any questions or suggestions please email