Are Value-Added Measures of High School Effectiveness Related to Students' Enrollment and Success in College?

A C T Re s eA R Ch Re p oR T se R i e s 2 0 1 6 ( 1 0 )Dina Bassiri, PhDAre Value-Added Measures of High School Effectiveness Related to Students’ Enrollment and Success in College? © 2016 by ACT, Inc. All rights reserved. MS488Dina Bassiri is a senior research scientist in the Statistical and Applied Research Department, specializing in educational outcomes research and student growth models.AcknowledgmentsThe author is grateful to Jeff Allen, Dick Buddin, Richard Sawyer, Jim Sconing, Teri Fisher, and Kurt Burkum for their review and helpful comments and suggestions on an earlier draft of this report. iiiContentsAbstract .................................................................. 1Introduction ............................................................... 1Report Organization ....................................................... 2No Child Left Behind and Every Student Succeeds Act ............................ 2Growth and Value-Added Models ............................................. 3The Validity of Accountability Systems ......................................... 5Research Question ........................................................ 6Data ...................................................................... 7School-level Data ......................................................... 7Student-level Data ........................................................ 8College Enrollment and Retention Data ....................................... 10College Course Grade Data ................................................ 11Value-Added Model ...................................................... 15Assessing the Predictive Strength of Value-added Scores on College Success ...... 17College Enrollment and College Retention ..................................... 18College Course Grades ................................................... 23Case Examples: Predictive Strength of Value-Added Measures of School Performance for Selected School Types ....................................... 27Discussion ............................................................... 28Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30References ............................................................... 31Appendix A ............................................................... 35Appendix B ............................................................... 36 ACT Research Report Are Value-Added Measures of High School Effectiveness Related to Students’ Enrollment and Success in College?iv 1AbstractOne outcome of the implementation of the No Child Left Behind Act of 2001 and its call for better accountability in public schools across the nation has been the use of student assessment data in measuring schools’ effectiveness. In general, inferences about schools’ effectiveness depend on the type of statistical model used to link student assessment results to schools. For example, characteristics beyond a school’s control (e.g., entering achievement level and socio-economic status of the students served by the school) can strongly influence simple proficiency rates. In contrast, measures derived from growth and value-added models potentially estimate school effects more accurately.This study investigated the predictive strength of value-added measures of high schools’ performance on their students’ enrollment and success in college. It is based on the data of 263,000 students who graduated in 2004 through 2009 from 1,119 high schools across the United States. The students had test scores from two time points (ACT Explore® in 8th grade and the ACT® college readiness assessment in 11th/12th grades).The findings indicate that value-added school effect estimates predict college enrollment and retention, as well as grades in first-year college courses in English/Language Arts, Mathematics, Natural Sciences, and Social Sciences, even after adjusting for student-level and other school-level characteristics. This study provides evidence that some high schools are much more successful than others at moving their students towards success in college. It does not, however, look at the potential determinants that make these schools more successful than others.IntroductionEducational accountability in the nation’s public schools has gained considerable attention over the last decade, primarily because it became the basis for rewarding or sanctioning teachers or schools. This is in large part due to the 2001 reauthorization of the federal Elementary and Secondary Education Act (ESEA) known as the No Child Left Behind Act (National Association of State Boards of Education, 2002) which provided a specific framework within which states must develop their educational accountability system. Much has been written about what works and what does not work to improve the delivery of education. Different types of statistical models for attributing student a