Validating the Use of ACT Assessment Scores and High School Grades for Remedial Course Placement in College

ACT Research Report Series89-4Validating the Use of ACT Assessment Scores and High School Grades for Remedial Course Placement in CollegeRichard SawyerAugust 1989 For additional copies write: ACT Research Report Series P.O. Box 168 Iowa City, Iowa 52243©1989 by The American College Testing Program. All rights reserved. VALIDATING THE USE OF ACT ASSESSMENT SCORES AND HIGH SCHOOL GRADESFOR REMEDIAL COURSE PLACEMENT IN COLLEGERichard SawyerThe author thanks Michael Kane for his advice on the issues discussed in th paper, and Mark. Houston, Jim Maxey, and Julie Noble for their comments earlier drafts. ABSTRACTAn important use of ACT Assessment test scores and self-reported high school grades is college f reshman course placement. An argument, based on the content fit between a coLlege course, the ACT Assessment tests, and students' high school course work., is described for justifying this use at particular institutions. A utiIity-based approach to quantifying the effectiveness of placement rules is then described, and its relationship to traditional predictive validity statistics, such as the multiple correlation and the standard error of estimate, is explained. VALIDATING THE USE OF ACT ASSESSMENT SCORES ANDHIGH SCHOOL GRADES FOR REMEDIAL COURSE PLACEMENT IN COLLEGEA typical and important use of ACT Assessment data is college freshman course placement, i.e., the matching of students with appropriate inst ruet ion. For example, students with low probability of success in a standard freshman English course might, on the basis of their ACT test scores and self-reported high school grades, be advised or required to enroll in a remedial English course. On the other hand, students with high probability of success in an accelerated English course might be encouraged to enroll in it. This paper considers the problem of validating placement procedures, that is, of determining their educational appropriateness.Cronbach (1988) argued that validating a test use is really evaluation, and must not only address the particular educational functions for which the test was designed, but must also consider the broader educational, political, and social consequences of using the test from the perspectives of different value systems. By this standard, validation is an enormous and never-ending task. This paper is concerned only with the traditional issue in Cronbach*s agenda, namely determining whether a test score serves the particulareducational function it was designed to serve (e.g., course placement), andwhether it does so in an economical way. For a discussion of procedures bywhich other aspects of placement systems can be evaluated, see Frisbie (1982).An impressive theoretical methodology, based on statistical decision theory, has been developed during the past twenty-five years for determining the effectiveness of selection systems. A principal goal of this paper is to interpret the decision theory methods developed by Cronbach and Gleser (1965) and by Petersen and Novick (1976) in the context of recent ideas about validation (Kane, 1987). Further, their decision models are extended to one 2that more easily addresses the concerns of an institution in measuring the effectiveness of its placement system. Methodological issues in quantifying system effectiveness are then examined, and are illustrated by an example.RemediationMany different techniques exist for matching college students with appropriate instruction. Willingham (1974) described in detail and classified various placement techniques that are based on test scores and other measures of academic ability. This paper focuses on only one particular placement function, remediation.At many postsecondary institutions, there are two levels of freshmancourses: a standard level course in which most freshmen enroll, and a lowerlevel course for students who are not academically prepared for the standardcourse. (At some institutions, there are "developmental" or "review" coursesin addition to the remedial course; in this paper, we consider only the single remedial lower-level course). In Willingham's classification of placement procedures, this instructional treatment is called "remediation" (Model 4).A placement device, such as a test, is but one component of a placementsystem. To be educationally effective, the placement system must have atleast all the following characteristics:a. There must be some way to ident