Differential Prediction/Impact in Course Placement for Ethnic and Gender Groups

ACT Rese&rcli Report Series 96.Differential Prediction/Impact in Course Placement for Ethnic and Gender GroupsJulie Noble Jill Crouse Matt Schulz For additional copies write:ACT Research Report Series P.O. Box 168Iowa City, Iowa 52243-0168© 1996 by ACT, Inc. All rights reserved. Differential Prediction/Impact in Course Placement for Ethnic and Gender GroupsJulie Noble Jill Crouse Matt Schulz AbstractThis research investigated course placement decisions for ethnic and gender groups. Success in four standard college freshman courses was predicted from ACT scores or high school subject area grade averages (SGA). The number of institutions varied from 13 to 50, depending on the course. Mean between-subgroup differences in estimated probability of success and three course placement validity indices were evaluated by gender or ethnic group within course type and institution. All statistics were summarized across institutions by course type and ethnic or gender group.The results of this study showed that, from a statistical perspective, both ACT scores and SGA slightly overpredict course success for blacks and males relative to whites and females. From a practical perspective, the differences between ethnic and gender groups were small. Research on differential prediction based on standardized test scores and high school grades (e.g., Sawyer, 1985; Young, 1994) has shown slight overprediction of the college GPAs of African-Americans relative to Caucasian-Americans, and of males relative to females. A limitation of these studies is that the criterion variables are either pooled subject area course grades (e.g., course grades from all English courses) or first- year GPA. Though generally more reliable than grades in specific courses, pooled grades or GPAs are less informative as measures of academic success in the first year in college. A second limitation is that the research does not consider or control for prior selection in admissions or course placem ent and resulting restriction of range problems (Linn, 1983). Finally, Linn (1994) states that such overprediction, although consistent across studies, may not be an indication of "bias," but of imperfect reliability. He suggests that the magnitude of group differences in prediction found in current research often do not represent "true" group differences.In course placem ent, the typical decision is whether a student should be placed into a standard-level course or into a lower-level course (e.g., standard freshman English vs. developmental English or intermediate algebra vs. elementary algebra). For this use of placem ent variables (e.g., test scores or high school grades), traditional statistics, such as R2 or %2 values, standard errors of estimate (SEE), or differences in linear regression slopes, appear less informative. A more meaningful approach is to determine how differential prediction affects the outcomes of placement or admissions decisions for specific subgroups. Sawyer (1993) developed an approach to placement validation thatDifferential Prediction/Impact in Course Placement for Ethnic and Gender Groups focusses on estimating the percentage of correct placement decisions made about an unselected group of students (students for whom a placement decision is to be made, referred to here as the "placement group"). The estimates are based on logistic regression models developed for students who completed a given course.This research investigated the differential impact of course placement decisions on male, female, African-American, and Caucasian-American students. Using Sawyer's approach would show, in practical terms, the implications of course placement decisions for these population subgroups. It would also correct for restriction of range problems identified by Linn (1984) by estimating the impact of course placement decisions on an unselected group of students (i.e., the placement group).Though research has shown that using test scores in combination with either high school grades, subject area grade averages (SGA), or high school GPA results in differential prediction, it has not compared the differential impact of using SGA in course placement with that of test scores. Therefore, test scores and SGA were used separately as predictor variables; the differential impact of course placement decisions based on SGA or on test scores was then compared.DataThe data for this study consisted of students' grades from over 80 institutions for 11 different college courses, ACT Assessment scores (in English and Mathematics), s