During last week's AuthorSpeak 2011 sponsored by Solution Tree, one of the lunch panel sessions included Dr. Thomas Guskey, renowned guru on educational assessment. I've attended sessions in the past that featured Dr. Guskey but I guess I never really heard him make a case against using the ACT (and SAT) in NCLB-forced high school assessments. As he claims, the assessments used by states such as Michigan are designed to measure academic achievement against state-adopted curriculum standards, however, the ACT and other tests like it are intended solely to compare and select potential college applicants. These two purposes are at odds with each other according to Dr. Guskey.
Assessments used for selection purposes, such as college entrance examinations like the ACT and SAT, are designed to be instructionally insensitive (Popham, 2007). That is, if a particular concept is taught well and, as a result, most students answer an assessment item related to that concept correctly, it no longer discriminates among students and is therefore eliminated from the assessment. These type of assessments maximize differences among students, thus facilitating the selection process.
...College entrance exams such as the ACT and SAT help colleges and universities decide whom to admit [but they] do not reflect any particular level of knowledge... rather where each student ranks in relation to others. Ranking makes the selection process easier.
Problems arise when a test designed for one purpose is used for another. ... tests like the ACT and SAT are labeled "instructionally insensitive." If instruction helps most students answer a question correctly, then that question is removed from the test, for it no longer serves its purpose. Even if the question asks about a vitally important concept, it no longer differentiates students and is eliminated.
This is why scores on selection tests are more strongly related to social and economic factors than are scores on competence tests. Aspects other than those influenced by instruction often account for the differences among students. It is also why it makes little sense to use a selection test like the ACT or SAT as a measure of the quality of instructional programs. Doing so would be analogous to using a ruler to measure a person's weight.
Having all students take a selection test such as the ACT or SAT may help some realize that they rank high enough to get into a college or university. That would be a good thing, especially for non-traditional students and those who come from disadvantaged backgrounds.
But to use the results of an "instructionally insensitive" selection test to assess the quality of instructional programs is educational sacrilege. No testing expert would agree to it -- and neither should any legislator or policy-maker.