I've been learning a lot about test bias lately thanks to David White, the founder of Testing for the Public. He's been coaching me on how to teach a prep-course for the GRE. Let me first say something about David White. He's an admirable iconoclast, a specialist in standardized testing with a law degree from Harvard, and he looks the part by resembling a mixture of Einstein and some guy from Haight/Ashbury. His insights into test bias have really put a dent in my beliefs in the usefulness and accuracy of standardized testing and should really tear down the institution of standaradized testing as it exists today.
The primary and most insidious problem of the Graduate Records Exam is that all questions are pretested for "consistency". This means questions tossed at test-takers in the "pretest" part of the exam (1 of 4 sections you'll have to take on the GRE). Your performance in the "pretest" doesn't affect your overall score, but it's there to test whether a question will be included in the scored part of a future exam. The thing test makers look for in these trial questions is consistency, the correlation between how well you do on the test and how well you do on the experimental question. Basically, this mechanism for introducing new questions perpetuate any bias that's in the test.
Let's take Armando who's from mexico. He might know a few words with Latin roots, say ferrous, that your average, white Joe Smith doesn't. Let's say he gets that question right on the pretest, he associates ferrous with ferrocarril and bingo - ferrous: iron containing. But, he does horribly on the GRE. Well, eventhough the question is legitimate, that question won't make it onto the GRE because it's inconsistent; upper-middle class, white Joe Smith got it wrong and he scored well on the "real" GRE. Hey, that's why when you're taking the test and you've narrowed it down to a few answer choices you're better off picking the one that's Anglo-Saxon-sounding. It's more likely to be right. Why? Because Joe Smith knows more Anglo-Saxon words. Sounds dumb, right? It did to me the first time I heard it, but as you do practice tests and take the real GRE test, it becomes clear that when you don't know the answer, picking the Anglo-Saxon word that sounds like grunts and groans is more likely to be right! Another tip: when anything relating to boats come up, that's the right answer. DOOR:ROOM::HATCH:HULL. Those are just a few examples of bias in standardized testing.
All this leads to the conclusion that standardized testing isn't used to find the smartest, most qualified people to fill the available spots in universities (I don't know what could). Its only pupose is to fill them with the same people that were accepted last year. It's not diversity or creativity they're looking for; it's strict conformity.
Comments (6)
All standardized tests are like this, the people who can afford to take classes on these tests (usually upper class students) will usually do better on the tests. The tests don't test your intelligence, it just shows how well you learned the tricks of the test. A very intelligent person from the ghetto could receive a lower score than a less intelligent person from a privileged background just because he did not have the opportunity to take a class on how to beat the test. These standardized tests are supposed to be constant across the board, but we all know that they aren't. The question is, what are we going to do about them? Shouldn't we have a standardized test to measure students across the board? But how do you make a test that isn't biased in any way?
Posted by lisa | October 2, 2002 7:36 AM
Posted on October 2, 2002 07:36
You have said it very accurately. The students who have the money to take the tests are the ones that are going to do the best. The students who do not have to work their way through college and have the luxury of time, and thus the luxury to study for the time, are going to do the best. The GRE, the MAT, all of the tests that are supposed to see if a student is properly prepared for graduate school, are really just testing to see if a student has practiced the test enough.
I know if I had not bought the MAT book and done the practice test over and over again there is no way in the world I would have had a chance on that test. As far as the GRE goes, if I was going to attempt that test, I would have to study for many hours, a luxury I do not have.
Money, in our world, does wonders for a person's academic standing. It's a shame. You can win scholarships, get grants and loans, but when it comes to the dreaded "Standardized Tests" the universities are still wearing blinders and using them as some kind of measuring point for intelligence.
Anyone who has been through a good undergraduate program knows that the ultimate measure of academic success is completing a high quality program with good grades. The world of undergraduates is tough and vigorous. Anyone can study for a test, especially if you have the money to pay for a review class. It takes mental stamina, emotional strength and academic success to make it through. Success in an undergraduate program should speak much more highly than any standardized test!
Posted by Bernice | January 29, 2003 5:17 PM
Posted on January 29, 2003 17:17
It seems the lot of this is plain excuse making. Sure, we can all see how inadequate standardized tests are for the results desired. But are they -biased-? No, grammar is grammar, chemistry is chemistry. I'm appauled at how many of us with "higher educations" have avoided the simple critical aspect of it all. Whether you are from "the ghetto" or Beverly Hills, "Bill don't know." is incorrect grammar. Change the general profile of the test, maybe, but if you can't simplify square roots, it isn't the test fault. It's your own, you should thirst to know these things. How can we expect to harbor such knowledge without such, through process of osmosis? Perhaps complaining until some sappy dean gives you your 25 bonus points for "cultural bias" will do the trick. Wake up...
Posted by Brett | February 21, 2003 12:40 PM
Posted on February 21, 2003 12:40
Very naive, Brett. As a graduate I had similar views. I've now had my eyes opened, having had to go back to education. Others on my course are intelligent, lively conversationalists but had fewer early educational opportunities. They are still unable to score well on these tests, despite their obvious ability. I still do score well, having had much practice as a child.
Posted by Helen | April 3, 2003 12:42 AM
Posted on April 3, 2003 00:42
The GRE, in spite of all its deficiencies, is the most equitable way to measure a student's potential in graduate school. I hear a lot of wining about how some economically challenged students are able to excel in the most stringent academic programs yet can't even do well on a simple standardized test because they either don't have time or the money to study for it properly. This argument is obviously flawed - because for one thing: how can anyone so fiscally marginalized, who thinks having the time to study is a luxury, do well in an academically rigorous program? Don't they test you in those schools and don't you have to study for them all the time? And more importantly, if these students can manage to maintain respectable grades in academically rigorous programs, which requires a lot of perseverence on their part, and since they obviously have ample leisure to study for their classes in order to do well, then why can't they study for the GRE with the same intensity and personal fervor that they lavished on their challenging classes? The current study on standardized testing in general, which I think is for the most part accurate, is that the affluent tend to do better than their penurious counterparts. What I don't agree is the interpretation therein that implies that there is any form of bias on the part of the tests, that explicitly and tendenciously disadvantages the poor, while concurrently conceding that the poor also tend to have inferior access to primary and secondary education. A better and more cogent conclusion drawn from this study, that the less privileged statistically receive lower grades on standardized tests as result of their inferior quality of basic education, is that instead of labeling the tests biased and the findings of the study contradictory, the tests accurately pinpointed the deficiencies and the biases of the American education system on race and economic status, which proves to be the primary exponent of score disparities on such tests as the GRE. There is nothing wrong with the GRE folks, it's the system at large that's biased. Easing up on standardized testing won't stop the prejudices in our institutions of higher learning.
Posted by Tony | June 25, 2003 8:44 PM
Posted on June 25, 2003 20:44
who gives a shit
Posted by jonny | February 5, 2004 3:00 PM
Posted on February 5, 2004 15:00