Education Reform and Testing

Total Spending Per Student

The New York Times published an editorial that defended student testing and made controversial statements, such as the following:

Groups that dislike standardized tests — and teacher accountability systems based on them — are blaming both for the cheating problem. But that’s like blaming the biopsy that turns up evidence of serious disease.

Yearly testing, required by the No Child Left Behind Act of 2002, is the cornerstone of the school reform effort. It allows districts to know which reforms are working and which are not. And it is the only way to ensure that poor and minority children are being taught to meet the same standards as their affluent and white counterparts.

To most people, this is a reasonable position. We spend a lot of money in this country educating students and we want to see that it is spent well, i.e. that it’s improving our substandard scores. However, some of the readers of the NYT disagree quite strongly. From the letters page:

Testing is fine as long as it has no high-stakes results; test scores along with other information help us know what reforms are working, as the editorial points out. But once these tests are linked to high-stakes consequences, like teacher pay, tenure or the closing of schools, the results are no longer dependable for diagnostic purposes, either for schools or individual students.

This phenomenon has more recently been borne out by the rapid state test score inflation that has occurred throughout the nation since No Child Left Behind was instituted, with the results on state exams increasingly diverging from the more reliable results of the no-stakes national exams, the National Assessment of Educational Progress.

Not surprisingly, the writer is the executive director of a group called “Class Size Matters”. Notice how she appears to be reasonable in the beginning (“Testing is fine as long as…”) but backtracks immediately by saying there must be nothing at stake. Given the name of the group she represents, this is not surprising. Judging from the name, she probably wants class sizes to go down, i.e. hire more teachers. But, what if hiring more teachers does nothing measurable to improve student performance? It’s easy to see why she opposes testing, because if hiring more teachers ends up wasting money, the reasonable thing to do would be to get rid of the additional teachers and use that money for something that does improve student performance. Of course, she would never allow that to happen.

Another writer has this to say:

Although the intense pressures on teachers don’t justify cheating, they certainly make it understandable. What would you do if your professional life and livelihood depended on students’ test scores, and the little demons wouldn’t come to school regularly, do their homework or even pay attention in class?

The only moral course of action left in these twisted times of test scores as king is nonviolent resistance on a grand scale: the faculties of entire schools and districts should teach what is best for their students and not waste time on test prep; refuse merit pay; let parents know that tests are not the true measure of their children’s knowledge; and stand together against all unfair dismissals of their colleagues.

Of course, the only moral course in response to her “nonviolent resistance” is to hold per capita pupil spending and teacher pay constant until she and her colleagues change their minds. Her dislike of testing is a little baffling, if you remove the self interest of the educators themselves. What is “the true measure of their children’s knowledge”? Is it quantifiable or can it be defined? If not, how is someone supposed to manage it? For that matter, why do teachers give tests at all? Teachers are presumably OK with giving their own tests; why not a test that is independently created and graded?

This is all very disheartening. As I’ve said in the past, education reform won’t mean anything until the system is uprooted and the likes of these are ignored.

FILED UNDER: Education, , , ,
Robert Prather
About Robert Prather
Robert Prather contributed over 80 posts to OTB between October 2005 and July 2013. He previously blogged at the now defunct Insults Unpunished. Follow him on Twitter @RobPrather.

Comments

  1. Michael Reynolds says:

    Testing is mostly nonsense. I nailed 99th percentile on the English SAT — fair enough — and an 85th percentile on the math SAT. I am functionally retarded in math. Trust me. 85th percentile because I’m good at beating tests.

    Smart kids, or kids who are good test-takers beat tests. Slow kids, or kids who freak out over tests fail tests. NCLB doesn’t raise IQ or make kids any more stress-resistant. So surprise: it doesn’t raise test scores.

    NCLB is not educational reform. No one is even close to realistic educational reform because what’s needed is a revolution. Unions aren’t the problem, teachers aren’t the problem, even money is not the problem, the problem is it’s not the 19th century. Our core assumptions about education are wrong. Our goals are wrong. Transferring data from teacher to student should no longer be the primary goal. Teaching independent learning, critical thinking, epistemology and doing it in a way that adapts to the talents of each student is, I believe, what’s required.

    Static classrooms in big buildings with kids arranged solely by age, being force-fed facts by teachers of varying levels of competence, all in the service of state-ordered tests that by virtue of their exaggerated importance actively discourage competent teaching, is in no way a system designed for the needs of kids or of society.

  2. Dave says:

    Teachers are presumably OK with giving their own tests; why not a test that is independently created and graded?

    Because standardized testing takes the power of curriculum away from teachers and places it into the hands of unaccountable bureaucrats who don’t know a thing about students or local communities. Why conservatives would so enthusiastically support the federalization of classroom curriculum is absolutely beyond me.

  3. john personna says:

    I wonder what it would look like if you took out energy costs.

    (I am all for “refactoring” education, and suspect that we could simply slash costs without seeing a real skills decline … but hey, I guess that experiment is happening right now, as schools reduce staff around the country. In my nephew’s school they are even reducing school days.)

  4. Michael,

    I’m not a big fan of NCLB, but it at least tries to measure accomplishment. How would we do so if your philosophy is embraced? What is an alternative to testing to determine performance? We can’t just forget testing unless we have an alternative because we need a way to measure effectiveness.

    My preferred solution, performance-based block grants to states, would allow for experimentation of the sort you describe, but we still need to measure success.

  5. Dave,

    So teachers would be able to determine their own criteria for success? Nice work if you can get it. Consider me unpersuaded. BTW, I don’t necessarily favor a national curriculum, but I do favor measuring progress.

  6. Michael,

    I do think the unions are a big part of the problem. Not surprisingly, the reforms they favor, such as smaller class sizes, tend to also benefit the unions.

  7. Jim says:

    Class size never established itself as a statistically valid variable in raising test scores.

    As can be inferred from the above graph, spending long ago decoupled from test scores as well, yet they keep rising faster. Obama just poured hundreds of billions into education in the stimulus bill.

    Most products now are custom designed to the customer even as the cost goes down, but we can’t get those two things happening in education, either in schools or universities. A public education now costs more than most private school alternatives, which do show statistically valid increases in test scores even for troubled students. Why do you think that is?

    I’ll give you a hint; health care (government pays over 50% of all health care and heavily regulated the rest before Obamacare), Medicaid, Social Security, Military, and education are the only sectors in society rising in price in the last 20 years.

    The system is beyond broken. And it is in general bankrupting us. An elected bureaucracy was never designed and is incapable of managing whole sectors of society; unless rationing and total lack of innovation is your version of management. It is what bureaucracies do without alternatives to prod them.

  8. john personna says:

    I loves the internets:

    Myth 1: Energy isn’t a major budget item for schools

    Fact: Not so. In many school districts, energy costs are second only
    to salaries, exceeding the cost of supplies and books. Nationally,
    K-12 schools spend more than $6 billion a year on energy and,
    according to the U.S. Department of Energy, at least a quarter of
    that could be saved through smarter energy management. Energy
    improvements could cut the nation’s school bill by $1.5 billion
    each year.

    The View from Grandview, Missouri:

    In the Grandview school district’s 1998/99 budget,
    utilities represented the largest area of expenditure
    after personnel. No other single category in the
    operating budget commanded as much money.
    While figures may vary from district to district —
    depending on size, number of students and facilities,
    State and local reporting requirements, and other
    factors—many districts, like Grandview, spend
    more on utilities than they do on textbooks.

    http://ase.org/uploaded_files/greenschools/School%20Energy%20Guidebook_9-04.pdf

  9. just me says:

    Personally I don’t mind the testing, although I am not sure everything with regards to school funding should ride on test outcomes, but I don’t have the problem with there being a stake and a consequence of some sort. I do think the 100% goal-including SPED set in NCLB is ridiculous because there are always going to be kids who don’t test well, who don’t know the information-even though they may have made tons of gains they may have not made enough-and of course kids who don’t care and decide they are just going to color little circles on their tests sheets.

    I think my real issue with the testing is that it compares this years third graders to next years third graders, when I think the test should be measuring knowledge gained in the same group from third grade to fourth grade. Comparing one kids progress to their own progress I think would tell more than comparing two completely different grades. The technology is there to get this information, the technology already follows teachers, transience and other factors related to SPED and SES.

    And there are the complaints that teachers will teach to the test, but at least with elementary school testing there isn’t anything on the tests that isn’t going to be taught in the curriculum anyway.

  10. Dave Schuler says:

    We’re educating different students than we were 40 years ago. I don’t just mean that trivially in the sense that the high school kids of 1970 are now in their fifties but that kids who would have been warehoused 40 years ago are receiving serious attention now. That reflects in higher costs.

    It’s also worth mentioning that healthcare costs are a serious component of personnel costs.

  11. just me says:

    It’s also worth mentioning that healthcare costs are a serious component of personnel costs.

    And this isn’t just for the teachers but other personnel. And teachers aren’t the only unionized district employees in most places. The custodian, paraprofessionals and in some cases food service are also unionized and often negotiate for and receive good health and other benefits.

    Transportation is also a big cost. Some of the districts that have tried 4 day weeks instead of 5 day weeks save an extra day of transportation in addition to costs for salaries and buildings.

  12. Michael Reynolds says:

    I’m not a big fan of NCLB, but it at least tries to measure accomplishment.

    It tests test-taking skills.

    Beyond that It tests the ability of teachers to teach to the test, which actively narrows the curriculum. Let’s say we know the test will have a section on the Civil War. We also know it won’t cover nullification or Dred Scott at one end and won’t cover Reconstruction at the other end. The test actively discourages teaching context. So if a classroom discussion leads naturally in the direction of exploring greater context the smart teacher will guide students away from that broader discussion back to the test questions.

    This is exactly the opposite of what we should be doing. Because the goal of education in the Google era can no longer be the imperfect transfer of data from indifferently-educated teacher or 20 year-old textbook to student. Data is essentially free, ubiquitous and untethered to time and place. Kids beyond, say, 4th grade should be learning a lot more about critical thinking and analysis and epistemology because they live in an environment of free, ubiquitous data. Data is a given. Understanding data, contextualizing it, knowing how we know whether data is useful or not, should be the larger goal of education in a Google-infused environment.

    But why on earth would a teacher take time away from hustling the NCLB test to teach context? Why would they take time to discuss how we know what we know? How much time will they spend teaching kids to use search engines and to understand the results?

    NCLB is making the classic mistake of fighting the last war. In the process it makes us less well-armed for the current war.

  13. TangoMan says:

    It tests test-taking skills.

    Too bad for your hypothesis that the SAT I & SAT II have greater predictive validity in measuring success in the first year of university than GPA. If all that tests actually measured were “test taking skills” then we should expect the GPA, which is more heavily weighted with subjective teacher assessment of a student’s skills, to have greater predictive validity.

    The test actively discourages teaching context. So if a classroom discussion leads naturally in the direction of exploring greater context the smart teacher will guide students away from that broader discussion back to the test questions.

    Nice hypothesis, now prove it. You’ve concocted a “jut so” story that perfectly fits with how you want this issue to be but the problem is that without a test there is no objective method of measuring student mastery of the material that’s been taught. Lot’s of students are capable of following a lecture as it’s unfolding and letting the material taught lead them to other material and then walking out of the classroom not having understood the lesson. You see this a lot in math classes where the teacher is explaining a procedure and a student is following the logic of what is going on, as it’s going on, and then when it comes time to do the homework they can’t recreate the logic or remember the procedures. Just because a lesson has been taught doesn’t mean that a lesson has been learned. The way to determine that a lesson has been learned is to test the student.

    Kids beyond, say, 4th grade should be learning a lot more about critical thinking and analysis and epistemology because they live in an environment of free, ubiquitous data.

    Let’s start with you first and see how that goes before we get too ambitious in our goals and try to teach critical thinking to 4th grade kids.

    Understanding data, contextualizing it, knowing how we know whether data is useful or not, should be the larger goal of education in a Google-infused environment.

    Critical thinking is built on two principal components, IQ is the hardware and skills and information that we’ve learned is the software. What you’re asking for is primarily a function of intelligence rather than a function of learned methods and tricks.

    NCLB is not educational reform. No one is even close to realistic educational reform because what’s needed is a revolution. Unions aren’t the problem, teachers aren’t the problem, even money is not the problem, the problem is it’s not the 19th century. Our core assumptions about education are wrong. Our goals are wrong. Transferring data from teacher to student should no longer be the primary goal. Teaching independent learning, critical thinking, epistemology and doing it in a way that adapts to the talents of each student is, I believe, what’s required.

    Unions are indeed a problem, as are dim teachers. The core assumptions of education and NCLB are not the problem, though I agree with you that the rigid institutional character of public education could use a revolution. However, the goals are still sound. Before we can expect independent and critical thinking a person needs to have mastery of fundamental knowledge. It’s a waste of time trying to get students to independently discover Newton’s Laws because a.) they’re not Newton and b.) the time spent by 12 years olds trying to replicate Newton’s insights about the physical world are better spent standing on his shoulders and benefiting from his unique insight and then moving onto the shoulders of some other intellectual giant and benefiting from the knowledge that he discovered. This way the kids develop a base of knowledge upon which they can leverage any critical thinking that they may do.

    Secondly, we’re not seeing any debilitating effects from what you seem to think is a misdirection of educational mission when we look at well educated children and when we look at international comparisons. You’re arguing for the effectiveness of this revolutionary approach based on your dreams, impressions and how you think the results would unfold and what you’re not doing is demonstrating your conclusions by pointing to some international efforts which are doing as you propose and delivering outstanding results. In fact, when we see outstanding results from the international data we often find systems that have rejected the progressive notions of education that have infected our system and instead we see much more focus on content mastery in these successful systems.

  14. Dave,

    Most of the warehousing you’re referring to ended decades ago. I agree that we are now teaching children who wouldn’t have been taught in the past, and have said as much myself, but how many decades of negative gains do we need to get past that?

  15. Michael,

    There’s a solution to the “teaching to the test” you’re referring to. For one thing, the test down’t just have to cover the Civil War, it could test the Civil War, what lead up to the Civil War and the aftermath. You’re tinkering with definitions and calling it context.

    Also, testing them on their ability to regurgitate facts is the least important kind of testing. Verbal skills and math are far more important and we’re not doing well on those either, to put it mildly.

  16. Dave says:

    So teachers would be able to determine their own criteria for success? Nice work if you can get it. Consider me unpersuaded. BTW, I don’t necessarily favor a national curriculum, but I do favor measuring progress.

    Teacher effectiveness should be measured by attendance rates, graduation rates and by peer, student, faculty and community review. Centralized standardized testing simply mandates curriculum, forcing teachers to teach to the test, wrestling control away from teachers and local school boards, the very people who actually know a thing or two about kids and the communities they live in.

    I don’t want my kid’s teacher looking at a past New York Regents exam and pulling out the nine questions on the European Renaissance to force feed her students. I want her working with our kids, with parents and the community, to create a school environment and curriculum that works for them and us, not for some fuddy-dud bureaucrat in Albany or Washington who thinks he knows what’s best for us and our kids.

  17. TangoMan,

    We’re in agreement. Well said.

  18. Dave (not Schuler),

    Look at the graph above. No amount of community review is going to change the fact that we’ve been spending money like crazy and getting nothing for it. Progress is measurable and we’ve been failing by every measure while spending tons of money doing so.

  19. john personna says:

    My dad was a teacher and administrator. Later, as a second career, my mom was a school secretary. The worked in a few schools between them. My second-hand opinion is that parents and community norms will make or break education. When my mom’s school tipped asian that did more to improve education (across the board) than anything else. They had the same staff, the same teachers, the same money, but suddenly parents who seriously cared.

    (In an amusing detail, the staff didn’t really get the shift until Teacher Day snuck up on them. They had never heard of such a thing.)

    Sure, test or not. Do what you can to make the national framework functional. But it will always be the parents that get hormework done, and demand the grades. And it’s parents who are spending extra on Kumon (where did that start?) and (sometimes too extreme) SAT preparation. Some of the parents that is, usually flocked together in districts with “good schools.”

  20. Dave (not Schuler) again,

    Also, attendance rates and graduation rates (given the incentives to move people theough the system) tell us nothing about what they know, only that they were there. This might be acceptable for a day care but is not acceptable for an institution that imparts knowledge.

  21. john,

    It’s hard to disagree with anything you said. At least testing will provide the data needed to focus whatever attention we need on getting parents involved.

  22. john personna says:

    (I don’t know why everyone assumes a graph ex-energy would look as bad. I documented it for you. Energy costs in education are huge, and energy costs have exceeded general inflation by a wide margin since 1970.)

  23. john personna says:

    Testing helps parents understand where they are, sure. Little tests do that too though, “your child reads at the X grade level,” or “your kid does math at the Y grade level.”

  24. Dave says:

    Look at the graph above. No amount of community review is going to change the fact that we’ve been spending money like crazy and getting nothing for it. Progress is measurable and we’ve been failing by every measure while spending tons of money doing so.

    So having the federal government writing classroom curriculum is the solution? Aren’t you the guy who suggested abolishing the Dept of Education?

    Progress is measurable, but content-oriented standardized testing written by bureaucrats is no way to do so. If we want to save cash and increase outcomes, there’s two quick things to do: bump class sizes up to 35 students a piece and end blanket teacher tenure, replacing it with a performance-based system of guaranteed job security. That way we can dump teachers who aren’t making the cut, and end this ridiculous “last one in, first one fired” system of teacher layoffs.

    But the performance criteria needs to be set by local communities, not Washington, and it can’t be based on student mastery of a federal curriculum, which is all standardized testing does. It’s graduation rates, attendance rates, community review. Leave curriculum up to local communities. A hatred of teachers unions shouldn’t turn every conservative in Washington into Ted Kennedy. Federalizing school curriculum tears apart communities and damages our kids’ education.

  25. john personna says:

    Well, maybe those guys at the energy site were a little hyperbolic. I found that Elkhart County schools “saved $223,767, or 18.8 percent, compared with those three months in 2009.” From that a yearly energy budget of $4.8 million. On another page I find their yearly total budget to be $132 million. So, not such a huge percentage, even if it is “#2.”

    http://www.indianaeconomicdigest.net/main.asp?SectionID=31&SubSectionID=62&ArticleID=54745

    http://www.elkhart.k12.in.us/content.php?id=331

  26. Trumwill says:

    Michael, the main problem I have with your saying is that once we’ve taught them to think for themselves and all that… we need to be able to measure progress. If what you’re proposing works better than the alternative, I think that would be great and I think that as a student I would have found it more interesting, but it could also turn into a sort of mess where nobody actually learns anything and there’s no way to measure what skills are and are not being learned.

    BTW, my standardized testing experiences were opposition of yours. I failed the reading portion of our standardized test in the 7th grade. So in the 8th grade I had to take remedial reading and the teacher absolutely, positively could not believe that I was in there. Then, at the end of 8th grade year, I went and failed the test all over again. I was making straight A’s by that point and they were seriously considering holding me back for a year. Or rather, the rules said they had to hold me back and they had to find a way to prevent that from happening.

    However, with these stories is the realization that if there is a problem it is with the tests used and not the existence of tests in the first place. I’m pretty flexible on how we go about the testing… but I just have trouble with the notion that they should not exist in the first place. At the very least, in places that are not tested, I would want it to be because the community explictly opted out of them (directly, not through the ed board, and vote on it in regular intervals) or I would want parents to have the option to take them out of a school that doesn’t test and put them into a school that does.

  27. Michael Reynolds says:

    TangoBrimelow:

    Yes, of course you love tests. You need tests to be terribly valid to support your “scientific racist” mission. Talking to you is as pointless as talking to a Scientologist or a Jehovah’s Witness or one of your friends in the Aryan movement. All discussion with you will lead back to your obsession with race. You don’t actually think at all in any real sense of the word: you take anything at hand and bend it to your racist mission.

    And really, what choice do you have now? You burned your career down around your ears by pursuing your obsession. Now you’re just a tired, irrelevant old fool fighting battles no one but you and a few meth-cookers in Idaho give a damn about. It’s 2010, Brimelow. And you’re just an embarrassment.

  28. Trumwill says:

    BTW, my standardized testing experiences were opposition of yours.

    Oy, it’s not hard to see why I would fail a reading test. This sort of thing always happens when I bite around the edges of a sentence and then fail to proofread it afterwards. Change “were oppositon of yours” to “slightly different from yours”…

  29. G.A.Phillips says:

    ***Michael, the main problem I have with your saying is that once we’ve taught them to think for themselves and all that… we need to be able to measure progress.*** we could let them post comments on this blog:)

  30. Michael Reynolds says:

    Various:

    Yes, tests are very good for either reassuring or scaring parents. But again at their very best what is it they measure? Did an SAT that put me in the 85th percentile measure my math knowledge? Of course not. I dropped out of school after completing 10th grade. If the test validly measured my knowledge, I’d have failed.

    And that is in a hard science with very specific answers. One has to guess that results for English, History and other less rigorous test-ready subjects would be less reliable.

    Of course kids still need facts. But I think what people aren’t getting is that facts are largely stored outside the brain. Most of us don’t memorize phone numbers, we store them outside of our biological memory. Increasingly we do the same with many other types of data. And kids now are born into an environment where Google is very nearly an internal rather than external reality. Soon it will be integrated fully. Kids are growing into a world where Google will always be on. Always.

    It’s as if in our day we always had our textbooks with us, and had all the answers highlighted and could open the book whenever we liked. Even that analogy doesn’t touch on how deep this change is.

    Given that kids will have access to ALL DATA, ALL THE TIME, EVERYWHERE, and given that this is different than at any previous time in history, we should be figuring out how to adjust. Our entire approach to school: 30 kids, 1 teacher, in a class for 8 hours makes no sense. It could just as easily be 100 kids in 100 homes with 1 teacher for 2 hours. Or it could be twelve kids in 10 different places with 1 teacher at midnight. Or 1,000 kids with 1,000 curricula spread around in schools, coffee shops, homes, airplanes, with no teachers.

    There is nothing magical about a building called school and a one-size-fits-all curriculum enforced by one-size-fits-all tests.

    The insistence on the physical school, and the existing model, creates an insistence on a single curriculum based entirely on age. But kids don’t learn by age, they learn by ability, they learn in fits and starts, they learn things in ways best suited to their own needs, at times best-suited to their biological clocks. For the first time in history we have the capacity to adapt teaching to the specific needs of individuals, rather than enforcing a rigid, mass-produced, industrial-age approach.

  31. Dave (not Schuler),

    I never said we must have a national curriculum, nor am I a fan of NCLB. The debate is about testing. One way or another, testing will play a role in this. If the budget for the DoE is broken into performance based block grants for the states, testing will still be needed to measure performance.

    Besides, your local focus is overwrought. Are you telling me that math is different from one city to the other? How about chemistry? There will need to be testing at the state level — and approved by the federal government — for the block grants to work.

  32. Michael,

    WTF are you talking about with TangoMan? Is there something I’m missing? You didn’t address his point and called him a racist.

  33. G.A.Phillips says:

    What if we got rid of public schools and teachers unions all together so every child could be home schooled or go to a private one?

    I would love to see the test scores of these kids from what ever manner of tests you would give them.

  34. G.A.,

    That seems like a bad idea. They would probably do well in spelling contests but suck at math and science, particularly if their parents are creationists.

  35. Michael Reynolds says:

    Robert:

    Long history there. I’m pretty sure TangoMan is a dude named Peter Brimelow. His next play, when left to his own devices, is to start posting blocs of text the purpose of which is to prove (to his own satisfaction) that blacks are inferior to whites.

  36. Michael,

    I did not know that. I’ll remain neutral in this little spat since I’m uninformed.

  37. G.A.Phillips says:

    ***That seems like a bad idea. They would probably do well in spelling contests but suck at math and science, particularly if their parents are creationists.***

    Think of all the money we would save in taxes alone, and all the new real jobs and real tax revanue we would create with a huge influx of private schools and Internet teachers.

    Home schooling does not mean Christian or parents only.

    The possibilities are limitless.

    And dude please, do you what to put like a 1 billion atheist schooled kids against say a 100 creationist schooled kids in math and science?

  38. GA,

    And dude please, do you what to put like a 1 billion atheist schooled kids against say a 100 creationist schooled kids in math and science?

    Yes, I would put ten against your 100, because many of your creationists would be like this moron who thinks water from the earth created craters on the moon.

  39. TangoMan says:

    Robert,

    I did not know that. I’ll remain neutral in this little spat since I’m uninformed

    Why are you calling him Micheal? I’m pretty sure that that fellow is David Duke because I noticed that “Micheal Reynolds” and Duke both used the world “but” in a sentence of their writing. I think that we’d all agree that that clue is air-tight.

    Reynolds has been a joke for a long time and this Brimelow accusation of his just compounds his idiocy. Whenever he gets trapped in a debate he pulls the racist card. You noticed that he ignored the substance of my comment and pulled his get out of jail card instead.

  40. Michael Reynolds says:

    Tango:
    The difference being that I deny being David Duke. You haven’t yet denied being Brimelow. And of course I use my real name, and tell people the name I write under, and I have no problem telling people who I am and what I do. My picture is all over the internet. Look, here I am right here:

    http://en.wikipedia.org/wiki/Michael_Grant_(young_adult_author)

    also here:

    http://www.alblit.com/blog/

    and here:

    http://sidewaysmencken.blogspot.com/

  41. TangoMan says:

    Did an SAT that put me in the 85th percentile measure my math knowledge? Of course not.
    If it didn’t measure your math knowledge then what did it measure, your knowledge of geography? If you believe that the test simply measured your skill at taking tests then I’m sure that we could devise a multiple choice test on quantum mechanics (which you presumably know nothing about) and you could deploy your awesome test taking skills to come out, again, at the 85th percentile without knowing anything about quantum mechanics. Then you could visit us on this blog and crow “Did that quantum mechanics multiple choice test that put me in the 85th percentile measure my knowledge of quantum mechanics? Of course not.”

    But I think what people aren’t getting is that facts are largely stored outside the brain.

    What you don’t seem to get is that people need facts as the foundation for critical thinking. For a kid who wants to be a mechanic, or an automotive engineer, if they don’t understand the physical forces at work within an engine then they can’t perform proper diagnosis (mechanic) nor can they innovate more efficient designs (engineer) because they’ll be no better than rote monkeys, looking up facts but not understanding the principles behind the facts.

    The way to scaffold up to understanding of general principles is to use logic. Logic in a vacuum absent facts is pretty useless. However, once facts are laid out, then a student can induce or deduce to principle. This applies to areas outside of science too. A student of history can see recurring behaviors (historical facts) which generally lead to a particular outcome. Being in possession of facts when more facts are introduced can illuminate a principle or pattern that is at work. The fact that facts can be accessed via Google servers doesn’t at all help the student who is unaware of the facts. To quote the inestimable Donald Rumsfeld:[quote]As we know,
    There are known knowns.
    There are things we know we know.
    We also know
    There are known unknowns.
    That is to say
    We know there are some things
    We do not know.
    But there are also unknown unknowns,
    The ones we don’t know
    We don’t know. [/quote]

    Google is mostly useful for the known knowns and the known unknowns. A student isn’t going to be able to engage in critical thinking or come to an understanding of principle if there exist facts that to the student are unknown unknowns. In other words, the student doesn’t even know to look on Google for a particular set of facts, facts which may be crucial to developing an understanding of some principle or to extending our understanding of a principle. The design of an educational curriculum tries to cover the basis on what experts believe are key facts and principles necessary to understand the world we live in and to function ably within it. The actual long term retention of facts, once they’ve been presented and mastery has been demonstrated, is mostly irrelevant to understanding the world and our ability to function within it. Plenty of people have likely forgotten the specific rules of grammar but they’ve internalized the lessons and so no longer need to be able to deconstruct a sentence. Plenty of people have likely forgotten the minutia of the War of Independence but they’ve internalized the principles of what was at stake and they can apply those principles to contemporary affairs when they read of oppressed peoples and revolutions. The way they came to understand these principles is through the introduction of facts and from facts a pattern developed. The facts can be, are most often are, forgotten, but the principle that was extracted usually sticks with people. It’s at this point that Google can come in handy, when people need to be reminded of the facts, then the intertubes are handy as a refresher source.

    Our entire approach to school: 30 kids, 1 teacher, in a class for 8 hours makes no sense.
    Any yet the brilliant minds which developed the intertubes are all likely products of this very system. The system makes a lot of sense, it’s just that the system isn’t perfect. You’re making perfect the enemy of the good and you’re proposing a revolution towards some pie in the sky system that’s never been, dare I say it, tested.

    There is nothing magical about a building called school and a one-size-fits-all curriculum enforced by one-size-fits-all tests.

    Of course there is nothing magical about the schooling model we have just as there is nothing magical about the internal combustion engine. We could implement a different model of schooling just like we could implement a vanadium redox engine system for all passenger cars. The reason we’ve settled on things as they are is because they tend to be the most efficient and cost effective methods available. I’m sure that 1:1 tutoring for each student would very likely produce achievement gains but the marginal utility gained is likely not worth the marginal cost.

    But kids don’t learn by age, they learn by ability, they learn in fits and starts, they learn things in ways best suited to their own needs, at times best-suited to their biological clocks.

    Education is a complex endeavor, so while it’s true that ability, rather than age, is the key factor underlying the learning process, it’s also true that kids of a similar age are undergoing the same (within fairly tight bounds) maturation and development process, and again, the hardware is maturing. If we exclude the tails of a normal distribution and we compare student achievement against age, we’re quite likely to find that age is a pretty good proxy for ability in that we’re not going to be seeing 30% of the 12 years olds performing to the ability standards of 18 year olds. There simply isn’t THAT MUCH variance in ability by age group.

    Secondly, much of education is a social endeavor rather than a strictly individual endeavor that is tailored to each students “own needs.” There is quite a bit of research which speaks to the importance of peer relationships to personality development and if we divorce children away from their peer networks and isolate them at 1,000 different points, with each child on a computer learning away at their own pace to meet their own needs, then this creates a problematic wrinkle with respect to peer relationships. Schools, as warehouses for the young, make life easier for parents in that they can free up a large portion of their day and not be responsible for the supervision of their children but if the child was educated at home, on their computer, working in isolation at their own pace, then that child would require parental supervision. Again, this speaks to the social benefits that arise from congregating kids into a common building.

  42. Michael,

    I’m not a fan of vdare and didn’t know who Peter Brimelow was until a little while ago. I’m pro-immigration so that site has nothing to offer me. Even if TangoMan is this guy, he has you beat on the substance on this issue, in my estimation.

  43. just me says:

    Beyond that It tests the ability of teachers to teach to the test, which actively narrows the curriculum.

    I mostly think this is a bogus argument against tests.

    The reality is that the tests do indeed test knowledge in a given subject area and some of the tests, at least the one given here has both multiple choice and short answer writing questions. In certain grades there is also a long answer writing test. There isn’t anything on the tests given that students don’t need to know.

    There is nothing in NCLB that says a teacher isn’t allowed to teach outside the test, but general the tests look for information that students need to know in order to demonstrate basic subject knowledge.

    I do think having everything ride on one test and eventually demanding 100% proficiency is unrealistic.

    I also really like the MAPS tests our school gives the kids. Rather than kids taking a standardized test where everyone gets the same answers, each child gets an individualized test (it is done on the computer) based on questions they get right or wrong. A right answer prompts a harder question and a wrong answer prompts a easier answer. A child’s progress is easily monitored and it is actually pretty easy for a teacher to note where knowledge gaps are, and work to teach that specific child what they are missing. There isn’t anyway for a teacher to “teach to the test” and a teacher has some really good data that is very specific to work with, when instructing students.

  44. Michael Reynolds says:

    There is nothing in NCLB that says a teacher isn’t allowed to teach outside the test, but general the tests look for information that students need to know in order to demonstrate basic subject knowledge.

    Of course there is. A teacher is graded on how well her students perform on a test. Why would she teach anything else?

  45. G.A.Phillips says:

    ***Yes, I would put ten against your 100, because many of your creationists would be like this moron who thinks water from the earth created craters on the moon.***

    ,This is what you think of most creationists lol!

    But I’ll bet this guy don’t think we came from earth worms or hot magical lava water, or that creatures just leap forward like the X-men cause all his crap fabricated evidence for evolution has been been proven to be false time and time and time again and they have no other explanation.

    Why ignore my proposal and start bringing up knuckleheads?. It was very simple and logical.

  46. Michael Reynolds says:

    Brimelow:

    Noting by the way that you have once again not denied that’s who you are.

    If it didn’t measure your math knowledge then what did it measure, your knowledge of geography?

    In effect it measures a particular type of IQ. I also beat the biology supplementary SAT test. I’d never taken biology. But I do well with language generally and was fluent in French. French gives you Latin roots, Latin roots gives you biology test. So the biology test tested my language skills.

    In terms of beating tests it’s absurd to pretend there aren’t skills and abilities that give you an edge, regardless of your knowledge. How much of an edge? I don’t know. But if you’re confident rather than sweaty, and can triage the questions, can pull apart words, can listen for the off-notes in questions, can quickly grasp context you end up with a sizable advantage.

    Simple tricks and a bit of an IQ edge lets you turn a multiple choice test into a true/false test by eliminating obvious outliers in the answers. If you have the speed and the confidence, the IQ and the language skills, you can move quickly through a multiple choice test answering the easy stuff, eliminating obviously wrong, or likely wrong answers, contrasting similar questions to arrive at a likely answer.

    Could I beat a multiple choice quantum physics test? I don’t know. But I beat the math SAT despite having attended no math class beyond 10th grade geometry and having gained a courtesy C in that only because my teacher, poor bastard, knew my Iowa score and didn’t have the nerve to flunk me. Also beat the biology test. And of course the English test despite the fact that — and I am not making this up — I don’t know what a participle is. I took a girlfriend’s college psych test and got an A for her.

    I understand why you would need to deny the fallibility of tests. Without tests you and Sailer and the rest of your tired old crew would have nothing. Tests are sort of your religious text, your Koran or Talmud. And here I am saying that you basically got nothing.

    By the way, even you should see the idiocy of claiming that SAT tests “predict” college performance. Yes: testing well in high school absolutely predicts that you will test well in college. Um . . . duh.

    One last point: my 13 year-old reads music quite well. He would score well on a test. IIRC BB King does not read music. He would flunk. Which of the two do you think is the better musician?

  47. TangoMan says:

    But if you’re confident rather than sweaty, and can triage the questions, can pull apart words, can listen for the off-notes in questions, can quickly grasp context you end up with a sizable advantage.

    Why do you denigrate these analytic skills and isolate their usefulness only to the task of taking tests? Also for these skills to be useful they must have been developed from a foundation of facts. These skills didn’t arise from thin air. “Pulling words apart” implies that one has had sufficient exposure to many words in order to be able to derive some general principles. The intelligence that is required to develop the knowledge and then to derive principles from that knowledge can also be used in other endeavors and in other academic tasks. What you’re describing is not some peculiar party trick with limited utility in the real world.

    Simple tricks and a bit of an IQ edge lets you turn a multiple choice test into a true/false test by eliminating obvious outliers in the answers. If you have the speed and the confidence, the IQ and the language skills, you can move quickly through a multiple choice test answering the easy stuff, eliminating obviously wrong, or likely wrong answers, contrasting similar questions to arrive at a likely answer.

    The tests are, you know, actually looking for IQ and language skills, so you having them is probably more relevant to the results you obtain than the party tricks you’ve learned on how to improve your test taking skills. Those party tricks require the test taker to have the ability to spot patterns, to discern errors in the choices, etc.

    Tests are sort of your religious text, your Koran or Talmud. And here I am saying that you basically got nothing. f

    My, my, you certainly are full of yourself, aren’t you. Look, the fact that David Dukes is saying that tests are irrelevant doesn’t make it so. Your proclamation here is meaningless. What matters is the external validity of tests and when we actually see things like health outcomes, income, and ability to more accurately target Patriot missiles all positively correlated with test performance then, despite your hot air denunciations, we know that the tests are actually measuring what they are designed to measure.

    By the way, even you should see the idiocy of claiming that SAT tests “predict” college performance. Yes: testing well in high school absolutely predicts that you will test well in college. Um . . . duh.

    The SATI & SATII are objective tests while first year university GPA is comprised of subjective measures (essays in class, pop quizes, finals, term papers).

  48. just me says:

    One last point: my 13 year-old reads music quite well. He would score well on a test. IIRC BB King does not read music. He would flunk. Which of the two do you think is the better musician?

    But a written music test is designed to test knowledge of music theory not whether one is a good/poor/excellent musician.

    There is sound reason to teach music theory, but a lack of music theory doesn’t equate to being a poor musician.

    My daughter took a guitar class at her high school this past year. A portion of the class taught theory, another portion of the class taught various scales, and another chords and melodies. The final test involved a measure of all three components.

    Of course there is. A teacher is graded on how well her students perform on a test. Why would she teach anything else?

    Well I work in a school, we don’t really care what is on the test specifically, but then what we teach is knowledge that every kid needs to know anyway. Which is really my point. What is specifically on a test that the students don’t need to know?

  49. Michael Reynolds says:

    Brimelow:

    The tests are, you know, actually looking for IQ and language skills, so you having them is probably more relevant to the results you obtain than the party tricks you’ve learned on how to improve your test taking skills. Those party tricks require the test taker to have the ability to spot patterns, to discern errors in the choices, etc.

    Right. In other words, IQ is really helpful in beating any test.

    I appreciate you agreeing with me that tests test test-taking ability. Yes, it’s essentially IQ, or more specifically linguistic IQ. Which is not what school tests purport to do. Let’s recall that NCLB is intended to test knowledge, not IQ. And teachers as well as students are graded on their ability to advance the ball.

    If NCLB is essentially just testing and re-testing IQ — my theory, with which you apparently agree now — then the process is a waste of everyone’s time and money. So far as we know no amount of teaching moves the needle very much on IQ. Given the flatlining of NCLB testing it supports my position that all we are testing is an essentially static test-taking ability.

    Would it be rude of me to say, game, set and match?

    Now, why should I suspect you have not really giving a rat’s ass about facts?

    http://anepigone.blogspot.com/2006/04/diversity-and-economic-disparity.html

    Tangoman:

    Good job on the analysis but if I can make a suggestion – you open the analysis to criticism that it is only capturing a discrimination effect, ie, blacks are discriminated against therefore their income is lower due to fewer opportunities. The criticism will be that it isn’t race that you’re measuring but discrimination.

    To counter this you need to find data that can serve as proxies for IQ, discrimination minimizing environments, etc. I’d look at things like rates of interracial marriage being a proxy for tolerance, maybe the per capita rate of EEOC racial discrimination complaints controlled for minority proportion of the population. Further, to proxy IQ I’d look at SAT results broken down by race, HS graduation rate, etc.

    Controlling for other factors will weaken the charge that you’re simply measuring the effects of a discriminatory environment on racial minorities. You could also weaken that charge by focusing on Asian performance compared to white performance.

    You know what that looks like to me? That looks like you advising one of your VDare buddies how to conceal his central motivation. It looks to me like a racist counseling a fellow racist in the fine art of bamboozling the suckers.

    You know, for a guy who worships the white man’s IQ, yours seems to have some limits. The other day you rather stupidly endorsed lesbian eugenics. And now you’ve effectively conceded that NCLB testing is just an endless retesting of the same IQ.

    One of the patterns we smart dudes look for is motive. Or call it presupposition. It works like gravity and distorts thought processes like yours. It’s how you end up arguing that the world would be a better place if only lesbians had children. It’s how you end up conceding my point while trying to refute it.

  50. Michael Reynolds says:

    Just Me:

    The problem comes from deciding what’s on the test, what it tests, and what should be done with the results.

    As you can see above, I believe tests test test-taking ability. They’re a sort of an intellectual closed loop, self-reaffirming. It’s not that there is absolutely no connection between learning and test results, rather that a combination of intellectual speed and efficiency, the confidence that comes from that, the effortless acquisition of background data not directly relevant to the subject but useful just the same (my French skills allowing me to beat a biology test, for example) mean that the tests are testing IQ or in other words, test-taking ability.

    Which would explain why teaching seems to have no great effect on test results. Smart kids will beat the test with a small amount of data, slower kids with the same exact amount of data will do more poorly. Think of it this way: two kids, one highly intelligent, one less so. Both are given a math test. Both know only a single math fact: 2+2=4. Same exact knowledge. Who do you think does better on the test? The smart kid. Because the smart kid can deduce that if 2+2=4 then 4/2 = 2. And 2 x 2 = 4.

    That’s the IQ edge.

    Now, set up a regime in which we have a school with 100% high IQ kids, and another school with only 10% high IQ kids. Give them identical teachers, identical resources. Then test them and weigh the tests against each other. Who wins? The high IQ school. The low IQ school ends up being labeled a failing school, while the property values around the high IQ school go up.

    As for the effect on curricula, it’s not that the knowledge in any given curriculum is useless, it’s that taking the entire set of students of a given age, and giving them the identical curriculum, serves the interests of the system, not the student.

    Let’s go with another example. Mozart was raised by his father who fed him a steady diet of music. Mozart was born with the talent, his father nurtured it. Would Mozart have been better off with a more rounded education? Should he have spent more time on sports and calculus and Latin verbs?

    Time spent pursuing that broader curriculum might have deprived the world of Mozart.

    Set aside Mozart and take average Joe who is never going to Stanford but might do very well as a mechanic. In fact he might become a prodigy at maintaining engines. Who knows? But if we conclude that we had best ignore Joe’s potential in favor of pursuing a curriculum designed for the convenience of the school, we are depriving the world of a great mechanic in favor of a mediocre generalist.

  51. just me says:

    As for the effect on curricula, it’s not that the knowledge in any given curriculum is useless, it’s that taking the entire set of students of a given age, and giving them the identical curriculum, serves the interests of the system, not the student.

    Although this isn’t really happening right now-at least not where I work. Differentiation is the name of the game where I work, where instruction, student work and activities are geared towards that student’s weaknesses or strengths. The curriculum isn’t identical, the eventual student outcomes are identical, but where a student has mastered that outcome we move on to the next skill level.

    I won’t disagree that a student with a high IQ is going to do better on a test and have better test taking skills in general, but the students themselves aren’t sitting there not learning anything either-low IQ or high IQ.

    I think in general a standardized test can be a good measure of learned knowledge, although I think comparing one group of peers this year to the previous years kids in the same grade doesn’t really provide useful data. What they should be comparing is a student’s 3rd grade scores to their scores from the previous year and measuring progress from one year to the next. That would indicate far more about teacher quality, curriculum quality and student’s learning.

    As i said earlier I think MAPS is a really good way to measure individual student progress as well-especially since it identifies very specific areas of strength and weakness.

  52. TangoMan says:

    I appreciate you agreeing with me that tests test test-taking ability.

    Where did I do that? You really need to work on your reading comprehension skills.

    Yes, it’s essentially IQ, or more specifically linguistic IQ.

    I find it interesting to note that you are a firm believer in IQ for usually you anti-testing folks also reject the notion of IQ. I’m surprised that your creationist tendencies don’t extend that far.

    Let’s recall that NCLB is intended to test knowledge, not IQ

    These two concepts, knowledge and IQ, are not completely separate. If you have the intelligence to reason your way to a correct answer on a test in a situation where you lack the specific knowledge that would provide the answer to the test question then you can do the same in real life when presented with a situation that calls for some specific knowledge that you don’t have. Your ability to recognize principles and patterns, to induce and deduce, is built on a foundation of knowledge and can be used to extend that knowledge.

    It doesn’t matter on the test whether you knew the answer because you had memorized it from a classroom lecture or whether you reasoned your way to the correct answer – what counts is that you knew the correct answer. When you provide the correct answer on the test you’ve demonstrated that you have mastery of the knowledge.

    If NCLB is essentially just testing and re-testing IQ — my theory, with which you apparently agree now — then the process is a waste of everyone’s time and money.

    Work on your reading comprehension dude, you’re embarrassing yourself. Clearly I don’t agree with you and clearly the testing is not a waste of everyone’s time and money. Reforms and improvements cannot be effective, or even directed, if everyone is stumbling around in the dark with regards to knowing what efforts are needed, who needs help, who doesn’t need help, and how schools, districts, and states compare across the metrics.

    The problem with NCLB testing is that it allows states to control the test design and this is resulting in a lowering of difficulty in order to increase the proficiency rate of students.

    You know what that looks like to me? That looks like you advising one of your VDare buddies how to conceal his central motivation. It looks to me like a racist counseling a fellow racist in the fine art of bamboozling the suckers.

    How is he concealing his central motivation? He clearly states that the analysis shows “. . economic disparity is absolutely correlated with race.” His analysis stands heads and shoulders above most blog postings on the internet. He’s formulating a question, seeking out data, analyzing the data and reaching a conclusion. I’m simply pointing out to him that he can make his conclusion stronger by accounting for a confounding effect, that critics will claim that economic disparity is correlated with race BECAUSE of discrimination. There’s no bamboozling here, it’s all straightforward. Again dude, take a remedial class on reading comprehension.

    Now, set up a regime in which we have a school with 100% high IQ kids, and another school with only 10% high IQ kids. Give them identical teachers, identical resources. Then test them and weigh the tests against each other. Who wins? The high IQ school. The low IQ school ends up being labeled a failing school, while the property values around the high IQ school go up.

    HAHAHA! Maybe you’re not such a creationist after all, you just like to pretend that you’re one. It’s refreshing to see you admit that IQ is important, that IQ varies between individuals, that IQ correlates to socioeconomic class, and that the distribution of student IQ is not uniform across schools.

    Secondly, we seem to agree that the goals of NCLB are never going to be achieved in that over here in the real world where IQ is the pink elephant in the room schools will never achieve 100% proficiency. The goals of NCLB are directed towards closing the Achievement Gap and NCLB is premised on a creationist model of reality where it is believed that simply tweaking methods and course design will result in a huge blooming of artificially suppressed talent in our students. The only point of disagreement you and I have left is on the usefulness of testing as a means of measuring reality. I think that tests are the best available method of doing so and you believe that teachers can seance with their students in order to determine their ability and knowledge levels.

  53. just me says:

    Secondly, we seem to agree that the goals of NCLB are never going to be achieved in that over here in the real world where IQ is the pink elephant in the room schools will never achieve 100% proficiency.

    I think this is very true. It isn’t so much that setting a goal for number of students proficient in basic knowledge areas is bad, but a 100% goal including the SPED population is unrealistic and unachievable no matter how much money is spent or how dedicated and fantastic the teacher.

    i do think the motivation behind the 100% is to keep schools from not even trying with their SPED populations and other populations that tend to have low performance on tests and low achievement in general.

    Rather than a 100% of students taking the test need to get a certain defined result, I would rather see a goal where 100% of the students make at least a minimum defined amount of progress. Doing this wouldn’t be that difficult and there are already tests that do this (the MAPS test I mentioned earlier individualizes reading, math, language arts and science and measures progress at the student level). Using this type of test, and defining expected progress would be a more effective way of testing students and measuring progress and honestly it would also be a way of holding teachers accountable, because the data is easily and immediately accessible by the teacher.

    Basically some sort of independently created test is really one of the few ways to measure whether something is working or not. My issue with NCLB is it compares apples to oranged (this years 3rd graders to next years 3rd graders etc) rather than apples to apples (one child’s scores this year compared to his scores the next year).

  54. Michael Reynolds says:

    Brimelow:

    Dude, now you’re just blustering trying to dig your way out of the hole you dug. You’re wasting my time.

  55. Michael Reynolds says:

    Just Me:

    You’re talking about individualizing the ways in which a set curriculum is taught. I haven’t seen that in my kid’s schools — in fact I pulled my son out of his alleged “gifted” school and now un-school him because they were wasting his time.

    I’m going further than suggesting individual modalities for teaching the same thing to the same kids at the same age. I think technology is offering us the power to individualize curricula as well as methods of teaching. I think it offers us the chance to abandon the school house where kids are assembled for a fixed number of hours on a set number of days.

    I watch my son learn the lessons I insist on: Algebra 2, Spanish, reading Shakespeare and more recent fiction, a bit of light physics, game theory, anthropology and I compare even that very free-wheeling curriculum to the way he learns his own chosen curriculum — computer languages, design, photography — and I have to tell you that his way is infinitely better. It is entirely self-motivated, runs in whichever direction suits his needs and is amazingly efficient.

    And in the process of learning things his way he picks up a very impressive amount of unrelated knowledge along the way. There are few better lessons in practical epistemology for example than digging through the history of changes in a Wikipedia article. (If that won’t teach you a degree of skepticism nothing will.) I didn’t “send” him there to learn that, he came across it while looking for something else.

    In contrast my daughter has dyslexia. Regular schools? Had nothing. Threw up their hands. Lindamood-Bell moved her two grade levels in two months of intense, (and expensive) one-on-one. Two grade levels in two months.

    So just in my own family I have two kids for whom regular schools and regular curricula are not just useless but actively impede learning by taking up time better spent on actual learning.

    And what is so necessary about the standard curriculum? The schools are still teaching cursive. Why? The schools are laboring to exclude the internet. Why? Why in God’s name are middle school kids schlepping 30 pounds of dead tree books around while their smart phones are banned? They’re teaching chunk method writing, a method that every professional writer on planet earth will tell you is stupid and actively harmful to writing. They teach history as dictated by political compromise and levels of math that are absolutely useless to at least 90% of students.

    Finally — and then I’ll end this rant — why are schools piling hours and hours of rote and repetitive homework onto kids who need sleep and play? To hit some magic number on some irrelevant politically-inspired test on a curriculum that has almost nothing to do with the lives kids lead? This is madness. It has nothing to do with kids, and everything to do with inertia, intellectual stagnation, parental ambitions and political partisanship.

  56. Grewgills says:

    But I beat the math SAT despite having attended no math class beyond 10th grade geometry

    The SAT didn’t include any math beyond 10th grade math and provided all formulas. If your algebra was reasonably solid and you were able to eliminate bad answers, work backwards from the multiple choice answers, and recognize the questions you did not know the answers to then scoring in the 85th percentile is entirely predictable.

    Michael I think that you over estimate the broadness of application of your particular circumstance and so underestimate the effectiveness of standardized testing.

    The tests are, you know, actually looking for IQ and language skills, so you having them is probably more relevant to the results you obtain than the party tricks you’ve learned on how to improve your test taking skills.

    I have taught those party tricks and the average results among my students was between 150 and 200 point increase in score on the 2 part SAT. Test taking tricks are definitely part of what is tested, in some cases a very large part of what is tested.
    In my experience teaching SAT prep and various other math and science courses in high school and community college there is a pretty strong correlation between performance on standardized test questions and in performance on more probing essay questions and with other direct assessment of student knowledge, but there is some deviation at both ends of the spectrum. In my experience that discrepancy amounts to in the neighborhood of 15% of the class. Whatever test is given to look at student performance needs to take that into account. ESL student performance on math, science, history, etc also needs to take into account the language barrier. Poor language skills do not necessarily mean poor math and science skills but they do generally mean poor performance on standardized math and science tests.

    Tango I think that you overestimate the effectiveness of standardized testing and too easily dismiss its limitations.

  57. TangoMan says:

    Tango I think that you overestimate the effectiveness of standardized testing and too easily dismiss its limitations.

    My estimation of test effectiveness is based on measures of external validity. When SATI & SATII have better validity than High School GPA in predicting the GPA of first year university then I would argue that those who think GPA is a better measure of student achievement are overestimating subjective teacher evaluations as being the best measures of student achievement.

    I’ve never argued that standardized testing was a perfect predictor, just one of the best. Until someone comes up with a better method of assessment, I’ll stick with this hand.

  58. Grewgills says:

    Tango,
    After a quick search of the literature it appears that SAT scores explain between 10% and 67% of the variation in 1st year GPA. That is quite a range and I don’t have the time or inclination to check the validity of the methodologies of the various studies since it won’t directly affect me or mine for more than a decade. Both SAT II score and high school class ranking (rather than GPA) were better predictors according to a number of sources. Class ranking also appears to be better at predicting success beyond the first year. SAT I has additional problems predicting the success of ESL students, African American students, and non-traditional students underpredicting success when one compares SAT I scores to undergrad class rank and graduation rate. Testing of some sort has its place as an admissions criteria among several, but is not the best predictor we have.

    The test also adds an yet another admissions advantage for those with the money to take the test multiple times and pay for test prep courses, which when competently administered increase board scores by more than 100 points. Students who learn these specific test taking tricks are not increasing their likelihood of first year success, but they are increasing their ability to be accepted for that first year.

  59. TangoMan says:

    As a University of California study on predicting college success found by examining the records of the 81,722 students who entered the system between the fall of 1996 and the fall of 1999, the SAT I & SAT II had greater predictive validity in measuring success in the first year of school than GPA. Further, if you delve into the details you see that over the course of the 4 year study the predictive validity of the GPA continued to fall, meaning “subjective grading standards” across a wide field of High Schools don’t capture a true reflection of content mastery as we see with objective testing. Further, GPA is a reflection of grading criteria that encompass measures such as attendance, class cooperation, work on group projects, etc whereas the objective test simply measures how well a student knows the material.

    Both SAT II score and high school class ranking (rather than GPA) were better predictors according to a number of sources.

    Class ranking isn’t a variable that is completely independent of GPA. Even the University of California study shows that this combination has the highest predictive validity. When data from both sources is available then it should be used, but this can’t be described as a single predictor.

    Class ranking studies are very interesting in that ranking methodologies can vary and quite often the ranking order is an informal reflection of cognitive ability and when this happens we’re bypassing the problems associated with a test of knowledge (SAT) and the problems associated with being rewarded for busy work (GPA) and we’re using intelligence as a variable to predict first year university outcomes.

    Class ranking also appears to be better at predicting success beyond the first year.

    Agree, though my agreement is contingent on the specific methodology of the ranking system.

    SAT I has additional problems predicting the success of ESL students, African American students, and non-traditional students underpredicting success when one compares SAT I scores to undergrad class rank and graduation rate.

    The College Board has released technical bulletins on this issue. They’re quite puzzled by why the SAT OVERPREDICTS black student performance in first year university:

    In terms of grade prediction, the common finding was one of overprediction of college grades for all of the minority groups (except for Asian Americans), although the magnitude differed for each group. With Asian American students, studies that employed grade adjustment methods found that underprediction of grades occurred.

    It appears to me that your concern about underprediction of performance for some groups, perhaps ESL students, is being generalized to other students for which this isn’t the case. Black and Hispanic students are actually given an admissions boost by including their SAT scores in the evaluation process.