« Caption Contest, Harbingers Of Doom Edition | Main | The Mommy Trap: When 'Having It All' Isn't 'All That' »

February 23, 2005

Never Assume...

Libertarian MOM (dang...I've been waiting all morning to do that) points out how easy it is to infer nonsensical conclusions from isolated stats:

Brad DeLong has a big headline, The Math Test Score Upper Tail: Is There Reason to Believe That Sociology Swamps Biology? And he then presents us with the following data:
In 1992, 2.8% of Asian-American women who took the Math SAT scored 750 or above.

In 1992, 2.1% of white men who took the Math SAT scored 750 or above.

In 1992, 0.4% of white women who took the Math SAT scored 750 or above.

In 1992, 0.2% of African-American men who took the Math SAT scored 750 or above.

Now what might a reasonable person infer from the above? I'll reserve my opinion for now.

And can he mention it in an academic setting without arousing the flouncing instincts of female scientists everywhere?

Well we already know the answer to that one, don't we?

Let's really throw some gasoline on the fire...

And if we're going to look at SAT scores (which I'm not at all sure is a valid approach), why not look at the amount of variability in male/female scores* as a possible indicator of whether Summers was on the right track?

Or would that be too empirical an approach to The Subject Which Dares Not Speak Its Name?


Posted by Cassandra at February 23, 2005 10:11 AM

Trackback Pings

TrackBack URL for this entry:
http://www.villainouscompany.com/mt/mt-tb.cgi/381

Comments

My basic reaction is "who cares?". But evidently some people do. To bad DeLong didn't provide the percentage for Asian-American males while he was at it.

Posted by: George at February 23, 2005 11:47 AM

I continue to find it amazing that:

(a) scientists in general would waste time arguing about the validity of mentioning a hypothesis when they could be trying to prove/disprove it

(b) feminist scientists aren't appalled at the behavior of twits like little Miss "I almost blacked out"

(c) anyone in this day and age would be horrified at the suggestion that physiological differences may have cognitive implications

Welcome to the Dark Ages...

Posted by: Cassandra at February 23, 2005 12:22 PM

Nice! And the IQ calculator is interesting. As you say, though, the SAT is not the best gauge.

It does appear that women take tests better than men, though ;-)

Posted by: Barb at February 23, 2005 12:25 PM

The IQ calculator was 20 points too low in my case. Not that one test is dispositive, but that's a BIG disparity - if you use a percentile-comparison approach with SAT scores and IQ it (at least in my case) tends to support Steve Sailer's original observation that IQ scores don't rise quickly enough with SAT scores.

Looking at the regression plots, it's hard to tell without sigma lines but unless something is really wacky I plot outside 3 sigmas. That can't be right.

Posted by: Cassandra at February 23, 2005 12:50 PM

Barb almost proved the point I wanted to make, but she did say "took tests better" not more "Intelligent".

What I expect some PC individual or liberal to conclude is that while Asian women are smarter than white men, white men are not smarter than white women.


Actually I don't believe there is enough data shown to conclude anything of value.

Posted by: Rodney Dill at February 23, 2005 12:56 PM

Actually I don't believe there is enough data shown to conclude anything of value.

Bingo!

Posted by: Cassandra at February 23, 2005 01:00 PM

Now what might a reasonable person infer from the above? I'll reserve my opinion for now. - Cass

Most people would (wrongly) think that because there are more people above a certain threshold in some populations than others that it was due to generally better scores.

However, these statistics (by themselves) are almost devoid of all meaning. Percentages above a threshold are not measures of central tendency. It could be that the average is exactly the same but the variance is wider. IOW, while there may be more people above a certain threshold in a population there may also be more people below a certain threshold for the same population. More Geniuses, but also more idiots.

Similarly, a fair die will have 1/6th of its rolls land on 6 and have an average of 3.5. A doctored die may always land on 5. While it would never roll a 6, it's average would be 5. The fair die would roll more sixes than the doctored one, but it would still have a lower average value.

That being said, if I had to roll a 6, I wouldn't look for the doctored die to produce it.

Posted by: Masked Menace© at February 23, 2005 01:11 PM

That was my point, Rodney, the data is inconclusive. It doesn't truly measure intelligence, instead it just demonstrates that some people test well, while others don't.

We fight this in the tech world all the time. People go out and get 'paper' MCSE certification which would indicate some prowess in designing networks (at least using MS products). In reality, it measures their ability to take the required tests on a given day.

Posted by: Barb at February 23, 2005 01:19 PM

It doesn't truly measure intelligence, instead it just demonstrates that some people test well, while others don't. - Barb

Careful Barb, as I pointed out in my post (that I snuck in while you were writing your response to Rodney) these stats (on their own) don't even imply better average test taking skills, only an increased likelihood of finding a good test taker.

Posted by: Masked Menace© at February 23, 2005 02:04 PM

*bows* Refinement/adjustment accepted. Your point is taken!

Posted by: Barb at February 23, 2005 02:32 PM

In an IT contract I was on for the Minnesota Dept. of Education, I swam in stats like these for months on end. To get to the bottom of the numbers (partly to check my own CALCULATING accuracy and reliable movement of the data from the test administration organization to MDE's servers), I had several long and deep meetings with some of the chief "education theory" luminaries of my State (as well as some techies who just made sure I wasn't missing rows and columns, etc.)

What I came to understand was partly what I already knew going into the project: any test, even a DIAGNOSTIC test, simply matches up a resulting behavior or indicator (a test answer) with actions taken to prepare for said test. A person could be extremely intelligent with remarkable IQ, but if a question asks for data that was never presented to the subject, he will still get that answer wrong. (The goal of IQ tests is to always make sure that the answer to a given question can be formulated by the data given along with the "question" or problem, itself.)

What one sees in aptitude, skill, and measurement tests given to students as a standard battery, is a measure of the degree to which the students in question were exposed to the information and material, studied it, reviewed in (in class or out of class), etc. It's key to understand this and not glaze our eyes over because we're coming up on the pivotal truth that blows things like "No Child Left Behind" out of the water: a major factor in student preparation for tests is the HOME environment. If daddy is in jail and mommy is a crackhead, Little Johnny's not likely to spend much time reading his assignments, studying, doing homework, or even thinking ABOUT school-related material. This can apply to any gender or ethnic group as the environmental factors will have the same impact on any subject of any shape, color, or reproductive equipment. Schools can't be held responsible for home life, and so measurement of "school performance" based on student test results, is fallacious.

The nuggets of wisdom that can be gleaned from the test results given here point mostly to differences in socialization and upbringing. It can be correlated by the data that in the environment in which Asian girls grow up, the support structure, parental pressure, peer pressure, etc., for the girls to excel at math are higher than the same structures and processes for students of other sex/ethnic groups. There is slightly more social cost on white males for nerdiness than for Asian females, among peers, and slightly less pressure by parents to study and do well by parents. And so on down the line.

The tragedy of ethnocentric interest groups (like "black activist" groups) is that they try to attack the test and say that whatever category of student had the most high-achievers at the test, the test was "geared toward them". Here we see that rather than a test geared toward while males, to be consistent they would have to take the nonsensical position that in a few short recent years they began to engineer tests to be easier for Asian females to comprehend. It's dialectic thin ice for them. What's really to be learned here, and needs to be understood by the ethnic activist groups, is that social and family conditions exist that need improvement in the home lives of such student groups as black males (assuming all these students tested attended the same school assigning the same lessons, homework, etc.) They could even blame "da man" for those conditions by some stretches of wild imagination, if they want to get all Dave Chappelle about it, but they bark up a wrong tree and never identify the true issues facing them then they obsess over whether or not the test should insult the student by posing questions in Ebonics.

Posted by: Ciggy at February 23, 2005 02:53 PM

Typo correction:

they bark up a wrong tree and never identify the true issues facing them then they obsess over whether or not the test should insult the student by posing questions in Ebonics

Should read:

they bark up a wrong tree and never identify the true issues facing when then they obsess over whether or not the test should insult the student by posing questions in Ebonics

Posted by: Ciggy at February 23, 2005 02:58 PM

I should just quit while I'm not ahead on typo corrections, LOL... fingers getting mighty fat lately.

Posted by: Ciggy at February 23, 2005 02:59 PM

Happens to me all the time - it was a great comment :)

re: Menace's Most people would (wrongly) think that because there are more people above a certain threshold in some populations than others that it was due to generally better scores.

I had read (elsewhere) that male and female AVERAGE scores (if this is an arithmetic mean I'm not sure I care, but whatever) are the same - as though one hires an 'average' rather than an individual - but the variability was greater in male test scores.

Then I saw a graphic that overlaid the distribution of male/female SAT scores and both the average and the bulk (the 'hump' of the bell curve) were well to the right of the female scores, which (HELLOOOOOOO - echo) suggests to me that males DO score better, overall, regardless of what the AVERAGE is.

And you know what - I can live with that if that's the case. You may be looking at a variety of factors, but the fact remains that there was enough of an intersect in both the average and high-scoring regions to suggest that schools/employers (all other things being equal, which of course they never are) should be able to find females with APTITUDE if they try hard enough.

But why should they? I don't go looking for prospective employees - they come looking for me if they want a job.

Posted by: Cassandra at February 23, 2005 03:19 PM

Hey, I'm not saying that men don't generally do better at math than women or that Asians don't generally do better at math than whites: just that the quoted data, in and of itself, doesn't get you there.

And as you have said, all the means, medians, and standard deviations in the world don't tell you crap when you've only got one trial.

Posted by: Masked Menace© at February 23, 2005 04:00 PM

Agreed - if it sounded as though I thought you were making that point, it was unintentional.

I knew you knew what you were saying... :)

Posted by: Cassandra at February 23, 2005 04:13 PM

As I believe I have pointed out before, even a reasonably accurate generalization is meaningless because it doesn't tell you anything about any particular individual.
I completely agree with you, Cassandra--why should schools and employers do the looking? The real problem is that they're as lazy as the rest of us and want an effortless way to make decisions. Not in this life.

Posted by: MrsPurpleRaider at February 23, 2005 05:42 PM

Yeah, what that Ciggy guy said.

Just another mindless sheep following the herd.

Based on the little formula on the linked website, my IQ (at age 17) was 124. What does that mean now, 30 plus years later? What did it mean then? Wouldn't a 3rd or 4th degree polynomial or a power series be a better fit to the data than some forced "linear" fit? What a bunch of educated simpletons. A 0.72 correlation ain't too impressive.
Who are these people trying to "engineer" society based on some achievement/aptitude test, anyways? knock it off.
And lastly, does it really matter? My limited anecdotal evidence indicates that most Asians (Chinese and Japanese) I have known are really good at math. Meaning? Tell me something I didn't know.
Gripe, gripe, gripe. I guess my job is marginally more secure than Larry Summers, at this moment in time.

Posted by: Don Brouhaha at February 23, 2005 06:12 PM

Don, in the C & R analysis I did for the USAF in manpower analysis and war planning, correlations of a potential workload factor to actual hours spent on a task had to be .94 or higher. Less than that, and we had to find a new WLF.

Some of the spend-happy bureaucrats on the MDE project wanted me to find correlations of spending to test scores, but the correlation there was in the area of about -.60. It almost looked as if the more money you spent on a school, the lower the test scores dropped (but then, that's due to the home life factors explained above, with the money not being a causative factor, but more of a symptom of the neighborhood problems--in a bad neighborhood, for example, you're going to get more broken windows that'll need replacing!)

Posted by: Ciggy at February 23, 2005 07:50 PM

Really.
At least that shows some rigor in analysis.

We do certain tests in our lab that require a degree of curve "fitting" (nothing too esoteric, mind you), but a correlation coefficient of less than 0.9 usually means "do it over". Of course, that is usually with only a handful of data points, not "thousands" of SAT scores.
Sometimes poor correlation indicates a difficult experiment to repeat, but the "relatively" poor/mediocre correlation of the SAT scores to IQ, etc. means something else, I suspect.

A S.W.A.G.: IT DOESN'T WORK!

Baffling people with bulls***, er, statistics, is an old game that wiser and sharper folks like Cass and the Masked Menace are game at de-bunking.
Welcome to the club of hard heads, Ciggy.

Posted by: Don Brouhaha at February 23, 2005 08:33 PM

Now I'm all cattywampussed.......I did the formulae for MY I.Q.(based on 1975 SAT), and on #1 son's I.Q.(based on 2003 SAT), and he scores 15 points lower, yet his math skills are an order of magnitude above mine.....I've always thought standardized testing is just a crutch for people who could not hold a stable job otherwise, now I'm convinced.....

Greg

Posted by: Greg at February 23, 2005 08:39 PM

Got 123.52 (yay, false precision) on the SAT-to-IQ formula. Gee, I tested higher when I was a kid. Prolly lower, now, what with the Budweiser treatments. I don't think those tests are all that accurate, though, on those of us who ain't neurotypical. Sometimes I'm dumber than I seem, sometimes smarter.

Posted by: Justthisguy at February 24, 2005 12:10 AM

By the way, when I said
"Just another mindless sheep following the herd", I meant ME (dummy Don), not Ciggy.
I thought his analysis was spot-on.

Duh. After re-reading the thread, I think I was grossly misunderstood above.

Posted by: Don Brouhaha at February 24, 2005 08:55 AM

It all seems pretty simple to me. Decide what kids should know before graduating high school. Teach that. Then test that. Can you apply FOIL? Do you know a gerund from a participle? Helk, can you read a map? Teach that, then test that. Are you then doing that dreaded "teacing for the test" thing? I sure as helk hope so.

Posted by: KJ at February 24, 2005 11:06 AM

Don, I've subtly shifted away from statistical analysis to more nuts 'n bolts database design and db programming lately. Part of the reason for that is to avoid having to defend the bullsh*t bafflegab to people who know better. ;)

KJ, the "test" is exactly what does need to be taught. It's just that if students do poorly at the test, it's not automatically the school's fault. Test the schools more on what sort of learning environment they provide to the students, not whether the students are slammed into a moron-mold by their parents and peers.

Posted by: Ciggy at February 24, 2005 11:38 AM

Well, we know that Bush has been awful to the learning environment, what with his water poisoning and air polluting friends running the EPA.

Posted by: KJ at February 24, 2005 02:25 PM

Post a comment




Remember Me?

(you may use HTML tags for style)