In the ongoing discussion about disparities between racial classifications on IQ tests, Dr. Bryan Pesta requested that we consider his paper, "Black-White differences on IQ and grades: The mediating role of elementary cognitive tasks." Because as he rightly points out, not everyone will have the background to evaluate the paper, I thought it would be helpful to discuss the paper in the context of the cognitive science literature.
Okay, this time I can't say I'm not doing science blogging. In order to take advantage of some of the functions of Research Blogging without the setup time, this one is posted at Greg Laden's Blog.
74 comments:
Interesting.
I don't think I'm worthy of this attention, but I appreciate anyone ripping me a new one over anything they perceive as crap re method, analyses conclusions, etc.
Thanks Stephanie.
Oh, and sorry to be the attention ***** by being the first two replies here, but I think I completely own the copyright on the paper Stephanie linked to (Elsevier owns the CR on the final final version published in the journal).
I hereby release / hold harmless / let you remove the tag and use this article for not-for-profit academic purposes including but not limited to exposing my ignorance.
Three in row.
I lurk at science blogs but will not post there again given my experience (let's not dredge it up here, suffice it to say that if anyone wants me to address their concerns, post them here).
Some prominent blogger elsewhere though should read up on the difference between a source of error variance and a confound. Just saying.
Bryan -
Blogs are free.
http://wordpress.com/
http://www.blogger.com/home
http://bloguni.com/
Or here:
http://www.christianblogsites.com/
Thanks Mike, but I'm not sure I get the comment-- you mean free, so that CR is not an issue? If so, I understand and agree that would keep this fairly in the realm of fair use.
If that's not what you meant, please reply.
Some comments addressing points elsewhere before I call it a night:
1. The black SD was indeed larger in my sample. Since SD is calculated by dividing by sample size, the most logical conclusion (but not the only possible one) is most of the difference is due to running about 3 x as many white subjects as black subjects.
2. Someone made an interesting point re race IQ and GPA.
I think they mis-read the tables.
Race and IQ showed a significant difference.
Race and GPA showed a significant difference
Race and ECT performance showed a significant difference.
So, it's not true that "race exerted an effect on GPA but not through IQ" (as measured by the Wonderlic). I didn't present that analysis. I would be willing to run it now and report it, if anyone thinks it's a key point.
It is true that the race effect on GPA could not at all be explained by the race differences on my ECTs.
May seem like a minor point, but I think it's critical to showing why this comment is incorrect.
Seems like there's a limit here on post / comment size.
Cont:
3. Someone else made the valid point that because the paper and pencil IQ test is speeded, it's not all that interesting that my ECTs mediated the race difference.
The Wonderlic also allows for non-speeded adminstration (I chose speeded because I adminstered them in group settings). IIRC, the raw score rises about 4 points when people do the unspeeded versus speeded version.
I will need to find a cite, but I would be lots the correlation between speeded and unspeeded versions is near unity. So, the results would be the same even if the Wonderlic were not speeded.
Also, it's important to realize that the IT task is NOT a measure of speed. Subjects could take as long as they wanted to decide which line was longer. I don't even report reaction times on that task in my article.
Instead, IT measures perceptual intake (how much information the cognitive system can process accurately, when that information is presented very briefly).
3. I agree the literature on RT (ignore IQ) is about as massive as any literature in social science. I am not familiar with the must-know authoritative cite on RT provided on another blog.
The most recent comprehensive review on RT as a measure of IQ is linked below.
My dissertation was on attention (negative priming) and I used RT tasks. It had nothing to do with IQ. My point is, I'm not so sure it's agreed upon that RT is basically a measure of attention.
I've done about a dozen RT studies-- only the latest 4 were on IQ. My experience is that they are a bitch to do correctly. One must have very careful controls, or will got no interesting results at all.
Except when using RT to measure IQ/g.
The IT task took about 8 minutes; the rt task took about 5 (iirc). That's not lots of trials, and I have more slop than I probably should have in terms of things like using keyboards versus response boxes, different fingers for different responses, right hand only, etc.
Despite this, the ECTs completely mediated the race difference on the IQ test. I admit this is an odd argument, but even a relatively sloppy measure of ECT performance nonetheless was powerful enough to mediate race differences (establishing an NP effect for my dissertation required so much control that I wasted about 6 months revising and pilot testing the stimulus set and procedure).
Finally, the difference between a confound and a source of error variance is going to be key here. Many are many mentioning effects on error variance (practice trials; very short tasks used by me)
Those cannot explain the race results because they did not covary with race. The only thing they will do is increase noise in all the data, reducing the power to detect any effects that might exist.
That is also a critical issue that should be understood.
Good night!
Sheppard, L., & Vernon, P. (2008). Intelligence and speed of information-processing:
A review of 50 years of research. Personality Individual Differences, 44, 535–551.
What I meant, Bryan, is that you seem to have a lot to write, and you are using other people's blogs to do so. Why not set up your own blog to write all you want to about it? Considering how you think that people should follow your rules on their blogs, I think it the right thing to do.
Gotcha Mike.
I started this whole thing with about a 1 sentence comment saying that a list of recommended readings in the area seem lopsided.
Now, I have a thread with my name on it, devoted to having someone somewhere expose me as a racist.
I stopped posting on all other blogs where this debate played out.
You really think posting here is somehow rude?
I didn't start this thread, and wasn't even told it was going to be posted-- I stumbled across it lurking on pharyngula.
I'm not going to address any other comments regarding the personal tiff between me and other bloggers.
Here, I'll just address comments on my paper and then I hope to get back with my life.
Oh, and please stop calling me Doctor. This is the interwebs and I sense a mocking tone to use of that title here.
Bryan, I would like you to run the numbers on the correlation of IQ and GPA if you haven't already.
You found (paraphrasing):
Significant correlation between race and IQ but smaller than expected.
Significant correlation between race and GPA, larger than expected from IQ data.
Significant correlation between race and ECT measures
Significant correlation between IQ and ECT measures
A factor score indicates that the correlation of IQ is mostly mediated by the ECT score, followed by age, with race being non-significant.
Not even partial correlation between ECT and GPA.
Given that IQ and ECT are highly correlated, I would expect that things that do not correlate with ECT would also not correlate with IQ. In thinking more about this, in principle that might not follow, but if IQ doesn’t correlate with GPA, doesn’t that call into question the importance of IQ?
Thanks D. for the comment.
First, I really think scienceblogs has crossed the line by calling me a racist and white supremacist (by inference) on their main page.
I'll do your suggested analysis and report it hopefully tomorrow.
I'm not sure I understand what you want.
All simple correlations are reported in Table 3.
Wonderlic IQ scores correlated .30 with GPA (coincidentally, exactly the validity co-efficient reported by Mckelvie and cited in my methods section).
Given that, do you still want me to run a mediated regression to see whether the race difference on GPA is statistically explained by the race difference on the Wonderlic (I'll do it, but I predict partial mediation with also sex and age being important)?
Other minor clarifications:
1. The race/iq difference I found (d=.45) was smaller than that typically reported in the literature (d=1.0).
2. Given my relatively small race/iq difference, my comment was the race/gpa difference was relatively large. I think the effect size I found for GPA is only nominally higher than what most other research reports (see my comments comparing my effect size to Gottfredson's in the paper).
3. Even IQ and ECT correlated in Table 3-- though weakly (r = -.17).
My *speculation* for that *here* is less than perfect reliability for both GPA and my ECTs.
4. I regret calling PCA a type of factor analysis. It is not. It's my mistake that unfortunately made it to the final version of the article. It's perhaps a minor detail, but every where I say "factor," I should instead have said "component".
I'm not trying to be evasive, so if you think my comments did not address what you had in mind, please take a minute to reply and clarify.
Ack, point number 3 should be:
Even ECT and *GPA* correlated weakly.
Sorry bout that.
Bryan, as someone who occasionally runs into trouble with obsessiveness, let me just say that you seem to be working up a dandy obsession on the topic of being called a racist. Your ideas are being pummeled for being predicated on a race-based view of genetics that isn't supported by history and for being used by racists, even within the field. You're being beat up slightly for overstating the conclusions your study will support based on its design. You're also being mocked by Greg for the argument, "I can't be a racist; I voted for Obama," which is based on a simplified definition of racism (to the point of being a straw man) that is painful to anyone who discusses the topic with any sort of rigor. And finally, you're getting random slings and arrows from the random set of people who find the internet to provide a great means of self-expression and who have discovered that you have a moderately thin skin and can dish it much better than you can take it.
That's it. No conspiracy. Just uncoordinated reactions to your inserting yourself into an ongoing debate. The most coordination that has gone on is between Greg and me to get my post on his blog, where it could be added to the feed on Research Blogging, which I support but haven't signed up with because I almost never do any of it. Greg had to ask me to send a text file with the html once he agreed to post it.
As for your other points, you asked us to look at your study. My post at Greg's clearly constitutes fair use and did not republish the pdf. You do report intra-individual variability in response times that are significantly different between racial categories, as would be expected if practice were a confounding variable (see Table 2).
I was hoping we wouldn't dredge up the lengthy interactions I've had with you and Greg on other blogs.
You mention things here out of context. The Obama thing-- Greg was offended by a photo I have on facebook with me and Obama smoking cigarettes. He accused me of something insane like a skinhead posting some type of racial slur.
It was such an off-base comment that I got into why I had the photo, and how voting for Obama demonstrates only that my photo was not a racial slur (I have not used the “I can’t be racist I voted for him” argument, as you suggest).
With context, it shows in my opinion how ignorant and unfair you guys are being in this “debate.” The rest of your examples are equally inane and I won't address them here.
We do share obsessions, as I think there are now 4 different blogs by you guys and 6 different threads where group hugs are had over my racism, despite anyone making any claims—valid or otherwise—about things like data.
I'm writing here for that one person who might step up and say the scienceblog emperor has no clothes, and expose Greg for the blathering idiot he is (of course last time that happened, Greg accused me of using sock puppets; deleted multiple threads, and then closed the thread off for further discussion). Censor, delete, lie. Scienceblogs awarded that by featuring the thread in question on their main page?
I wish your blogs had readership beyond your kool-aid click (yes, that's a ding!). Could you at least get a scientist to comment on my methodology? At this point, by scientist I mean someone who can get stuff past peer review, and understands the difference between error variance and confounds (look it up already, and perhaps apply the distinction to the practice effect you think is a confound).
To further expose your ignorance, you guys exude extreme authority on all matters scientific, with a key focus on intelligence research.
You legitimize yourselves by bastardizing the name "science" by posting on scienceblogs.com. When I call you on it, you claim I'm whining because despite pretending to be authorities and using "science" to describe your activities, you have no obligation (under scienceblogs’ TOS) to actually reason scientifically.
You are self-proclaimed arrogant experts on IQ, yet Greg was not aware that a link existed between IQ and job (among other things, despite 1000s!!! of studies showing it). Now you offer “scientific” criticisms of my article. Specifically, you just claimed race differences on the SD of RT and IT support your argument that I have a confound.
You thus demonstrate complete ignorance of a well-known effect in the literature (that the SD of RT is as good—if not better—a measure of IQ then is the RT itself! Look up the “worst performance rule” in the literature you are obviously ignorant of).
So, present someone qualified to evaluate this or stfu already. You are doing an extreme disservice to Science here and sooner or later, that will be exposed.
Bryan, you're getting hysterical again. I've said your experiment has a potential confound in practice, because practice also decreases reaction time. Citing the worst performance rule doesn't change that. The worst performance rule would predict that as subjects find the task "easier" due to practice effects, we'd see less variability in the results. It in no way rules out a practice effect.
Your comment about voting for Obama is here: http://scienceblogs.com/gregladen/2009/12/the_argument_that_different_ra.php#comment-2160573
And you really shouldn't comment on blogging without knowing something about it. Complicated subjects, as IQ is, frequently get series of posts about them. This one started in response to a white supremacist before you showed up. This is NOT all about you.
Thank god for google. But, I baited you here.
An expert in the area should know why the RPR is irrelevant to my data.
So, why is it not a factor here?
Be careful, you may have to cite Arthur Jensen to support your claim.
Yes, I'm not expert in blogging. I admit it. Is there anything you or Greg claim to not be expert in?
Is it possible that lack of expertise in a rich, highly technical literature might lead one to faulty conclusions about a study's merits?
Bryan, there are plenty of things I'm not expert in. For example, I don't know how to milk a cow. Nor have I claimed expertise. I told you to assume I hadn't completed high school if it made you happy. Remember?
It is, of course, also possible that lack of expertise can lead to faulty conclusions. However, it's pretty typical in that case that additional information is offered, rather than the expert waving their hands and saying, "Eek! They're out to get me!"
As I have no idea what you mean by RPR, you'll have to explain yourself.
daedalus2u (elsewhere) said:
All humans have self-tuning neural networks. All humans exhibit improved performance in all tasks with practice. The unpracticed ability might have a pretty strong genetic component, but after a few thousand hours of practice, it is mostly the practice.***
I agree with this (and it also helps address Stephanie's comments on practice effects).
You seem to concede that unpracticed performance might have a genetic component.
What about the learning curve for individuals?
Could that differ?
All of us would get better with practice and then reach some asymptote.
Would you expect the slope of the performance curve to be identical for all people?
Would some increase performance sooner than others, and would some show final levels of performance much higher than others?
Stated differently: We all start as naive empiricist / blank slates, and then perform the same speeded, but simple, cognitive task.
Trial 1 differences might have a genetic component according to you.
What about trial 10,000 after 20 hours of practice?
And, what about the slope of the performance curves across people from T1 to T10,000?
Would all people be equally fast within some margin of error?
My answer: the individual differences in final performance and steepness of slopes; that is the essence of g. That is intelligence!
Stephanie: Good. The probability of a lay person being able to critique a study with a literature this deep (in any field) and add something useful is pretty low.
There's a reason most people who publish (in ANY scientific area) tend to have ph.d.s in the area they publish in. There are notable exceptions, but good luck trying to be one.
You need to know some pretty specific stuff in your field before you are ever going to contribute something new or important (the conventional wisdom is 10 years of serious practice or study to become an expert in something).
Does not mean the Ph.D. is correct. Does mean good luck trying to educate lay people to be skeptical consumers of scientific journal articles in technical areas.
I'm qualified to evaluate any publication in biological anthropology (e.g.), but I am willing to admit it.
***
The typical reported measure in the IT task is the final IT value the subject reaches at the end of the task. It's not their reaction time, and it's the value at the very last trial, which makes having 10 more practice trials in the beginning irrelevant, especially with the adaptive staircase method that very quickly and efficiently hones in on a person's true "IT".
I pilot-studied the 3 letters RT task on my then 9 year old son.
He performed will under a second per trial with very few mistakes.
The means for my college sample were just under 1/2 a second. This task is so simple that speed has to be the measure.
How much practice should one get to complete this task: Where does the letter A appear?
SAS
Position 2.
Ok, let's start.
Bryan, I don't need a PhD to know that practice effects are found in much simpler reaction time tasks than that. All I had to do was pay attention. Ando, Kida and Oda (2002) found that up to three weeks of practice decreased simple reaction times (for the layfolk: see stimulus; press button).
And seriously, you just told me, "How do I know that black adults don't need practice on a reaction time test? I asked my white kid." I don't even need to add anything to that. Not one thing.
Bryan, you are wasting your time trying to explain to this blogger about how to understand things at the PhD level. One has to have experienced the process, passed the exams at various levels (depending on the program) and earned the PhD before really being able to understand peer reviewed papers. I'm sure you know this from your own experience.
Which is why this whole OpenAccess thing is a waste of time. There is no reason for anyone outside the academic sphere to be seeing any of this work. If the publication business model was extended to allow that to happen we would be distributing resources in a very ineffective way. There was a time when our society could afford to pay for everyone's access to all kinds of resources, training, teaching, higher (and thus frightfully more expensive) quality educational opportunities, and so on. But the fundamental nature of the economy has changed from a kind of post World War II bubble. In a world where most middle income families must have two incomes to maintain quality of life equivalent to middle income families in the 1950s or 1960s, we simply have to focus the distribution of resources where they can be most effectively used. This is why research like that being ridiculed by this blogger is so important.
If we truly want to live in a brave new world ... some of us have to be brave.
Okay, I couldn't bear to read all of these comments and other blog posts & comments in their entirety, so please forgive me if I'm not addressing anything of interest here. I have read some of it and, most importantly, I've looked over the methodology of the paper.
I have a few comments and I hope I am enough of a "scientist" (whatever that means) to give my comments credence.
1 - Practice is clearly not a confound (which must covary with the IV) in this study and in these types of tasks practice is actually necessary. It even increases reliability. As long as everyone practiced the same number of trials, it's not an issue.
2 - I hate to nit-pick, but I have serious problems with some of the language used or quoted here because the wording often suggests things which are quite different from what is meant. For example, "effect of" in a statistical sense does not mean anything close to "cause", which is suggested by "exerted an effect on". Likewise, OF COURSE there are differences between race and IQ. They are different things. What you mean is that IQ differs among races.
These things may seem trivial, but I assure you they are not. Undergrads in sciences learn and use this language. As a result, most fail to understand common (in research) terms like "significance" and "effect".
3 - As mentioned briefly in the discussion...
Age is a confound.
Gender is a confound.
Only in the analysis which controls for these things are they not confounds. Of course, race is also confounded with a ton of other things, most of which cannot even be identified, but that is why studies which attempt to explain differences among them are difficult and usually involve large sample sizes drawn from the populations to which they intend to generalize. In most studies in cognitive psychology, college students are reasonable subjects. In studies of race, age, gender, and other uncontrolled variables, not so much.
4 - Differences in the effect sizes in the IQ comparison are likely due to both limited sampling methods (as mention) and a lack of sensitivity in the IQ measure.
5 - Paragraph 2: "This measure is not typically reported in the literature. We included it here because it showed significant race differences, and also correlated with various other measures in the study".
Translation: "We reported it because the p-value was less than .05".
Newsflash: "It was significant" is not a good reason to report something completely irrelevant. To steal from Neil deGrasse-Tyson, science doesn't care about the square root of meatloaf.
All this info tells me is that I should worry that the differences in sample sizes make the analysis of it unreliable, even if the variances are not significantly heterogeneous.
Why was this even tested? WTF do those tell anyone in this case?
More importantly, how much has alpha been inflated by all of these tests?
6 - You did a pilot with your 9yo son? WTF?
7 - Finally, the entire study appears to rely on the finding that ECT mediates the relationship between race and GPA, yet this analysis is weak. Why wasn't there a statistical test of this mediation? The stepwise regression is not sufficient given that B is not zero in the full model, but it is easily tested. Why did you fail to report the standard errors of the Bs so that readers could test this themselves? Would you provide this information?
Even if a test revealed a significant mediation, I would hesitate to draw any conclusions from this study given the number of statistical tests, the populations from which the samples are drawn, the unexplained differences in age and gender, and the hugely unequal (and relatively small) samples.
While I applaud your effort in a difficult field, I would take this article much more seriously if the findings were more clearly interesting. Instead, it just screams of the need to publish in order to earn tenure.
please, explain to me the difference between a confound and source of error variance.
It was 60 trials and 6 practice trials.
If you want, I'll spend literally hours recoding the data and run the results on just the last 10 trials for each subject.
That gives 56 trials of practice. Is that enough?
I modelled the IT task nearly exactly after that used by Luciano et al.
Their studies did not look at race, but identical twins. Some of the kids were young enough that they framed it as fishing task where the two lines were worms and they had to pick the longer worm to fish with.
They thought practice effects so critical that here's how much detail they devoted to it in the method section: "instructions were presented along with several demonstration trials"
"Several" doesn't sound like a lot (I guess we'll never know), but if it were so critical a detail to control for, you should send them a letter of complaint.
see page 446:
http://genepi.qimr.edu.au/staff/nick_pdf/CV314.pdf
To Anon: WOOT, and thanks.
Badrescher,
Thanks much. I sincerely appreciate substantive comments from people.
It's 2 am here and I am going to bed. I'll peruse your post tomorrow.
One quick thing: I agree, the vast majority of publishing in science is more about keeping jobs than advancing science.
Do you know what the median citation rate is for an article published in a scientific journal.
Zero!
So, I put heavy weight on impact factors-- I know many hate the things.
I wonder too now whether the attention I am getting here is bumping my publish or perish (google scholar based rankings) statistics.
So, while I was writing, "Anonymous" posted a "they are simply ridiculing what they don't understand" comment.
Although I disagree with Stephanie about the issue of practice, the reason that Stephanie persists in the argument is that Bryan's response was "Google 'confound'" rather than answer with a very simple,7-word explanation for why it is not a confound (A confound covaries with the independent variable).
Stephanie is perfectly capable of understanding that explanation.
Her other points (especially those regarding Bryan's defensiveness) are valid.
There are certainly times, as there were for me today in another discussion, when it is appropriate to tell someone that their knowledge in a field is far too removed from the discussion they are trying to have about it. This is not one of them.
Barbara, the practice that I'm concerned about is practice in similar tasks outside the study that may transfer to competence in this task. I mention it in a comment somewhere instead of in the OP, where I'm not altogether clear in my objection, but researchers are seeing a decline in the gender difference between boys and girls over time that some are hypothesizing may be due to higher sports participation by girls. If there are similar differences between the racial classifications in participation, transferable practice effects may be a confound.
I see three possibilities to address this that I see from where I sit. A survey of participation in related activities could be developed, scored and controlled for. Additional task-specific practice might ameliorate some of the differences, although that would have to be independently verified. Or both of those could be noted as being beyond the scope of the paper and big caveats added in the discussion. I don't see deciding it isn't a problem based on one's kid's reaction being an option.
And I apologize for any addition to your teaching load. Let me know if there are specific things I wrote you'd like to see cleaned up.
I suppose thanks are in order, both to Bryan and to Anonymous for condescending to speak to the layfolk about subjects that are clearly beyond their ken. You're doing a public service by undertaking a task we obviously don't have the resources to waste on, since you can't understand anything as complicated as what you're doing without a PhD. Clearly we need to develop some sort of caste system where the high priests of science explain their interpretations of experiments in a fashion not unlike the high priests of religion interpreting their dogmatic texts.
Sarcasm aside, what the hell dude. WHAT THE HELL. Didn't you try to say you have been nothing but gentlemanly? Then when a layperson calls your bluff and analyzes your methodologies AT YOUR OWN REQUEST, you pull the "oh, I own the copyright" card and "hold [her] harmless" for doing something that is not only clearly within the scope of fair use as the paper in question was not republished, but also AT YOUR REQUEST!?
And then the semantics games. Because clearly you need a PhD to understand the difference between a confound and error variance, so you can't just explain where one might have been misused and score proper rhetorical points for your argument that way. Meanwhile, other PhDs assure you that Stephanie can "hang with the big boys" just fine, so you must naturally dismiss them as not being in the field.
This is not how science works, pal. I'm a layman myself and I know this. Science works best if done in the open, and you're advocating anything but, throughout the entire damn discourse.
Feel free to dismiss everything I say as stupid prole bleating though.
One can ask the question: "Given two 12 year old children, one with an IQ of 115, the other an IQ of 85, what can we know about the differences between those to kids? The answer can only be tentative. But if instead we ask "Given two sixth grade lases, one for which the average IQ is 115, the other 85, what can we say about this difference, and their average prospects for the future? ... then there is a lot that can be said. We can make useful statements that can help affect decisions and policy for the school, educational policy in general, and even social policy.
That is one reason why this kind of research is valuable, and it is not fair to judge the whole approach on the basis of inevitable and minor perceived flaws with one study. All studies have such flaws, it is the nature of research but this is often not understood by those who do the research.
Bad; more later.
My snarky tone here can only be appreciated by suffering through the 100s of posts me her and Greg have had on this.
I would say it's wholly not worth reading. But, since your comment about Stephanie being fair and me being unfairly rude and dismissive, spend about an hour on greg's blog.
PLEASE!
Start with the thread censored and closed on xmas.
To save time, skip any post that isn't me greg or stephanie.
Come back here and tell me if your jaw dropped.
more later.
Oh, and count how many times I've mentioned confound versus error variance...which is why I suggested perhaps she google it.
To save time, skip any post that isn't me greg or stephanie.
That is a very interesting comment. You see, my closing of that thread (which is reopened) was all about nasty people making nasty comments that I did not want to see on my site. Those people were not Bryan. I deleted about a dozen posts that were not the normal spam (that would be a dozen more) not by Bryan. My commments and concerns regarding racist bullshit that was developing on the post were only minorly directed a Bryan.
But Bryan, for some reason, reasons of his own, reasons that may explain some things (if you get my drift) took every anti-racist comment by mea and others as against him, even though most of them were directed at others.
But, if you read ONLY posts by Bryan and me, you will see what looks like a dialog. But it is only partly a dialog between he and I. It is mostly a dialog bewtween me and two or three people who are "off stage" (since you are skipping the posts as per Bryan's instructions)
But Bryan wants you to believe that I attacked him specifically and was a meany to him specifically and that I'm out to get him specifically.
And to get you to believe this he adduces ... CHERRY PICKED EVIDENCE.
Interesting. Very very interesting. One wonders of Bryan conducts his scientific life in this manner as well. I don't know if he does nor not. I'm just sayin'
Oh, and count how many times I've mentioned confound versus error variance.
You have indeed mentioned this frequently. I myself had not commented on this, so I've been waiting for the dialog to develop along these lines, but it has not.
So, Bryan, what are you talking about here? Why should we care? What are you trying to say? I am truly interested.
Bryan said: "You thus demonstrate complete ignorance of a well-known effect in the literature"
Yes, and it is important to keep the larger picture in view also. Only by facing up to the fact that for numerous intractable social problems there is a demonstrated correlation between a high incidence of the problem and the low intelligence of those causing, and often victimized by, that problem, including unemployment, drug use, crime, can we make positive social change. There are great differences in intelligence between groups, and although it would be possible in theory to issue everyone a crutch or a save for this problem, in practice it is impossible to do this given the limited, and increasingly limited, resources we have available to us as a society.
Just as an employer might quite justifiably exclude those with a lower IQ from a high intelligence demanding job they are paying someone to do society at the broader level needs to focus its resources and efforts.
Do we tell every kid that they can be President and that they should not be preparing for any other position in life? Does a major corporation randomly pick from the people who happen to be passing by the front of the main office who will be CFO?
Bryan is doing important research and although it is always appropriate to scrutinize research it is better to do it from the point of view of knowledge and experience of the field than from some politically correct self serving point of view.
Bryan, people should absolutely not skip posts in the history of this discussion. If they were to do that, they would miss gems like you enthusiastically endorsing the dogwhistles of Anonymous here. Of course, your assertion that you need to work specifically in intelligence research--unless you're a judge--is entirely in your own statements.
Greg said: You see, my closing of that thread (which is reopened) was all about nasty people making nasty comments that I did not want to see on my site. Those people were not Bryan. I deleted about a dozen posts that were not the normal spam (that would be a dozen more) not by Bryan. My commments and concerns regarding racist bullshit that was developing on the post were only minorly directed a Bryan.
And I was around for that. I saw that. While it was happening. So when I went back to that thread, I actually had the context you suggested I didn't need, Brian. So my jaw didn't, in fact, drop.
People here and at Greg's have you dead to rights. You might not even realize the depth of your own biases here -- I'll grant you the possibility that you're ignorant of them -- but you're exposing your underlying biases in the way you've handled your methodology and the way you've defended yourself (especially against attacks that were in no way aimed at you!). Your defensiveness is a case of "the lady doth protest too much". And your suggestion that only serious scientists could understand the information of great import that you bring them is hilarious in its patent bias.
Perhaps you could refer to your thousands and thousands of studies on which you ostensibly lean, to find something to prove that scientists are the only ones capable of understanding the scientific methodology and spotting flaws in your reasoning? And that, therefore, those of us pathetic laymen that suggest you are extraordinarily skilled in defending a position you came to for silly reasons (cf. Rushton), are obviously wrong?
Thanks Jason-- we obviously disagree and my request was made to Bad (I predict he has a ph.d. in social science; possibly cognitive psych. I am very interested in his reaction to the other blog postings).
Welcome Greg.
Patiently waiting...
Oh and this thread's getting derailed, but it's not my call.
Greg could end lots of the derail by showing the deleted posts; if that's administratively possible.
I specifically remember the one from D. Then things started disappearing. Didn't see any kind of spamming, though I was in the blog exactly when Greg was whacking threads to save face.
Bryan, badrescher is Barbara Drescher. I think she's had her say on your article.
Not familiar with the name, but thanks.
I guess I will reply to the nice comments she made and then move on.
Dr. Drescher:
My apologies I think I assumed you were a he/not a she. I too am/was a cognitive psychologist. I hope that doesn’t make you regret your degree! My only real contribution there is I have a few papers on false memory (I am one of the few people I know that has gotten the F word and others published in an APA journal).
I understand you’re done commenting on this blog. No problem. I do think I owe you a reply, given the time you spent on it, even if you don’t care to read it.
You said: 1 - Practice is clearly not a confound (which must covary with the IV) in this study and in these types of tasks practice is actually necessary. It even increases reliability. As long as everyone practiced the same number of trials, it's not an issue.
Me: Agreed.
You said: 2 - I hate to nit-pick, but I have serious problems with some of the language used or quoted here because the wording often suggests things which are quite different from what is meant. For example, "effect of" in a statistical sense does not mean anything close to "cause", which is suggested by "exerted an effect on". Likewise, OF COURSE there are differences between race and IQ. They are different things. What you mean is that IQ differs among races.
These things may seem trivial, but I assure you they are not. Undergrads in sciences learn and use this language. As a result, most fail to understand common (in research) terms like "significance" and "effect".
Me: Conceded. I’ve struggled with writing well all my adult life (not saying I write well now). Those are all functions of poor writing on my part. I decided to adopt a very informal writing style when trying to publish as it seems to help me avoid verbosity and passive voice. It does lead to less-than-accurate phrases from time to time. I don’t think I have an obligation, though, to write with an eye toward facilitating a grad student’s education. That’s not at all the purpose. Good writing, though, probably solves the problem anyway.
You said: 3 - As mentioned briefly in the discussion...
Age is a confound.
Gender is a confound.
Only in the analysis which controls for these things are they not confounds. Of course, race is also confounded with a ton of other things, most of which cannot even be identified, but that is why studies which attempt to explain differences among them are difficult and usually involve large sample sizes drawn from the populations to which they intend to generalize. In most studies in cognitive psychology, college students are reasonable subjects. In studies of race, age, gender, and other uncontrolled variables, not so much.
Me: Partially conceded. I agree age and gender are confounding, but I controlled for them in the key analyses (mediated regressions). What else can one do with an after-the-fact confound? As an instructor, I’ve always wondered why black females are so well-represented at my university but black males are not. I have no explanation for this.
My black sample was indeed small. I ran all semester then stopped. I didn’t want to wait til the next semester to continue, so I didn’t. I argue that small sample sizes (power) are only an issue when non-significant results are found. That was not the case here on the key effects in the paper (ECT/IQ and race). In fact, race correlated significantly with every variable I reported. So too did IQ (iirc).
I also think small sample sizes are perfectly fine from a statistical / power point of view if one has strong manipulations. Multiple trial RT medians are pretty stable in ranking people (had I to do this over again, I would have picked a slightly more complex RT task and ran 100 trials instead of 60). The IT task “staircase” method is very efficient at quickly, reliably honing in on a person’s true IT.
So, small sample sizes are not a problem when effects are found; and my strong IV manipulations helped off-set the small sample for Black students.
Ack, no one's gonna read this!
Continued (sorry for the length)
You said: 4 - Differences in the effect sizes in the IQ comparison are likely due to both limited sampling methods (as mention) and a lack of sensitivity in the IQ measure.
Me: Disagree. I think restriction of range (college students) was the key culprit. And, the Wonderlic reports very impressive reliabilities in its manual. In fact, the Wonderlic did its job nicely in all cases, except for the smaller than expected race difference.
I think the lack of sensitivity came with GPA, which is a very sloppy, multi-dimensional measure of something (what it measures I haven’t figured out yet).
You said: 5 - Paragraph 2: "This measure is not typically reported in the literature. We included it here because it showed significant race differences, and also correlated with various other measures in the study"…Why was this even tested? WTF do those tell anyone in this case?
More importantly, how much has alpha been inflated by all of these tests?
Me: ITSD is not usually reported in the literature. There is no standard way to score IT, unfortunately. Some use a complex math equation (luciano et al.); others use the last trial IT (need to dig for a cite). I opted to use the average IT across all trials. That then let me calculate a SD.
Why this was not a fishing experiment: There is strong evidence in the literature that a person’s variability is more important than his/her mean performance. SD’s predict IQ better than median speed! This has led a few people to propose a theory of IQ/g based on oscillations in how the nervous system times its response to stimuli. I don’t know this area well, but I did know that SD was important enough in the literature that if I had it, I should report it.
Alpha inflation: This was a pretty simple study; one experiment and some variables. You must present the simple correlation matrix. Any study would. Alpha is therefore inflated there, but simple correlations were not the focus of the study. The key analyses were two mediated regressions. That’s not much in the way of data analysis compared to the average study. Given the large literature in the area, and that my results squared with it (on ECT, not GPA) the odds I’m making type I errors on key analyses are far less than the odds I’m making a type II error on gpa.
(last)
You said: 6 - You did a pilot with your 9yo son? WTF?
Me: I pilot-studied the whole family. I am not a computer programmer. I used mel/eprime and a borrowed script from Luciano. I had to modify it to do what I wanted. Took me 6 weeks. I had to get into literature on how the eye processes rapidly presented info on a CRT versus LCD monitor. Pretty technical crap for me (I went with the CRT). My point: My pilot study was just to see whether the thing would run; present the lines the way they should, and accurately code the data. It did. Why would this lead to a WTF response?
You said: 7 - Finally, the entire study appears to rely on the finding that ECT mediates the relationship between race and GPA, yet this analysis is weak. Why wasn't there a statistical test of this mediation? The stepwise regression is not sufficient given that B is not zero in the full model, but it is easily tested. Why did you fail to report the standard errors of the Bs so that readers could test this themselves? Would you provide this information?
Me: ECT mediated the relationship between race and IQ (it failed to mediate for GPA). This I think is the key contribution: race differences on a paper and pencil IQ test were “statistically” explained by race differences on a speed task, and a line-length-judging task. I think it’s a novel test of Spearman’s hypothesis. In fact, I was surprised that I couldn’t find anyone who had done it before. That’s really the contribution. Whether it’s trivial or not depends on whether it’s never been cited 10 years from now.
You: While I applaud your effort in a difficult field, I would take this article much more seriously if the findings were more clearly interesting. Instead, it just screams of the need to publish in order to earn tenure.
Me: Thank you. Again, whether it’s crap done for tenure or a contribution to the literature will depend on how often it’s cited. We’ll see (I’m sure we’ll still be posting here then, so check back!).
Oh, if you do reply, can you briefly comment on whether the discourse here is very similar to what an author might do when submitting a revision that addresses concerns the reviewers had on the first draft?
"And I apologize for any addition to your teaching load."
?? I'll flog you later for your incompetence. Or recognize that you're being silly. :P***
I see your point about prior practice and it serves to emphasize my comments about the overall value of the findings. Although explaining differences in RT is beyond the scope of the study, there is little value in findings this weak when so many confounds are possible. Funny that one of daedalus2u's comments on GL's blog (and one I seem to have skipped over initially) mentioned "stereotype threat", which is the first confound that springs to my mind whenever literature attempts to study racial differences in performance.
Bryan, I don't think Stephanie meant that I wouldn't comment at all; she knows that I have some distractions at the moment and that my comments addressed your paper, not discussions on Greg's blog. In fact, I spent hours reading more of that and it was a complete waste of time, imo.
Regarding what you've said here:
1 - You've not only mistaken me for a man, but you seem to think that I have more credentials than you. I don't. I'm ABD and you have published more than I although I like to think that an article in Psych Science carries the weight of several others. BUT... how many publications or degrees one has is totally irrelevant and I have been a bit disturbed by references to expertise here and on Greg's blog. I commented on it earlier.
There are times when the arguments someone makes are faulty because the arguer assumes that their layperson's view is accurate. Greg's & Stephanie's did not fit that category
In that vein, there are lots of scientists who don't think that people outside their field have anything to say and professors who teach BS pop psych as if it is fact (e.g., "multiple intelligences" and "learning styles"). IMO, they shame our profession. It seems to me that you have the intelligence to rise above these idiots.
2 - Regarding the use of language, my comments were in no way meant to imply the you are obligated to teach anyone, but I would think that you would want to avoid misunderstandings. If you don't say what you mean, you're saying something other than what you mean. This venue actually makes it more, not less, important that one's language is clear, especially given that much of the discussion is with people who do not have an education in one's specific field.
3 - Small sample sizes are not just an issue of power and my greater concerns are with the fact that your samples were not representative and grossly unequal.
4 - "I think restriction of range (college students) was the key culprit."
?? What did you think I meant by "limited sampling methods"??
I'm sure the Wonderlic is very reliable. I questioned its sensitivity, not its reliability.
5 - You called your equipment test a "pilot study" and used it as evidence to support the need for practice. That's what prompted the WTF response.
6 - Obviously, I didn't proof my post well enough (I switched GPA and IQ). However, you failed to answer my question and, excuse the all-caps, but I really want to get your attention: IT IS AN IMPORTANT QUESTION.
I'm not interested inhow you tested Spearman's hypothesis.
This is what I asked:
"Why wasn't there a statistical test of this mediation? The stepwise regression is not sufficient given that B is not zero in the full model, but it is easily tested. Why did you fail to report the standard errors of the Bs so that readers could test this themselves? Would you provide this information?"
You claim to have found a mediation effect, but you provided inadequate evidence to support it.
So, I will ask again.
Since you did not publish a statistical test of this effect, would you please provide the standard errors of Bs for each of the three regression equations related to the effect you reported in your paper?
7 - You've got more experience with reviewer comments than I, but I have found that they vary much like the quality of our colleagues: from highly constructive to total crap. For example, I ran an entire experiment to please a reviewer (although I knew it was unnecessary). The editor sent the new submission to only one reviewer who agreed it was totally unnecessary and the first draft was published.
Jason, where can I get a tribble that purrs and vibrates?
Barbara: You've not only mistaken me for a man, but you seem to think that I have more credentials than you. I don't. I'm ABD and you have published more than I although I like to think that an article in Psych Science carries the weight of several others. BUT... how many publications or degrees one has is totally irrelevant and I have been a bit disturbed by references to expertise here and on Greg's blog. I commented on it earlier.
Have you also noticed that, if someone sides with Bryan, they automatically get an honorary PhD in Agreeswithmeology? If you agree with Bryan, your credentials are above reproach. If you disagree, you'd better have twice as many degrees and published papers at least, or your position carries no weight whatsoever, no matter how well evidenced that position is. (Sorry if this seems outright rude, Bryan -- but I don't recall a single instance where you've emphasized or questioned the credentials of your anonymous supporters.)
As for the tribbles, if you're not planning on being anywhere near Deep Space Station K7 any time soon, you could mail-order one from Think Geek. They have direct supply lines from Cyrano Jones. :)
Bryan,
You frequently state that you have demonstrated the validity of "race differences" or the effect of "race" on IQ or other variables. Yet in your methods section, you accepted your subjects own selection of "race" from a list. Since you did not establish what "race" means, other than a subjective "racial identity." Whatever results you have obtained, therefore can in no way be said to have any genetic basis whatsoever, but show instead differences between "racial identities," which are social constructs built around varying experiences of social inclusion/exclusion, stigma/acceptance, poverty/comfort, etc.
The term "stereotype threat" - an excellent term introduced by daedalus2u - is a good word for this, and leads into another area completely disregarded in your description of methods. You describe your subjects as students - ie there is a risk of a power dynamic between students/professor biasing your results, for better or worse. You did not discuss how the research was explained and presented to the subjects. They are no doubt aware of your history of studying "race differences in intelligence" and this may introduce an implicit "stereotype" threat for the self-identified "Black" students.
Also, you did not specify whether the test was administered by self-identified "White" or "Black" testers, but this may also bias your results - expectations, or imputed expectations may affect how people perform under the pressure of a "test" situation - do they believe the testers expect them to fail. Would a "Black" student perform better if the tester was "Black"? These are interesting questions, which might throw a lot of light on your results, but which are not addressed by you.
Scotlyn: Well said.
Oh Jason, look at the first page of my article Stephanie posted.
It's not clear to me that Elsevier would approve of it being linked here.
That was my only point.
I have lot of respect for Elsevier and their completely cool CR policy. I don't want it violated.
To the extent I own a limited CR on this article; I released it.
I'm not pulling no CR card to hide. In fact, blog on all my published articles as it will help my publish or perish stats! Please do!
I still don't see your point, Bryan. It was not reproduced, distributed or used commercially. It was merely linked-to. If anyone had, say, uploaded the PDF to their website, or printed it out and mailed it to someone, or if, say, they used a page of it in a full-page ad in a newspaper for instance, then I could see your concern. And even where lines were directly quoted, you might like to research the Fair Use doctrine.
Talk about out of nowhere...
I thought the citation thing was way in the past, but I have to say this:
If your institution counts citations in blogs toward tenure, I've lost whatever respect I had for it (the default for me is "total respect").
Ah stereotype threat, finally!
I've been biting my tongue a bit (I know that will be ironic) here just in case some heavy hitter stumbles by.
But, I'll say it: The stupidity of the whole idea is laughable, especially when the manipulation doesn't reduce the gap and can't explain why asians and jewish people score higher.
A cheesey instructional manipulation is going to explain 100 years of data? Ok.
Gots no proof of this here as I am not going to get into this whacky literature. All we have is the test of time. We may perhaps all be dead, but in 10 or 20 years, no one will be doing research on ST.
It's one of Jensen's "blind alley" explanations.
Agree completely with replies that I am presenting no evidence on the point. This is my commentary for lols on an area I perceive as being cheese.
Jason, you can be rude. For some reason your comments never really motivate me to click the reply button.
Barbara -- I am honestly really sorry for assuming you were male. I have tried to write he/she gender neutral stuff throughout all these blogs.
No excuse though.
Good luck with your degree, and a psych science pub kicks major ass (I've been rejected there!).
If you are on the job market soon, I'd be happy to write you a letter of reference given our interactions so far....!
Gimme some time to reply, esp requests for more stat analyses. My wife is going to divorce me if I keep obsessing over this, so I'm trying to find some balance between life and the interwebs. She happens to be out now, so I can post some more!
Lols. No, my department doesn't even accept impact factors from SCI (well, they would if it something they published, but they don't when it's something I published. I suspect anything-- even a garage sale ad in the paper would count for them re promotion, but not me. But, that's a long story..).
75% of them voted against me for tenure and my final dossier is now under review at the university level.
Many people like google scholar as a measure of prestige better than impact factors. Publish or Perish uses it to calculate tons of stats on authors and journals.
The point is, I bet this blog will be picked up in the p or p database, and so will artificially inflate my statistics.
Meaning: I really think impact factors are much better than google scholar.
Jason. This is on the title page of the article Stephanie links to in this blog thread:
This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for ****internal**** non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or ****posting to personal, institutional or third party
websites are prohibited.***
The parts I asterisked make it not clear to me whether linking to my article by a third party is a CR violation.
I honestly don't know if this is fair use-- the differnece between posting the actual PDF third party or a link to it.
I think if I had posted the link, it would not be a VR violation, because I am not a third party.
But I dunno (send, lawyers, guns and money?)
Scott, you make good points and I think I promised to reply to you earlier, but haven't. If wife comes home this post may end abruptly. And, after this, I need to scoot for awhile...I don't have time to really proof this post so please ignore any stupid typos.
I have nowhere here or in my paper ever said that race differences are genetic.
I am agnostic on this issue (recall I was called out for being a wimp by not coming to a conclusion now.)
But, should I link to an older blog of Greg's where he says he is AGNOSTIC on whether man evolved from apes (perhaps not a verbatim quote, but he said he was agnostic regarding his belief on the point)?
Greg can be agnostic on an issue and it doesn't mean something fishy is going on. I can be agnostic too.
I will stipulate that the vast majority of studies on race and IQ use self-reports of race.
This has biological (race may be wholly a cultural construct, but skin color at least is biological) and social / cultural baggage. It's not for sure 100% accurate as a measure of whatever is the biology of race.
But, race is a measured variable just like any other variable in science.
And anything measured can be assessed by two simple concepts: reliability and validity.
A measure for a scientific paper has utility if it's reliability is at least .70.
There are books on different types of reliabilities one can calculate.
Self - reports of race would have "internal consistency" reliability. Meaning, were a person to participate in 3 unrelated studies, he or she is very likely to check the same race box each time. Finding one person who claimed he was white to screw my study would not be a confound!
So, even if that happened, it would be just a source of error variance. The result would be to lower the reliability of race as a measure. If enough people did this, race would be so unreliable that it could not statistically correlate with anything else. That doesn't happen in the literature.
Race is also reliable in an inter-rater reliability sense.
Me, you and someone else are the judges. Show us pictures of randomly selected people in the USA.
We have three options. The person is white, black or neither (e.g., Asian or too hard to tell because perhaps the person is mixed race).
The correlation between our ratings would be a measure of reliability and I have no doubt it would surpass the .70 barrier (I doubt anyone's done this study and I am not going to search the literature for a cite, so call it my intuition and weight it as such).
Conversely, I also have no doubt the IC reliability would be less than 1.0
There are other ways to measure reliability, but above I think is reasonable to address part of your comment.
Validity is a tougher issue, and I will post more later.
The short answer is I believe self-reports of race correlate strongly with whatever it is that causes the biology of race (even if we don't know what is yet). The clustering studies linked on Greg's blogs are good examples.
No, race cannot be measured with 100% accuracy. But, nothing can. Further, we can specify how much error exists in the measure with a number. We do this all the time with other variables and we use those variables in scientific research.
I'd bet the R and V of self-reports of race is higher than the R and V of the blood pressure test as most doctor's offices.
Most important first:
Regarding giving you time to answer my request for more analysis, I did not ask for more analysis. I asked for the standard errors of the Bs you reported in your paper. You MUST have calculated them since they were necessary for the statistical tests you reported. If you don't want to provide them just say so, but be prepared for questions about why that is.
I appreciate the offer for a reference, but I have a job and doubt that a letter from someone with which I exchanged comments on a blog would carry much weight.
Ah stereotype threat, finally!
??? Do you actually READ the comments to which you are replying? Stereotype Threat was discussed in one of the early comments on Greg's blog - I mentioned the comment it here last night. Scotlyn has simply summed up comments made elsewhere (some by scotlyn) because they were obviously not addressed.
"Stereotype threat" is "heavy hitting"? It's introductory social psychology!
I am not even going to touch the rest of your comment, but I'd brace myself for some hate mail if I were you. If you want to know why some commentors accuse you of racism, it's comments like that.
Finally, links are links. They have no content. They are citations with a physical connection to the article which saves the reader time - that's it. The CR question is a dead horse.
If the SEs arent in the paper I have to either dig up the hard copy analyses at my office or re-run.
I will re-run, but give me time.
On the fly comment: I think the baron and kenney test of mediation is complete in my paper (i.e., unless you claim the analyses is incorrectly ran, presenting the SEs only let you verify that I ran the stuff correctly).
That said, they should have been reported. The biggest thing I had to do for acceptance was drastically cut the length of the paper. I don't know if that's an excuse, but I concede they're not there.
Later!
Oh and I won't debate stereotype threat here. It's too much crap to bother wasting my time on (which is why I am not surprised it's in an intro text. I bet mismeasure of man is cited there too).
You guys win re any claims you want to make about ST.
Last comments on ST:
This year's ISIR conference ended last week or so. I did not attend. It does though represent state of the art in IQ research (even if you deem the research to be all crap).
So, what the field thinks is key/important high priority is well-represented by the papers at the conference.
Here's the entire PDF file of all papers this year.
Exactly one on ST.
A meta-analysis showing publication bias explains most the effect (gosh, that's surprising).
Greg and Step will immediately dismiss it as racists protecting an agenda.
But, there it is (and it is a meta-analysis).
Page 68:
http://www.isironline.org/meeting/pdfs/program2009.pdf
Btw, seeing ST and Mismeasure cited as good ideas for suggested readings in IQ is what prompted me to--fatally-- make my first one sentence post in bizarro world.
Actually, Bryan, I have only two comments on the description of the meta-analysis you link to. First, nowhere does it say that publication bias explains most of the effect of stereotype threat. Second, it merely says that stereotype threat is not the only factor in the testing differences. This is hardly surprising.
What is surprising is that you seem to continually mistake "doesn't explain all the difference" for "doesn't explain any of the difference." As with SES. As with parental preference for gendered toys.
The way that you tested for mediation is incomplete.
Kenny's website on mediation analysis discusses testing the significance of the difference between B1 in the regression and B1 in the multiple regression. Read the section titled "Measuring Mediation or the Indirect Effect".
http://davidakenny.net/cm/mediate.htm
The reason I keep harping on this is that although cognitive psych is my content area, I am primarily a methodologist; my MA is in quantitative (experimental) psych (My PhD, when I've completed the dissertation, will be in cognitive and perceptual sciences).
More later; sad I can't stop posting here...new year's eve even.
I scoured through the B&K article when doing this study; checked and double checked. It's possible I screwed it up-- if so, that's bad enough to require an erratum, and I will do it -- submit it to the journal-- if you show that I messed up. I will even credit you / cite you for pointing out the error (that would look nice on a vita were I considering someone in the job market).
If we keep interacting and you keep coming up with expert points I will offer to write a glowing letter of rec (if indeed you are on the market). I'm an HR prof and very good at these and I can write the truth if our interactions lead to that (plus some in field are ok with my expertise, and maybe two or three have read something I've done so it might be helpful-- if it's not, that's cool too).
But I really do need time to dig through my files find it, reframe myself re the analysis.
Why I think I am right; I also spent a fair amount of time on this issue for another paper-- but it was not the simple B&K article; it was the one on moderated mediation and mediated moderation (holy fuck!). I think I actually understood though, so I'd be surprised if I flucked up the simple analysis here.
Happy new year!
OMG, this is funny.
Brian Pesta, the evidence that you are a moron mounts. (That means increases.)
Didn't see any kind of spamming, though I was in the blog exactly when Greg was whacking threads to save face.
Seriously? Do you know what this tells us about how you think? OMG...
Hold still while I 'splain:
A given post on that thread (or any active thread on a controversial topic) may or may not have appeared AFTER I freed it from moderation (probably about 10 of them over a couple of days in this case). A given post may have been posted, I deemed it, as is my right, inappropriate and deleted it. That's maybe a dozen or so posts, over a period of a couple of days. A given post may have been held in moderation and deleted by me, never seeing the light of day (and therefore never being seen by anyone). Another dozen. Another unknown number never even made it into the moderation bin because they were caught by a spam filter.
So there is a flux of posts going in and out of given active thread. All the time.
(Aside: When you cried out that you had seen several posts deleted, that was impossible because only one or two visible posts had been deleted during your self proclaimed window of observation, and that led me to conclude tentatively but probably correctly that you knew about comments that you could have not known about unless you had posted them or knew someone who posted them and were in communication with that person. Like a friend or a colleague in, say, Colorado.)
Back to the point: That there is a flux of posts and that it is fairly complicated and that it runs over a period of time are all obvious facts, and anyone with two neurons to rub together would know that.
But you observed the blog for various brief periods of time and during one of those periods of time you saw (so you claim) a couple of posts that i deemed inappropriate (which is my right) disappear.
And from this you develop accusations, a model of conspiracy, and some very bad blood.
Bryan, listen: This is now how to make and interpret observations. For instance, when you close your eyes the world does not actually disappear. It is still there. There are often multiple things going on that you may not, at first, know about, and just because you find out about something later in the process of observing and studying data does not mean that these later factors have to be ruled out.
This explains your inability to understand context in your research. You have a simplistic view of development and genetics because you can not comprehend a larger and more complex world ... THE larger world in which biological processes play out. You are a simple minded fool, and an annoying one at that.
Example:
The idea that a three point difference in IQ between males and females (which is highly questionable to begin with) could be considered explanatory of centuries of exclusion of women from certain jobs is THE FUCKING MOST ABSURD AND OFFENSIVE, MISOGYNOUS AND STUPID IDIOTIC MORONIC (DID I SAY STUPID) THING I've heard from anybody in WEEKS. And I'm a BLOGGER!!!!!
OMFG!!!!
Sorry, your ongoing idiocy over the question of comment deletion reminded me of that charming moment.
Stephanie.
I spent about 30 seconds searching the pdf and then scanning the abstract. I won't be reading the article unless it's for entertainment purposes at my leisure.
I've seen this happen too often with other effects, even when the topic is not so politically sensitive as race and IQ. Fortunately, with time, the effects become historical blind alleys.
A classic example is my dissertation: negative priming. This was the shits in the 90s. People published like crazy on it, and it was advanced as the new explanation for cognitive aging. My dissertation was skeptical of the leading theoretical explanation for the effect. I had 2 or 3 RT tasks that on balance showed a problem with the theory.
Holy crap was that a mother of an effect to find. It's 10 ms in good cases (a mean difference in RT 3 times quicker than an eye blink, yet people thought it was a domain global explanation for many things in cognitive psych). I dunno; there might be 100 articles now on NP (very few since the 1990s as the area is and should be – I think— dead).
I bet / assert that more than half of these 100 studies could not be replicated would anyone care to try. I would be surprised if my dissertation would completely replicate, and I know exactly that I did nothing suspect. Something fishy's going on in this literature, but welcome to social science.
You'd have to be wholly suicidal to do this stuff with race and IQ (if you are arguing the non-pc track). I don’t care what anyone says about Rushton but no way no how would he keep his job (tenured / union or not) if he smegged his data. I bet with all I have that Jensen and Rushton are more honest scientists than the vast lot of us.
I am not claiming fraud in the NP data, but when tenure / promotion / grant $'s on the line, shit tends to find a way to get published. Effects tend to emerge as significant, etc.
Other effects that have been just as much bullshit (and these are just the ones I am aware of in a narrow sub-discipline). Try not to think about white bears?! (The ironic effects of repetition); the 2d4d stuff (fascinating; I linked to it on greg’s blog, but it’s oversold for sure. Watch what happens with time). Stereotype threat. Guaranteed. Don’t waste your time researching it if you need a publication stream that lasts til tenure.
Scientists are biased as shit. The only thing saving humanity is for every one who shares your bias, the other doesn’t. So, only closer and closer approximations to truth survive. Thank God.
Again, I believe this so strongly that I have the damn word tattooed on my arm. There’s nothing noble about being a scientist except that if your stuff stands the test of time and individually-biased-on-average-but-washed-out-at-large critical review, it becomes science.
(Said differently: the expected value of bias in any science area, caused by individual scientists’ quirky biases, is zero across time. This is a variant of one of Spearman’s ideas—the guy who discovered g, and invented the concept of reliability to better measure things; oh, and factor analysis too!). He killed himself in late life by jumping out a hospital window…
Done for today; g’night!
Greg, shall we call a truce for the holiday and bash this out while crappy college football is on tomorrow?
Happy NY.
Bryan, I'm not concerned with your feelings about how the research on stereotype threat is going to come out. I am concerned with the fact that you're misrepresenting research like the meta-analysis you claimed said stereotype threat was a dud.
"I scoured through the B&K article when doing this study;"
Bryan, the B&K article you cited was published in 1986. Things pop up in peer review, discussion, and revision, like the fact that a test can and should be done to determine the probability that c - c' is significantly different from zero. Taking a look at the articles which have cited this in the more than 20 years since its publication would have been a good idea, as would consulting a more recent book on the topic. In this case, as with most statistical issues, Googling it will do the job. Statisticians tend to be pretty computer savvy.
Perhaps you fell prey to the myth that there is nothing much to discuss in statistics, but that's certainly not true given that, unlike pure mathematics, it is a science.
Your comment makes me wonder, though, if you read the page to which I linked. He discussed the issues raised by others and the simple test which would settle the question pretty clearly.
I cannot determine if the mediation effect is robust or not without those few missing bits of info, but I am certainly happy to calculate it for you. Should it turn out that your effect is supported, I see no reason to amend or note anything.
If, however, the effect does not hold up, it's not the end of the world by any stretch of the imagination. As mistakes go, failing to run that last (albeit important) step is about the best kind you can make. An erratum would fix the problem - hopefully before someone else catches it, although I am shocked that nobody has so far - and perhaps give you more hypotheses to test.
Run the numbers or provide them so that I can do so. If the effect disappears, we can talk about what to do with that info. Until then, it's just an unanswered question hanging over your findings.
Since Greg is still seething, I'll add a comment from among those I jotted down, but did not post. In regard to this:
"The idea that a three point difference in IQ between males and females (which is highly questionable to begin with) could be considered explanatory of centuries of exclusion of women from certain jobs is THE FUCKING MOST ABSURD AND OFFENSIVE, MISOGYNOUS AND STUPID IDIOTIC MORONIC (DID I SAY STUPID) THING I've heard from anybody in WEEKS."
Given the amount of eye-rolling that Greg has done over commentors in the past month, this is a pretty strong statement. I wonder if the problem here is not that you believe that "a three point difference in IQ between males and females could be considered explanatory of centuries of exclusion of women from certain jobs", but that you don't think that you have said this. You have.
The comment I did not make last night was in response to your statement "You don't seem to understand how a small mean difference at a group level can lead to big under-representation/over-representation at the extreme ends of a distribution?"
Okay, setting aside the wording problem (statistics don't lead to data point distributions), mean differences in normal distributions (which, by definition, have moderate variance) describe the amount that two similar distributions overlap. Small effect sizes means small differences. They don't get larger as you move toward the tails.
But the bigger problem is the one Greg's been harping on: You've basically said that gender or race - take your pick (A) is correlated with IQ (B) and occupation (C), which are correlated with each other, therefore B explains the correlation of A with C.
Having just defended a paper in which mediation analysis is key, I hope you can see the error in this logic. If not, here's an analogy:
Weight is correlated with height, which is correlated with the number of years of education, therefore taller people have more education because they weigh more. That is, of course, insanely stupid.
If this thread hits 100 posts, will Stephanie get a 1-up? So far as I can tell, she's gotten many already.
I re-ran the stuff for table 4 (the ect mediation analysis). All numbers are correct in the table.
It hit me why I didn't report standard errors-- I have standardized beta weights in the table (I prefer them as the un-standardized ones are hard to interpret, and since my ECT variable is a z score anyway).
So, that's why no standard errors.
Here's the un-standardized values:
path a (race to ect) the b is .776 and the SE is .176
path b (ect to IQ) is -2.36 and .414 respectively.
Sobel's test produces a value of 3.49, with a p value of .000428
So, I think it's clear I have a nice mediation effect, despite a small sample size for black students (no erratum needed!).
The Sobel test has crappy power? If so, despite that, the conclusion is the same as mine in the article; mediation existed.
More later; thanks for yer postings here.
Hi Stephanie, and HNY.
If we continue this here, and I suspect we will, could you post it on a blog that has better comment-editing capabilities than this one.
Or, if my unfamiliarity with blog stuff is preventing me from using whatever editing features are available on this blog, please advise.
Even cutting and pasting from Word is hard here, at least for me.
Bryan -
I am not a fan of Blogger commenting, either, however I have given you resources so that you can choose a platform of your own. Do you expect Stephanie to do all the work for you?
You really aren't so stupid as to be unable to start a blog, are you? Of course not, you are a white male.
Mike, bite me.
I'm only here because this is a thread -- not started by me-- devoted to exposing my racist ignorant professional career.
If someone did that to whatever you do for a living, I'd bet you'd be posting replies in defense.
If these are your resources, ban me or wait til the thread dies; otherwise stfu.
"Sobel's test produces a value of 3.49, with a p value of .000428"
Very nice! Bullet dodged and you must be very relieved. Low power or not, that confirms the finding you reported.
As I said originally, I am still very hesitant to draw any conclusions from the method, but that was (potentially) a very serious problem you can check off of my list.
I've got to concentrate on other things now and I think we've all had our say, so I probably won't check the comments of this post again. If you want to reach me, I link to my blog with every comment.
No; I'm good. Perhaps it's time to put this puppy to rest.
Thanks Mike for hosting.
Just stay away from Laden's blog. Those Scienceblog bloggers tend to stick together and gang up on intruders.
Post a Comment