Dr Dennis Jensen is a Liberal MP from WA who has a PhD in “materials science and physics”.

He has a blog and has written an interesting article on the peer review process. I find his example pretty disturbing (I have looked at powerline/cancer studies and meta-studies online and found the majority find no effect, the opposite of his hypothetical, yet extremely detailed, example – strawman?) and he is obviously pushing the climate change denier angle.

I found his blog after reading this http://nofibs.com.au/2013/08/08/jensen-skips-climate-change-forum-but-faces-our-reporter-at-morning-tea/

Comments?

http://www.dennisjensen.com.au/blog/120/peer-review-my-view In full:

There has been a lot of discussion about science, peer review, and the basic scientific process. There is a lot in the way of misunderstanding of the process, so I thought that I would put out in layman’s terms what the process is, and what it means.

Most people tend to think that the process of peer review essentially proves that a study that is written up in a peer-reviewed paper is correct and accurate. This is not the case. Peer reviewers are supposed to look at the methodology used in a scientific paper to ensure that the methodology used is sound. There is no analysis of the results and conclusions specifically as part of the process.

So, let’s take an example. Let’s assume we are examining the issue of cancer clusters associated with high-tension powerlines. I am not going to make any judgements as to whether this exists or not, it is simply an example. In this example, let’s hypothesise that there is actually no effect.

Let’s say that there are 100 epidemiological studies undertaken. Due to the nature of statistics, we assume that 80 studies will show no effect, 10 will have positive correlation (more cancer associated with powerlines) and 10 will have negative correlation (there are less cancers associated with powerlines).

Now, those 100 papers will all be completely scientifically legitimate. Now, assuming all are submitted (usually, most of those with no correlation would not even be submitted), the issue of editorial bias will enter into it. The fact is, scientific journals are a business to make money, so the more noteworthy the paper, the more journals will be sold. So, the editor will not even submit (in general) the non-correlation papers for review, they will not be accepted. Of the others, those showing a positive correlation will likely be sent to reviewers, and possibly 2 of those showing a negative correlation (negative correlation fairly interesting, but not as striking, newsworthy etc. as the positive correlation).

So, what you end up with is 10 papers showing a positive correlation, 2 showing a negative correlation that are published (just dealing with editorial policy here, not the reviewers themselves). Now you get another scientist who conducts a meta-analysis of the peer reviewed literature on the subject, and comes to the conclusion that after a survey of the literature 83% of peer-reviewed papers on the subject of cancer clusters associated with high-tension powerlines show that there is an increased incidence of cancer for those living close to those powerlines. This then becomes the headline that you see in the newspaper, and all about an issue where there is no effect!

Now, let’s move to the issue of peer review. In many fields (particularly when you drill down) there is a fairly small number of researchers that are active. So, when a paper is submitted by one of the active scientists, it will probably be reviewed by one or more where there is a personal relationship, so that the paper will likely be accepted. A new entrant will have a far higher probability of a paper rejected. Indeed, this was found in a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected. This is where the peer review in essence turns to “pal review”.

Worse even than “pal review” is where editors and reviewers of a specific scientific predisposition will attempt to reject any paper with a contrary viewpoint. Phil Jones wrote of Intergovernmental Panel on Climate Change Report AR4 and of some peer-reviewed papers he did not agree with: “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!” This is quite stunning, and shows the potential for perversion of a process that even when operating without systemic bias has its problems.

Hopefully, this will give a bit of a better idea of the peer review process, and of some of the shortcomings of the process, and that caveat emptor exists even regarding peer-reviewed literature. Remember, the consensus view of the fraud of Piltdown Man as being the missing link was accepted science for decades.

> I find his example pretty disturbing

It wasn’t an example, it was an analogy.

Well ok, but he calls it an example!

How about: I find his use of words disturbing…

podzol said:

Well ok, but he calls it an example!

How about: I find his use of words disturbing…

that works.

podzol said:

Well ok, but he calls it an example!

How about: I find his use of words disturbing…

Why?

What’s that famous quote, “science progresses by one dying scientist at a time”, or something like that.

He probably could have used a real example and used the theory of plate tetonics.

>>Why?

Seriously?

Where do I start… how about the strawman argument in his analogy (which he calls an example, and cites he is not making a judgement but then goes into great detail over it). Surely a better analogy would be something less controversial like “red smarties make people blink more”?

Or how about “a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected.” Got a ref for that one doc?

Then he is stunned: “Worse even than “pal review” is where editors and reviewers of a specific scientific predisposition will attempt to reject any paper with a contrary viewpoint. Phil Jones wrote of Intergovernmental Panel on Climate Change Report AR4 and of some peer-reviewed papers he did not agree with: “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!” This is quite stunning, and shows the potential for perversion of a process that even when operating without systemic bias has its problems.”

However these have been refuted here: http://www.ucsusa.org/global_warming/science_and_impacts/global_warming_contrarians/debunking-misinformation-stolen-emails-climategate.html

>>“In a later e-mail, Phil Jones references two other papers he didn’t hold in high esteem. “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow – even if we have to redefine what the peer-review literature is!”

>>Yet, the papers in question made it into the IPCC report, indicating that no restrictions on their incorporation were made. The IPCC process contains hundreds of authors and reviewers, with an exacting and transparent review process.

>>The fact that groups opposing action on climate change are crying “conspiracy” shows how desperate they are to discredit scientists.

Or maybe even: “Remember, the consensus view of the fraud of Piltdown Man as being the missing link was accepted science for decades.” That was one blokes theory that was accepted for 40 years. Not thousands of scientists holding true the same theory. Farrrkkk.

>>Why?

Seriously?

Where do I start… how about the strawman argument in his analogy (which he calls an example, and cites he is not making a judgement but then goes into great detail over it). Surely a better analogy would be something less controversial like “red smarties make people blink more”?

Or how about “a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected.” Got a ref for that one doc?

Then he is stunned: “Worse even than “pal review” is where editors and reviewers of a specific scientific predisposition will attempt to reject any paper with a contrary viewpoint. Phil Jones wrote of Intergovernmental Panel on Climate Change Report AR4 and of some peer-reviewed papers he did not agree with: “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!” This is quite stunning, and shows the potential for perversion of a process that even when operating without systemic bias has its problems.”

However these have been refuted here: http://www.ucsusa.org/global_warming/science_and_impacts/global_warming_contrarians/debunking-misinformation-stolen-emails-climategate.html

>>“In a later e-mail, Phil Jones references two other papers he didn’t hold in high esteem. “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow – even if we have to redefine what the peer-review literature is!”

>>Yet, the papers in question made it into the IPCC report, indicating that no restrictions on their incorporation were made. The IPCC process contains hundreds of authors and reviewers, with an exacting and transparent review process.

>>The fact that groups opposing action on climate change are crying “conspiracy” shows how desperate they are to discredit scientists.

Or maybe even: “Remember, the consensus view of the fraud of Piltdown Man as being the missing link was accepted science for decades.” That was one blokes theory that was accepted for 40 years. Not thousands of scientists holding true the same theory. Farrrkkk.

Where do I start… how about the strawman argument in his analogy (which he calls an example, and cites he is not making a judgement but then goes into great detail over it). Surely a better analogy would be something less controversial like “red smarties make people blink more”?

Reread it.

“So, let’s take an example. Let’s assume we are examining the issue of cancer clusters…..”

That’s an obvious analogy. He’s telling the reader that it is a hypothetical case. He really couldn’t be more straightforward.

Or how about “a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected.” Got a ref for that one doc?

His words:

“A new entrant will have a far higher probability of a paper rejected. Indeed, this was found in a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected. This is where the peer review in essence turns to “pal review”.

I really can’t find anything amazing about that above statement. It’s human nature coming to the fore. He’s also writing a blog, and is not exactly going to be giving a annotated reference.

As to the rest, I find the claims and counter claims rather tedious. I’m just a plodder of an engineer. Identify a potential problem, propose a solution.

sibeen said:

Where do I start… how about the strawman argument in his analogy (which he calls an example, and cites he is not making a judgement but then goes into great detail over it). Surely a better analogy would be something less controversial like “red smarties make people blink more”?

Reread it.

“So, let’s take an example. Let’s assume we are examining the issue of cancer clusters…..”

That’s an obvious analogy. He’s telling the reader that it is a hypothetical case. He really couldn’t be more straightforward.

Or how about “a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected.” Got a ref for that one doc?

His words:

“A new entrant will have a far higher probability of a paper rejected. Indeed, this was found in a recent study, where many papers which had been published in the journal in the past, when submitted by an unknown author from an unknown institution were rejected. This is where the peer review in essence turns to “pal review”.

I really can’t find anything amazing about that above statement. It’s human nature coming to the fore. He’s also writing a blog, and is not exactly going to be giving a annotated reference.

As to the rest, I find the claims and counter claims rather tedious. I’m just a plodder of an engineer. Identify a potential problem, propose a solution.

It is ALL about getting a reaction.

I can’t believe I am saying this about Dennis Jensen, but he does have a point about publishing statistically-based results. There’s a clear publication bias for positive result – and for filing away negative results – which means that quoted significance levels may be hugely overstated. I’ll look up a better article on this.

OTOH, he is quite wrong to apply this to the climate situation because most of the _primary_results in that field are not derived by statistics but by physical modelling, paleoclimatic evidence etc. If conventional climatology is wrong it will be because these models are wrong, or the evidence is misunderstood, not because the statistical significance is badly estimated. (There seems to be a persistent myth amongst denialists that the primary argument in climatology is ‘current rate of warming is statistically unusual therefore It’s all CO2.)

Here’s the (IMO) more considered article on the substantive issue raised by the OP.

http://theconversation.com/putting-psychological-research-to-the-test-with-the-reproducibility-project-7052

>>As to the rest, I find the claims and counter claims rather tedious.

Tedious, ouch. As you asked me why I found his words disturbing and I replied.

You find the IPCC report and email controversy tedious?
You find the comparison of the Piltdown Man controversy with the climate change denial controversy tedious?

Not sure why you queried my opinion in the first place…