|
Post by marchesarosa on Feb 10, 2011 10:52:06 GMT 1
"j" Feb 10, 2011 at 9:03 AM at Bishop Hill concludes
"The lasting outcome of all this is that a paper that implied near-catastrophic evolution of temperatures in the Antarctic got published in Nature, put on the front cover, and bounced around newspapers and TV, whereas the paper showing that this previous result is almost entirely wrong did get published, but only in a much less prominent journal and without all the PR in the media."
-------
And remember "Nature", according to Judith Curry's post above at #33, practises the "pal review" system whereby authors nominate their own reviewers! Strong stuff, these cliques!
How much different would the reception of the original Steig paper in "Nature" have been if subject to secret "hostile" review by real statisticians! Would it have been published at all, one wonders?
|
|
|
Post by louise on Feb 10, 2011 16:39:27 GMT 1
|
|
|
Post by marchesarosa on Feb 10, 2011 17:23:31 GMT 1
Poor Steig. He admits he's no statistician but still thinks he can blind people with statistics that he doesn't actually understand himself. He should have gone to SpecSa - Steve McIntyre first.
|
|
|
Post by speakertoanimals on Feb 10, 2011 21:15:57 GMT 1
Seems to me he's just doing what any decent reviewer does -- says when there is something he is not totally au fait with.
Else where would you be -- climate scientists who can't review because they're not statistician enough, and statisticians who can't review because they aren't climate scientists enough.
And your mate Steve would get a bit tired if he had to review them all................
Plus Natures Peer-Review page says:
So, if they think they need a statistician, they go find one.
These claims about pal review sound a bit spurious anyway -- MANY journals ask authors to recommend reviewers. Not so you can get your mates to do it, but just because they sometimes have problems deciding WHO is most approriate to review if -- because if it's a specialist field, the relevant sub-editor might not know who best to ask. Hence many journals like to be given a bit of a hand as to who knows enough to do it.
Plus they don't usually use more than one that you recommend yourself. And in most cases (like the one being referred to here I think), the journal editors would have been well aware if one of their reviewers was author of the paper whose conclusions were being attacked in the submitted paper. In fact, you could say this is the HONEST way of doing it -- if you are saying paper A is full of crap, let the author of paper A have his say, and if you can answer his objections, then the editors can see that.
Equally, if the editors see that the author of paper A is being a right pain in the arse just to try and block a paper that disagrees with him, then they can see that as well. Even if he writes umpteen pages, and says reject it is a big pile of pooh -- the editors csan totally ignore that if they think he isn't playing fair.
Because it is the editors that make the final decision, based on what the referees say, and on how you answer their criticisms. Even if they are all your mates, and all just say -- brilliant, no problems anywhere -- why assume that the editors are too daft to spot this. They'll just get another review from someone who does have some useful comments to make.
If you're going to assume that all climate scientists are corrupt and unfit to review each others papers, then you're screwed either way! Who do you propose should review them, your mates on the various web sites? I don't think that would agree with the usual editorial standards of Nature..........
|
|
|
Post by marchesarosa on Feb 10, 2011 21:20:51 GMT 1
I have already said I have no objection to hostile review - so long as reviewers put their name to their work. It is secret PAL review that is so potentially corrosive of quality.
|
|
|
Post by speakertoanimals on Feb 10, 2011 21:22:36 GMT 1
This isn't necessarily the case -- in some areas, you are expected to submit data etc as supplementary material, it isn't necessarily JUST the printed version.
THe point is always going to be that at some point, something has to be taken on trust. Not all papers published are correct, and never will be -- which is why we wait to se what others come up with, and try to establish a consensus. If A has a conclusion that no one else manages to get anything else to agree with, then that's that -- a lone conclusion, with no external support.
IF someone wnated to totally falsify a paper and results, they could and always have been able to. Except in the end, they get caught out, when no one else agrees with their conclusions or can replicate what they say they have done. That is how science works. So we have detailed review by specialists before anything is published, then we have that paper left to fight it out with what everyone else is publishing before a consensus is established. A system with many checks to try and ensure that the consensus that emerges is a good one.
Easy to criticise this, of course, if you've never actually done any of it. I have, both authored and reviewed, and it is hard work. And we don't even get paid for it....................
|
|
|
Post by speakertoanimals on Feb 10, 2011 21:27:12 GMT 1
Bollocks. The editors know who's who, and they have the final say. Plus the point about reviewing is that since it is anonymous as regards the author knowing the identity of the reviewer, it then gives the reviewer MORE opportunity to say what they really think. Otherwise, no one junior would ever dare disagree with anyone more senior. And let's face it, someone junior always loves the opportunity to do just that!
Again, even if you recommened reviewers as an author, doesn't mean they use them all. ANd if we don't trust the editors, who would you have us trust M? O let me guess, your mates on the blogs.............................
|
|
|
Post by marchesarosa on Feb 10, 2011 21:36:20 GMT 1
You can't have it all ways, speaker!
"IF there are supposedly thousands of climate scientists out there who all agree on CAGW why couldn't one of them have done the peer review? It's because the number of climate scientists who are also competent statisticians is roughly zero."
so said Justice4Rinka on Bishop Hill
I agree.
|
|
|
Post by marchesarosa on Feb 10, 2011 21:58:37 GMT 1
Speaker apparently thinks everything in the garden of climate peer review is lovely in the best of all possible worlds!
How nice! No reforms necessary then. Continue as usual!
|
|
|
Post by speakertoanimals on Feb 11, 2011 2:01:32 GMT 1
Which is daft -- some are, some aren't, depends on exactly what they do. Just as in any other area of science -- some researchers know a lot about stats, others don't.
I'm not saying it's perfect M, I'm just saying that I don't think your criticisms are valid, and comparing them with the procedure in other areas of science, it all seems pretty normal. So, as I pointed out, the supposed pal review that you seem to think is so heinous is actually quite usual, and as far as I can see, is there mostly to give journal editors a hand.
Even when you apply for a research grant from RCUK, you get to list 3 possible reviewers, and they usually use one you supplied. It's so that the applicant can't say -- none of the reviewers you picked knew what they were talking about. O no, says RCUK, but you picked one of them, so how can that be?
The other point of course, is WHO do you propose to review stuff apart from other researchers in the same field? Someone from outside generally won't know enough to do the detailed job required.
And the point to remember, when it comes to journals or research grants, is that it isn't the reviewers making the final decision, it's the editors or panel, guided by the reviews. The author or grant applicant gets a chance to answer criticisms, and in the case of papers, the process can go round the loop several times, with the editors calling in extra reviewers if they aren't satisifed.
And finally, it is very difficult to get a true measure of the peer review process unless you have been involved in it, on both sides, as both author and reviewer.
And reading some other blogs, I found a nice comment -- what is missing in this whole shebang is the concept of an HONEST disagreement. Which does happen quite often amongst honorable researchers, but difficult to have when personal accusations start flying, as seems to be the norm in the anti-AGW camp..............
|
|
|
Post by mak2 on Feb 11, 2011 15:09:47 GMT 1
As Winston Churchill said about democracy, it is not a good system but it is better than all the alternatives. The same could be said about peer review.
Just because a paper has passed peer review does not mean that it is right. What is worse, the system may suppress research that is right but goes against the prevailing orthodoxy.
|
|
|
Post by speakertoanimals on Feb 11, 2011 15:46:52 GMT 1
Another common misunderstanding. Actually, loads of research that goes against the prevailing orthodoxy (as some for some reason want to brand things) IS published. The criteria are, IF you are going to say something that goes against what many people think, you must be able to give a good enough reason, because there are usually damn good reasons WHY the prevailing orthodoxy is what it is.
Its the ole problem of what the process looks like to those outside it. Science isn't against new ideas, becfause after all, where most of us do research is right at the edge where little is known, hence not knowing and new ideas is the order of the day!
So, if someone came up with a new idea today that explained as much as relativity, but disagreed with it, rather than some suppress it as a heresy, people would love it! And in fact such papers have been published.
The real problem here is that many of those who claim to be opposing the orthodoxy actuallyn just have very poor data or arguments, and that is why they don't get published. Just bad papers, not heretical papers................
|
|
|
Post by speakertoanimals on Feb 11, 2011 16:05:25 GMT 1
The point really about review, whether it be for conferences, or journals, is who is to judge what is most interesting to the research community from a scientific point of view apart from that community itself?
I can see arguments that when it comes to handling the interface between research results and the sort of discourse that policy makers need to engage in, then I can understand widening the net, to make sure that what climate scientists, say, is supported by what statisticians have to say as well. But when it comes to day-to-day journal papers and reviews,the only people who are willing and qualified to do the job are the researchers themselves.
What it certainly isn't is the biased view that some people would have us believe. They seem to think that letting researchers review their own subject area would let a whole research area just go off on one and con the rest of us, which is just a rather sad conspiracy theory. Ditto the pal review stuff -- sounds so suspicious to those who don't do it, but actually rather benign in my experience.
SO, seems to me these people aren't really interested in trying to improve the scientific publishing process, just throwing mud at climate science and hoping enough sticks. As was the case with all those hacked emails. More mud, more mud, seems to be the rallying cry.
|
|
|
Post by louise on Feb 11, 2011 20:08:24 GMT 1
|
|
|
Post by mak2 on Feb 12, 2011 11:04:15 GMT 1
Science isn't against new ideas I did not think that it was, but individual scientists are often against new ideas that conflict with their own. Nobody likes to be proved wrong. I feel that the global warming issue has become so polarised that scientific objectivity has been lost. Much of what is published is propaganda rather than science.
|
|