[Taxacom] How good is peer review...

[Weakley, Alan]

So, here's _one_ possible distillation of this.

Bad papers get published in all "systems".  Peer review vs. not, open access vs. paywall, print vs. e-journal -- all complex issues with no black-and-white.  

Perhaps a standard by which to judge all these controversies is what benefits or damages the advance of science?  I think exploitative journals publishing bad papers for a bribe (essentially) are morally reprehensible - but I'd be interested to know how much real damage or "impact" on the advancement of science they have.  I'll assert that we each in our own subfield can recognize real research from junk, and we ignore the junk.  Certainly, if not ignored immediately, it falls by the wayside very fast.  And, there is nothing new here - there has always been a lot of junk, and it has always been ignored.  Wheat and chaff, part of the business of science.  Modern digital world = lots more junk and lots more value:  sort it out!  So, in my opinion, they do little damage (though I would be glad to evaluate contrary evidence).  If, indeed, they do little real damage, we as a community need to be careful that any counter-measures (policing) do not cause more damage than the problem.  

Peer review is sometimes held up as a kind of holy preserver of quality.  Yet, we all know that peer review is often done negligently and casually, and sometimes with actual scientific malfeasance:  the suppression of new radical ideas by the established experts who perceive a threat to their world view and their dominance.  The history of science is littered with examples.  Here's one {pardon a clip from Wikipedia!}:  "Lynn Margulis attended the University of Chicago, earned a master's degree from the University of Wisconsin-Madison in 1960, and received her Ph.D. in 1963 from UC Berkeley. In 1966, as a young faculty member at Boston University, she wrote a theoretical paper entitled The Origin of Mitosing Eukaryotic Cells. The paper however was "rejected by about fifteen scientific journals," Margulis recalled".  

Even when not directly suppressive, peer review is fundamentally elitist:  "well, that is an okay paper, but it doesn't warrant publication in OUR journal, which screens for only the most impactful papers".  But, are we really very good at judging that beforehand? {cue Margulis}".  Or, "that paper's by X, a leader in the field, so must be important and clearly publishable and important"; if the same paper came in from a 25-year old, would it receive the same review and decision?  So, there is a conservative and status-quo component to peer-review which can be antithetical to science at its best, admittedly balanced against its benefits.  A retort might be, "well, Margulis's fundamental new paradigm DID get out", but the appropriate re-retort is "how many major ideas didn't, or were (more) significantly delayed?"  Probably no easy answer to that.

An additional point.  Perhaps we are all good at telling real scientific merit aside from any considerations of the author's ethnicity, gender, continent, institution, quality of English, or journal of publication - but I doubt it.  Note that this study used "fictitious authors ... affiliated with fictitious African institutions" created by "randomly combining Swahili words and African names with generic institutional words and African capital cities. My hope was that using developing world authors and institutions would arouse less suspicion if a curious editor were to find nothing about them on the Internet."  I think the real, tacit assumption here was that the African names and African institutions maximized the likelihood of skepticism about the quality of the paper and therefore if even despite that and its questionable content it was accepted, that was the strongest possible indictment of the journal's review policy.  

Perhaps an equally interesting study (and with appropriate control!) might have been to submit essentially similar (but more sophisticated and potentially scientifically valuable) papers to a reputable set of journals with different author ethnicities, genders, career stages, levels of fame or respect in the field, and institutions -- and judge how they were peer-reviewed (and accepted for publication or not) - I think there have been some studies showing gender bias.

Shouldn't we err on the side of promoting that ideas be published (=promulgated openly to the judgment of other scientists) by whatever means, and then judge them on their merit, over time?  

Alan

-----Original Message-----
From: taxacom-bounces at mailman.nhm.ku.edu [mailto:taxacom-bounces at mailman.nhm.ku.edu] On Behalf Of Bob Mesibov
Sent: Saturday, October 05, 2013 5:59 PM
To: TAXACOM
Subject: Re: [Taxacom] How good is peer review...

Philipp Wagner wrote:

'In nomenclature the open access journals are not the great problem I think. A much more serious problem are online-only journals. With modern technology every private person is able to develop a journal and can write dubious ideas without any review. This is in fact the problem.'

The problem is larger than private-person journals, as the spoof paper described in Science demonstrates. The ICZN moved on this problem and amended section 8 of the Code to allow electronic-only (valid) publishing only under certain conditions (see, for example, http://www.pensoft.net/journals/zookeys/article/3944/).

Do you think these restrictions are inadequate?
--
Dr Robert Mesibov
Honorary Research Associate
Queen Victoria Museum and Art Gallery, and School of Agricultural Science, University of Tasmania Home contact:
PO Box 101, Penguin, Tasmania, Australia 7316
(03) 64371195; 61 3 64371195

_______________________________________________
Taxacom Mailing List
Taxacom at mailman.nhm.ku.edu
http://mailman.nhm.ku.edu/mailman/listinfo/taxacom

The Taxacom Archive back to 1992 may be searched with either of these methods:

(1) by visiting http://taxacom.markmail.org

(2) a Google search specified as:  site:mailman.nhm.ku.edu/pipermail/taxacom  your search terms here

Celebrating 26 years of Taxacom in 2013.