[Taxacom] Open review as a wiki

[Doug Yanega]

Steve Gaimari wrote:

>A couple of issues I'd like to address. I'll start with a random 
>thought, then a couple of specific points.
>
>1) With the 1000's of scientific journals out there, publishing all 
>kinds of scientific research on a variety of topics, wouldn't it 
>potentially hurt taxonomy to distance itself from mainstream 
>science? A typical reader of one of these scientific journals would 
>potentially NEVER read a taxonomic paper, and would NEVER gain an 
>appreciation for the science of taxonomy.

I would argue the exact opposite: most of the journals publishing 
taxonomy of non-charismatic groups are read primarily by taxonomists 
and systematists. Most have a negligible impact factor. Moreover, 
ordinary folks from the public are not generally going to be reading 
ANY of these print journals. Having all of taxonomy in a single 
website addresses all of these issues at the same time. (1) Any 
non-taxonomist scientist who has specific taxonomic needs would have 
a SINGLE resource to refer to, instead of having to hunt around, 
making their life easier. That means that an author working on 
obscure groups, and used to publishing in specialist journals that no 
one other than specialists read, will be getting their work exposed 
to a broader audience because the way things are right now, their 
potential readership *has no easy way* to even FIND their 
publications, but they WILL be directed to their publications when 
everything is all in one place (e.g., if some agronomist wants a key 
to aphids, they could presently have an ordeal trying to track 
something down - but it'd be easy if there was a centralized 
taxonomic resource). (2) If all taxonomy is published in a single 
source, then it will level the playing field for taxonomists AND give 
the field more prominence; instead of dinosaur taxonomy being 
published in journals with an IF of 40, while most other taxonomic 
journals are no higher than a 2, everyone would suddenly be 
publishing in an online journal with an IF of 20-30 (because anyone 
citing ANY taxonomic work would boost the rating for this one 
resource; that's how IF works). That would *increase* the perception 
of the value of taxonomy to science. (3) With a universal, 
international web resource, the accessibility to ordinary people 
would be vastly more than present print journals. Here's an idea: 
have the home page with a "Species of the Day" randomly selected from 
among those with color photos available, to encourage non-scientists 
to bookmark and/or visit the site regularly. One *could* easily do 
things like that to make the site more than just a place for 
taxonomists.

>2) "bear in mind that  traditional reviews such as these only see a 
>manuscript when the author(s) believe it is absolutely ready to 
>publish!"
>
>I've reviewed enough papers to know that even when the author(s) 
>believe something is ready to publish that it may not be so. In my 
>opinion, a reviewer should devote his precious time to a manuscript, 
>aimed at assessing and improving it, when it is ready. Too much time 
>is wasted on manuscripts that are clearly not ready for this stage 
>in the publication process.

There are many dimensions to this you're neglecting. What about 
preventing an author from wasting *everyone's* time (including their 
own) with something that is flat-out WRONG? Suppose author X has a 
new taxon to describe, or desires to make a nomenclatural change. If 
they spend a year working on this, and they wait to submit it until 
they have everything completed, what if it turns out their new taxon 
is NOT new, or their nomenclatural change is in violation of the 
Code? Do you honestly believe it is better for them to learn that 
AFTER spending a year of their life on it, when they could have been 
told BEFORE they did so? Which is more embarrassing: being told you 
made a mistake on day 1, or being told after months of labor? 
Wouldn't you feel better as a reviewer *preventing* someone from an 
embarrassing and time-consuming mistake rather than correcting them 
long after the fact? Right now, a substantial number of new names 
make it into print that are synonyms - how do 
authors/reviewers/everyone else NOT benefit if, in the future, 
would-be synonyms never make it past the first draft stage?

Do you believe it's better to publish a key that has been seen only 
by three reviewers, who had it for only one month, than to make a 
draft available online to everyone in the world, so it can be given a 
thorough work-out over the course of several months? Do you believe 
it's preferrable to believe a type is "lost" (because it's not where 
it's supposed to be) and have someone point out that they know where 
it is AFTER you've gone ahead and made a number of incorrect 
decisions (e.g., a neotype designation) based on the absence of that 
type, rather than having them inform you right there at the start of 
your study that the "lost" type is now housed elsewhere? Do you 
believe it's useful to have two or even three authors privately 
working up descriptions of new taxa, only to find out AFTER one of 
them publishes, that all of them were describing the same taxon 
independently? Several of my colleagues, including close personal 
friends, died or quit while working on revisions - and everything 
they had not yet published (in one case, 30 years of research on a 
single genus that has still never been revised) was lost. Other folks 
have had their computers crash, and lost their data and their work, 
sometimes (though rarely) irretrievably. These are all problems that 
arise from taxonomists working in isolation until AFTER they have 
completed their work. How does the scientific community benefit from 
that sort of thing, which we could help avoid if manuscripts were 
composed online, in public?

>3) "Imagine an author who takes 4 years to get a manuscript ready 
>for submission - if it takes another 4 months for it to get 
>reviewed, revised, and printed, then that's great. Now, imagine that 
>same author puts the first draft online for review right at the 
>start .... etc."
>
>I'm sorry - this made no sense to me. First, how many times do you 
>expect an expert to review and rereview a manuscript?

How many times does an expert "review" a Wikipedia article? It could 
be HUNDREDS of times in a single week, but each "review" after the 
very first one lasts about 5 seconds (reviewing a WP article, for 
those of you who aren't familiar, means displaying the differences 
between the version of the article which you first read, and the 
present version of the article; as long as no changes were made that 
were plainly detrimental, your "review" is done - and detrimental 
changes can be undone in about 3 seconds). You and I are using 
completely different concepts of what "review" means. I 
simultaneously "review" nearly 100 Wikipedia articles (mostly 
taxonomic) every day, and it rarely requires more than 15 minutes 
total - and most of that is removing vandalism, which wouldn't be 
happening on a taxonomy website. Also, if you're so concerned with 
efficiency, how is it a good use of reviewers' time to send out three 
copies of a manuscript, and then have three reviewers each spend 
several hours making an identical set of punctuation, grammar, and 
formatting corrections? Isn't it more efficient to have ONE copy of a 
manuscript, being edited simultaneously by ALL the reviewers, *plus* 
the authors? Wouldn't you, as a reviewer, prefer to be able to 
*discuss* revisions with the author, instead of only being given one 
chance to comment - and potentially be ignored?

>Does each specialist want to take the time to continually massage 
>someone else's research?

That's an unfair way of phrasing the question. It depends on whether 
you like "win-win" scenarios, or accept the concept of a "manuscript" 
that has no fixed publication date, and continually evolves as new 
information and concepts come to light (that's what a wiki *is*!). 
Every work, by every author, will be perpetually "massaged" by both 
the authors and reviewers, to the benefit of all (the only things 
with fixed dates will be new taxon descriptions, when they are 
accepted and registered). I don't know about you, but if I came 
across a peculiar fly in my collection, I'd be much happier if the 
key I was using to ID it was a key whose last revision was ten HOURS 
ago, rather than ten YEARS ago. If I found that my fly was new, or 
that a couplet was unclear regarding it, I'd be much happier if the 
author of the key could be contacted immediately, and - if necessary 
- that species could be described and *added to* the existing 
manuscript and key within a matter of days or weeks, instead of 
having to wait for a completely new revision of the genus to get 
published. That new taxon would have its own date of registration 
(maybe even a different author), independent of the manuscript in 
which it appears. If I had 5 specimens of a rare species in that 
revision, from 5 localities not listed therein, it would be nice if I 
could "massage" the work to include those distributional records, 
since no print journal would publish a paper on my 5 specimens. If I 
had a host record for a species in that manuscript with no recorded 
hosts, I'd rather "massage" that information into the manuscript, 
since no print journal would bother to publish something so trivial. 
Do you see my point here? What you treat as a negative, can be looked 
at as a positive. There are TONS of information that a taxonomist, 
working in isolation, with a handful of material, either won't be 
able to include in a traditional publication (either because they had 
no access to it, or because it added too much text that the print 
journal editor thought was superfluous), or which will only come to 
light AFTER their work is printed. If we get away from that 
18th-century mentality of publications having to be permanent and 
immutable, then we can make all that OTHER information available to 
the entire scientific community as soon as it becomes available, 
rather than being limited to just those things which a print journal 
is willing to publish. If I were the author of such an online work, 
I'd be ECSTATIC if I had reviewers helping me by adding information 
that I might never otherwise have had access to. Again, the topic 
here is "open review as a wiki" - and the wiki model is utterly 
different from the traditional model.

>When does this constant improvement give authorship to the reviewer?

Since when does correcting grammar/spelling, or asking questions, 
constitute authorship? Authorship is based on contribution of 
*content*. I know from personal experience that even in the print 
journal world, if a review expands to the point where a reviewer 
CONTRIBUTES actual content instead of merely fixing problems, they 
sometimes DO get invited for co-authorship. In a public forum, how to 
give credit where it's due will be even MORE obvious because it will 
be there in black and white for everyone to see, in perpetuity in the 
document history.

>And for this draft submission to act as a de facto specimen request? 
>I don't think so. If someone is tackling a revision, they better 
>know the community of experts in a group from whom they should 
>request specimens, and which collections house important materials.

There's a vast portion of the taxasphere for which there ARE no 
living experts, nor single "exemplary" collections. I could probably 
rattle off a list of some 50 insect families that fit that category - 
not every group in existence is well-studied, with *known* experts 
and *known* repositories. If you wanted to revise a genus of 
cave-dwelling beetles from France, would you KNOW that there are 
specimens in the Illinois Natural History Survey that you might need 
to see? There is a HUGE gap between what a collection's *known* 
strengths are, and what that collection actually contains - and we 
could narrow that gap by making the revisionary process public right 
from its earliest stages. Your own institution is a perfect example: 
there's a LOT of valuable material in the CDFA collection that no one 
ever asks to see, and it's a tremendously underutlized resource, 
because so few people know what your collection contains; that 
*wouldn't* be the case if everyone in the world who had specimen 
requests made them in a public forum where every collection manager 
or curator in the world (including you) could see them. Also, 
soliciting material from only major institutions does NOTHING to 
alleviate the low percentages of expertly-IDed material in smaller 
institutions' collections, and does NOTHING to help prevent their 
demise from disuse. With so many collections struggling these days, 
we need to do more to get them and their holdings involved in the 
larger endeavor, and including them in a forum like this would be a 
great help, instead of everyone just sticking with the handful of 
collections that "house important materials". How can you *know* 
there are no "important materials" in a given collection unless you 
ASK? Don't smaller institutions deserve to have experts examine and 
curate THEIR holdings, too? If we can agree that broadcasting 
material requests is beneficial, how can you not also agree that it's 
more beneficial to make such requests early, so the material is all 
on-hand while the revision is being done, rather than finding out 
about all the specimens one COULD have seen only AFTER the work is 
"published"? Is it not better to have a few dozen paratypes from 
several different institutions rather than only a solitary holotype 
specimen?

>4) "many of the *nastiest* problems facing the taxonomic community 
>can be traced to the lack of a universal standard of review"
>
>Having such an open-review system would not create a universal 
>standard of review in any way that I can see. Rather, it would 
>promote more of a free-for-all, completely unstandardized. The 
>review process, no matter whether it starts from the rough draft 
>stage or after a manuscript is finished, will still be A) dependent 
>upon the community interested enough in that group to work with the 
>manuscript, B) dependent upon the individual personalities and 
>opinions of those reviewers, and C) dependent upon the subjective 
>opinion of the assigned "impartial referee" as to whether criticisms 
>were adequately addressed.

I speak mostly of two very specific things, and the multitude of 
ramifications thereof: there is presently no universal standard that 
all taxonomic publications must be reviewed to begin with, and no 
standard that all nomenclatural acts published must comply with ICxN 
requirements. That is the "universal standard of review" I'm 
advocating, and it won't work without open review. Basically, lots of 
journal editors and reviewers don't KNOW the respective Codes well 
enough to ensure compliance of every reviewed work, AND many things 
are published that aren't reviewed at ALL. We need to stop both these 
things from happening. Ever.

>Sorry - taxonomy is not a template science. Yes, there are certain 
>elements that are critical such as those listed above, but an 
>acceptable manuscript is not one that simply fits into the template. 
>What a boring science this would be if it was a matter of filling in 
>a template! Maybe we could all just fill in the blanks on a 
>web-form, and that will be the paper submitted for review? (sorry - 
>yes - that was sarcasm...).

We are not in disagreement here; I was only pointing out that a 
template would offer a minimum standard that needed to be met before 
a work could be *considered* acceptable (there is no such minimum 
standard now). Hopefully, conscientious authors and reviewers and 
referees would see to it that most works exceeded these minimal 
requirements!

>As for addressing reviewer criticisms - isn't this impartial referee 
>the same thing as a journal editor, who weighs the criticisms and 
>the authors responses when deciding on publication?

No. A journal editor has no one watching HIM. He doesn't have to 
*explain* how or why he rendered any given decision on any given 
point (yes, most good editors will do so, but they don't *have* to). 
A journal editor could be biased or foolish or ignorant, and no one 
would ever know. That is NOT going to be true in a public forum. Same 
with reviewers. You may not be aware of it, but even in Wikipedia, 
some contributors get banned from participation, if they repeatedly 
demonstrate bias, foolishness, or ignorance that is disruptive to the 
community (I even know one such case where the banned individual was 
a taxonomist!!). You'd be amazed at how much more reasonable people 
can be when there is the potential for public condemnation of 
UNreasonable behavior.

Modes of behavior, and of what costs/benefits there are for authors 
and reviewers, are utterly different in a print-journal-based world, 
and under a wiki model.

Sincerely,
-- 

Doug Yanega        Dept. of Entomology         Entomology Research Museum
Univ. of California, Riverside, CA 92521-0314        skype: dyanega
phone: (951) 827-4315 (standard disclaimer: opinions are mine, not UCR's)
              http://cache.ucr.edu/~heraty/yanega.html
   "There are some enterprises in which a careful disorderliness
         is the true method" - Herman Melville, Moby Dick, Chap. 82