[Taxacom] Open review as a wiki

[Bjarte Henry Jordal]

I think Doug Yanega and some others have posted some brilliant ideas
about publishing taxonomic papers on web. Most of these suggestions will
definately be a standard practice in 30 or 50 years, but I really hope
the change in practice will take some momentum quite soon, preferably
within the next 5-10 years. If there is not too much opposition in the
taxonomic community it should be possible to make several of these
suggetsion mandatory in the ICNs. The problems associated with an open
wiki system is mainly technical but these should be fairly easy to solve.

First, potential reviewers must be registered experts that recive some
sort of message from the taxonomy server. The system should of course
not be completely closed to other researchers that may watch the
process, but there should not be too many involved in the actual review
process. Imagine what will happens when a new fox or Papilio butterfly
is described and perhaps hundreds of happy amateurs writes their lengthy
opinions based on very limited data (yes, I know some amateurs are the
real experst, but not the majority). A 'half open' Wiki system will
therefore be easier to administrate. RSS sounds like a good option,
especially if that system can cope with directed messages to potential
reviewers.

Secondly, I believe the review process should be shorter than one year
in most cases. Larger revisions are indeed time consuming and need
longer time for review, but remeber that the majority of taxonomic work
contains a handful of new descriptions which is done more quickly.
Waiting for a year or two for 'acceptance' for such small papers would
be very annoying. Also, I am not too happy with Doug's suggestion that
very preliminary work should be posted. It is certainly a good idea to
post the manuscript if only a few samples are still not examined etc,
but it would be a real headache if reviewers should struggle with
something half written or scribbled notes that are nearly
incomprehensible. This stage works better in a scratchpad environment. I
don't see the problem with taking a few more months to write a more
complete manuscript before posting it for review? It might be time
saving for the author, but I would not be reviewing half-done work
because in most cases it would be a really frustrating task.

Finally, how do we ensure long term funding for running and
administrating a taxonomic webserver? Any opinions about who and where?
Because a single webjournal/database is our goal, it will need
continuous attention such as GenBank.

Bjarte

-- 
Bjarte H. Jordal, PhD

Associate Professor in Systematic Entomology
Museum of Natural History, University of Bergen
Musèplass 3
NO-5007 Bergen

Phone:  55582233
http://www.bio.uib.no/pages/forsker.php?pid=1481&lang=E


Doug Yanega wrote:
> Neal Evenhuis asked:
> 
>> At 10:29 AM -0700 3/31/08, Doug Yanega wrote:
>>> the model some of us are advocating
>>> would be OPEN review - real-time, online, non-anonymous - in the
>>> fashion of a Wiki. When all the criticisms of a submitted work have
>>> been dealt with (by accommodating the valid criticisms and dismissing
>>> the inappropriate ones)
>> This is an admirable dream. I've been dreaming this too but being an 
>> editor for over 30 years I've come up with many questions relating 
>> to problems that ultimately need the human factor. If you've got 
>> some answers, let me/us know.
>>
>> 1. How would it be accomplished to take in the vagaries of 
>> reviewers' habits including not responding in a timely manner?
> 
> How do various journals presently accommodate? For most, they solicit 
> reviews, ask for a commitment to a deadline, and therefore skip 
> potential reviewers who can't commit. Most print journals probably 
> have only a slightly longer turnaround time than Zootaxa, where 
> (unless I'm mistaken) the major time-saving step is the vastly 
> shorter interval between approval and publication. However, bear in 
> mind that  traditional reviews such as these only see a manuscript 
> when the author(s) believe it is absolutely ready to publish!
> 
> Imagine an author who takes 4 years to get a manuscript ready for 
> submission - if it takes another 4 months for it to get reviewed, 
> revised, and printed, then that's great. Now, imagine that same 
> author puts the first draft online for review right at the start; 
> with feedback occurring throughout the process, and the streamlining 
> influence of having a template to work from, it would very likely NOT 
> take four years to finish it - maybe it would take only two years, 
> let's say - in which case a statement like "Oh my god, we can't 
> possibly tolerate a review process that takes two years!" is 
> comparing apples to oranges (in this case, the manuscript got into 
> print FASTER as a result of online review). As I said earlier, the 
> submission of a manuscript would be part of the process of making 
> specimen requests - it is certainly not uncommon for a study to take 
> years to complete, if the amount of material being examined is 
> substantial (for an entomologist, it would not be unusual to have to 
> look over thousands of specimens from dozens of institutions). If 
> reviewers can make comments during the time the author is examining 
> specimens, then that leaves a VERY long time window, and even the 
> laziest reviewer can't say that a year or more is not enough time to 
> find an opportunity to make comments.
> 
> Remember also that the open review process will involve potentially 
> every conceivable qualified reviewer, in which case an additional 
> question should be asked: "Which is better, (A) a paper which goes 
> from initial submission (as a finished work) to print in two months 
> but is seen by only three reviewers, and is printed with errors and 
> problems which a larger group of reviewers could have detected and 
> fixed BEFORE it was printed, or (B) a paper which goes from initial 
> submission (as a draft) to print in a year but which has been 
> scrutinized by dozens of reviewers, including all of the pertinent 
> authorities?" Just ask yourself how many times you've seen something 
> incorrect or inappropriate come out in print that you KNOW you could 
> have prevented had you been involved in the review. Under this 
> system, everyone would *have* that opportunity, and everyone would 
> benefit!
> 
> I would argue that many of the *nastiest* problems facing the 
> taxonomic community can be traced to the lack of a universal standard 
> of review; I believe firmly that making the review process as lengthy 
> and exhaustive as possible is a desirable goal, with benefits that 
> far outweigh any perception of encumbrance. High-quality work that 
> meets the highest standards will, by its nature, pass through the 
> system quickly and easily; work of lower quality will be held up only 
> just as long as is necessary to bring it up to the higher standards 
> desired.
> 
>> 2. What happens when there may only be one specialist who could give 
>> a good review and he/she is out of email contact for four weeks 
>> while in the field?
> 
> Four weeks is nothing when a manuscript is taking shape over a year 
> or two. Surely, some time during the development of a work there will 
> be an opportunity for everyone with an interest to make their 
> opinions known to the author(s), other reviewers, and referee.
> 
>> 3. How are time limits to review set (three weeks, a month?)-- or 
>> does the manuscript languish online for months or years on end in 
>> hopes "someone" will notice it? The web is passive after all.
>> 4. Will there be some sort of registration of interested parties so 
>> that they can get email automatic notices when their speciality 
>> taxon group comes up for review?
>> 5. How do you know that everyone that is a specialist in a 
>> particular group knows a manuscript is open for review?
> 
> These three are linked: there *would* be registration of interested 
> parties, with automatic notification. There would also be a 
> backtracking feature that would indicate who has not seen their 
> notifications yet. Each incoming manuscript would have an impartial 
> referee assigned to it, basically making sure the discussion stayed 
> civil, that as many of the registered interested parties had made 
> comments as seemed practical, and there would be a checklist of 
> criteria that determined when something was ready to be accepted; 
> most of these would be objectively-defined content (type deposition, 
> etymology, non-homonymy, diagnoses, images/illustrations, etc.) that 
> are BUILT INTO the template (so anything left blank would be 
> glaringly obvious), but the crucial subjective decision would be "Are 
> there any criticisms which have not been adequately addressed or 
> refuted?". Ultimately, in such a system, I believe that the only 
> manuscripts that would "languish" are those for which the author(s) 
> failed to fill out the template completely in a timely manner, or 
> could not (or would not) deal with criticisms adequately - remember, 
> again, that the manuscripts would be online for a long, long time. If 
> an author tries to conceal ongoing projects and refuses to submit 
> them for review until they are "ready", then *those* authors may find 
> the resulting review phase to be onerous, but only because they did 
> not utilize the system in the manner it was intended.
> 
>> 6. Bookmarking the "Open Review" site as one's web browser home page 
>> might solve some of these problems but in reality, someone who sets 
>> their web browser home page to default to news of a bunch of 
>> taxonomy manuscripts available for review online would be a VERY 
>> weird person who doesn't have much of a life. Any solution to this?
> 
> Many times the argument has been raised that people couldn't engage 
> in this sort of activity AND continue to review manuscripts for other 
> journals; my counter is that this website would REPLACE each and 
> every one of those print journals, at least insofar as taxonomic 
> publication - there wouldn't BE any other manuscripts being published 
> in any other journals that would require your reviews. If this 
> website was the only place in the world that taxonomic publications 
> were being submitted, reviewed, and published, then every taxonomist 
> in the world would by default go there to submit their own works, and 
> read those of their colleagues. I know that if I had a manuscript 
> online and turned on my computer to see a message telling me "There 
> are 7 new comments on your manuscript today" that I would certainly 
> pay attention; another message saying "There are 3 new submissions in 
> your target subjects, by authors X, Y, and Z, the second of which has 
> specifically requested your review" would also get my attention. If 
> each of those 7 comments took me only a few minutes to address, then 
> it wouldn't be much of a disruption of my daily routine, nor would 
> the 3 manuscripts, since I could look them over at my leisure, and 
> make comments and corrections in a similarly casual fashion.
> 
> Which would be easier for you as a reviewer: (A) to receive 10 
> manuscripts a year, received at random intervals, with deadlines by 
> which you needed to read through the entirety of each, with only ONE 
> opportunity to make ALL of your comments (many of which would be 
> completely redundant with the comments from other reviewers), and NO 
> opportunity to see any responses to your criticisms before it was in 
> print, or (B) to have 10 manuscripts available online simultaneously, 
> over a period of as long as a year or two, where you can edit them 
> directly, and any corrections or improvements made by other reviewers 
> would appear instantly (which is what a Wiki is all about), and you 
> can take however little or as much time as you want to review any 
> part of any manuscript, and get direct and timely feedback from the 
> author(s) when you submit a question or criticism?
> 
> Using that kind of system, even the most time-stressed of us would, I 
> believe, find the review process far more friendly and useful, both 
> as a reviewer and as an author. It might also result in more 
> completed works involving co-authorship, as a by-product of the 
> collaborative nature of the review process (e.g., if author X submits 
> a revisionary work describing new species in Genus A, and author Y 
> turns out to also be revising that genus, and they have a few 
> additional species that author A does not have, then the wiki process 
> would allow them to compare notes and collaborate online, rather than 
> each competing to see who can publish their revision first).
> 
> Hopefully, this is sounding a little less like a dream and more like 
> something attainable?
> 
> Sincerely,