December 30, 2014

Am Nat Switches to Double Blind Review in the New Year




Over the next few months, the American Naturalist will be making the transition to a form of double blind reviewing--reviewers will no longer be informed of the authors' identities in an effort to minimize explicit or implicit biases triggered by author names or institutions. As a speaker on the problem said, "Implicit bias produces 'micro-inequities'" that are not always easy to detect.

This transition began at Evolution 2014 in Raleigh, North Carolina, where workshops run by About Women in Science explored issues facing early career women. A presentation for the post-graduate workshop is here:
http://www.clfs.umd.edu/biology/dudashlab/Evol2014%20Post-Tenure%20Workshop%20ImplicitBias%20Final.pdf

And another on implicit bias here:
http://www.clfs.umd.edu/biology/dudashlab/Resources%20to%20learn%20more%202July2014.pdf

The editorial board of the American Naturalist met during the annual meeting and discussed ways we might address the problem of implicit bias. At that time, it was proposed that we try a form of double-blind review. We would blind the authors' identities in the system but not attempt to anonymize the paper itself.

Many people have commented that double-blind review won't work in the sciences, at least in its most exacting form. I tended to agree because I started out in publishing at a true double-blind social science journal and that's how I understood its workings. To anonymize a paper, authors were required to write their papers in such a way that their connection to their previous work was disguised. This produced some very unclear and often turgid writing since it required writing in a passive voice and in the third person to avoid indicating who had done the author's previous work in the same area. Reviewers would criticize the writing. Then after acceptance, the papers had to be thoroughly rewritten. This rewriting and re-rewriting is a labor-intensive process that puts the burden on the authors and requires extensive investments in journal office staff  (to police every page of 900 submissions and/or rewrite every successful manuscript after acceptance). However, the field itself sought anonymity and took the process of double blinding very seriously, in spite of the extra work. I'm using the past tense because even the humanities and social sciences are questioning whether double blind is "a fiction" in the "age of Google." https://www.insidehighered.com/news/2011/05/31/american_economic_association_abandons_double_blind_journal_reviewing

In science, there has always been the problem of how to disguise the origins of a paper from a particular lab or research group. The study organism, the methods, and the references to prior publications can make it all too easy to guess the principal investigator no matter how much a paper is rewritten. Research that has already been presented at conferences is readily searchable and might even have been heard by a reviewer. If authors post preprints, then a quick Google will reveal all. It's not enough to say that reviewers should be too ethical to Google the topic. Checking on the state of the topic or looking up particular references cited are natural parts of the reviewing process. Though not all papers would get "unmasked" in these ways, it does make rewriting and re-rewriting papers to achieve double blind purity a bad investment of everyone's time--and it would require considerable disruption of regular professional activity. If, however, the goal is to minimize unconscious bias, simply removing author names, email addresses, and institutions from title pages and correspondence may make a difference in leveling the playing field at very little cost. There is a claim that just increasing the uncertainty of authorship has a positive effect in leveling the playing field.
http://www.cs.utexas.edu/users/mckinley/notes/snodgrass-tods-2007.pdf

Increasing the use of forms of double-blind review in the sciences seems to be an idea that's spreading:



And so, the American Naturalist will experiment with redacting author identities throughout the system and in reviewer correspondence to minimize bias. Starting in January 2015, authors will be asked to provide title pages with no author names, affiliations, or email addresses. Authors will instead provide that information inside the submission system. They will also be asked to upload their acknowledgments separately during review.

It's taken awhile to figure out how to convert the system. It's just going to have to be a bit messy for a few months! It will be an interesting New Year.


Trish Morse
Managing Editor

February 7, 2014

The Awards of Asilomar #ASN2014


Don Abbott Postdoc Research Award:

Winner: 

Carl Boettiger, University of California, Santa Cruz
"Ecological management for an uncertain world: robust decision theory in face of regime shifts"

Runner-ups

Alex Jordan, University of Texas, Austin
"Reproductive foraging theory: spider males choose mates by selecting among competitive environments"

Benjamin Callahan, Stanford University
"Niche construction evolves quickly and repeatably in experimental microbial populations"


Ed Ricketts Student Talk Award

Winner

Marjorie Weber, Cornell University
"Merging phylogenetic and experimental methods to test hypotheses about the evolution of mutualistic defensive traits in plants"

Runner-ups

Jason Shapiro, Yale University
"The role of pleiotropy in horizontally transmitted mutualistic symbioses"

Rachael Bay, Stanford University
"Genomic differences reflect fitness over a small-scale thermal gradient"

Ruth Patrick Student Poster Award

Winner

David Harris, University of California, Davis
"Generating realistic species assemblies with a partially observed Markov random field"

Tweets of Asilomar #ASN2014

There was a hum of talk in person and on Twitter during the meetings, so I decided to capture a few for posterity--and to inspire the next one!

Rachael Bay (@RachaelABay)
Excited to be at the first ASN meeting in 12 years! #ASN2014

Holly Kindsvater (@HollyKindsvater)
Wow live tweeting a meeting is harder than I expected! Many interesting convos going on in life & online! #ASN2014

Ben Sheldon (@Ben_Sheldon_EGI)
Wi-fi problems during symp, but highlights tour-de-force presentations by de Meester and Nosil on microgeographic adaptation #ASN2014

Caitlin MacKenzie (@CaitlinInMaine)
#ASN2014 has pushed me out of my comfort zone -- both in model complexity & west-coast study systems. But, I love it!

Alex Perkins (@TAlexPerkins)
@cboettig thanks for #asn2014 debate coverage. sounds like a real brawl. we need these at every meeting!
  
Jenna Morgan Lang (@jennomics)
Some talks are SO hard to live tweet because the speaker is rapid-firing too much cool stuff. I'm looking at you, @TadashiFukami #ASN2014

Caitlin MacKenzie (@CaitlinInMaine)
Overheard at #ASN2014 "I'm definitely being stretched here, but stretched in a good way. Mental yoga."

Marc Mangel (@MarcMangel1)
Am Nat is my favorite journal and this ASN meeting is superb!
Do it again, Council
#ASN2014
@ASNAmNat

Luke Harmon (@lukejharmon)
Had a great time at #ASN2014. Thanks for all the zombie love. Shout out to Dan Bolnick for an amazing mtg and Trevor Price for the debate.


Marc Johnson (@evoecolab)
@ASNAmNat Regretting not going to #ASN2014. Sounded amazing! My type of meeting ...


Holly Kindsvater (@HollyKindsvater)
Andrew Beckerman: daphnia have locally adapted developmental plasticity. Sweet combo of common garden & genomics, much to chew on. #ASN2014


Jenna Morgan Lang (@jennomics)
.@boechera just gave THE most awesomest talk. I'm totally jonesing to read soil microbe/plant phenology paper now! #ASN2014

Holly Kindsvater (@HollyKindsvater)
Spent a satisfying afternoon working on #science with @dr_k_lo #ASN2014

Andrew Beckerman (@beckerhopper)
So long #ASN2014 and thanks for all the sunny science. Well done team @DanielBolnick

Holly Kindsvater (@HollyKindsvater)
Sorry to be leaving Asilomar today. Thanks for a great meeting @ASNAmNat ! #asn2014



Daniel Bolnick (@DanielBolnick)
If you didn't get a chance to buy one of Alex Wild's photo in the raffle / silent auction, visit alexanderwild.com & mention #ASN2014