December 30, 2014

Am Nat Switches to Double Blind Review in the New Year




Over the next few months, the American Naturalist will be making the transition to a form of double blind reviewing--reviewers will no longer be informed of the authors' identities in an effort to minimize explicit or implicit biases triggered by author names or institutions. As a speaker on the problem said, "Implicit bias produces 'micro-inequities'" that are not always easy to detect.

This transition began at Evolution 2014 in Raleigh, North Carolina, where workshops run by About Women in Science explored issues facing early career women. A presentation for the post-graduate workshop is here:
http://www.clfs.umd.edu/biology/dudashlab/Evol2014%20Post-Tenure%20Workshop%20ImplicitBias%20Final.pdf

And another on implicit bias here:
http://www.clfs.umd.edu/biology/dudashlab/Resources%20to%20learn%20more%202July2014.pdf

The editorial board of the American Naturalist met during the annual meeting and discussed ways we might address the problem of implicit bias. At that time, it was proposed that we try a form of double-blind review. We would blind the authors' identities in the system but not attempt to anonymize the paper itself.

Many people have commented that double-blind review won't work in the sciences, at least in its most exacting form. I tended to agree because I started out in publishing at a true double-blind social science journal and that's how I understood its workings. To anonymize a paper, authors were required to write their papers in such a way that their connection to their previous work was disguised. This produced some very unclear and often turgid writing since it required writing in a passive voice and in the third person to avoid indicating who had done the author's previous work in the same area. Reviewers would criticize the writing. Then after acceptance, the papers had to be thoroughly rewritten. This rewriting and re-rewriting is a labor-intensive process that puts the burden on the authors and requires extensive investments in journal office staff  (to police every page of 900 submissions and/or rewrite every successful manuscript after acceptance). However, the field itself sought anonymity and took the process of double blinding very seriously, in spite of the extra work. I'm using the past tense because even the humanities and social sciences are questioning whether double blind is "a fiction" in the "age of Google." https://www.insidehighered.com/news/2011/05/31/american_economic_association_abandons_double_blind_journal_reviewing

In science, there has always been the problem of how to disguise the origins of a paper from a particular lab or research group. The study organism, the methods, and the references to prior publications can make it all too easy to guess the principal investigator no matter how much a paper is rewritten. Research that has already been presented at conferences is readily searchable and might even have been heard by a reviewer. If authors post preprints, then a quick Google will reveal all. It's not enough to say that reviewers should be too ethical to Google the topic. Checking on the state of the topic or looking up particular references cited are natural parts of the reviewing process. Though not all papers would get "unmasked" in these ways, it does make rewriting and re-rewriting papers to achieve double blind purity a bad investment of everyone's time--and it would require considerable disruption of regular professional activity. If, however, the goal is to minimize unconscious bias, simply removing author names, email addresses, and institutions from title pages and correspondence may make a difference in leveling the playing field at very little cost. There is a claim that just increasing the uncertainty of authorship has a positive effect in leveling the playing field.
http://www.cs.utexas.edu/users/mckinley/notes/snodgrass-tods-2007.pdf

Increasing the use of forms of double-blind review in the sciences seems to be an idea that's spreading:



And so, the American Naturalist will experiment with redacting author identities throughout the system and in reviewer correspondence to minimize bias. Starting in January 2015, authors will be asked to provide title pages with no author names, affiliations, or email addresses. Authors will instead provide that information inside the submission system. They will also be asked to upload their acknowledgments separately during review.

It's taken awhile to figure out how to convert the system. It's just going to have to be a bit messy for a few months! It will be an interesting New Year.


Trish Morse
Managing Editor

February 7, 2014

The Awards of Asilomar #ASN2014


Don Abbott Postdoc Research Award:

Winner: 

Carl Boettiger, University of California, Santa Cruz
"Ecological management for an uncertain world: robust decision theory in face of regime shifts"

Runner-ups

Alex Jordan, University of Texas, Austin
"Reproductive foraging theory: spider males choose mates by selecting among competitive environments"

Benjamin Callahan, Stanford University
"Niche construction evolves quickly and repeatably in experimental microbial populations"


Ed Ricketts Student Talk Award

Winner

Marjorie Weber, Cornell University
"Merging phylogenetic and experimental methods to test hypotheses about the evolution of mutualistic defensive traits in plants"

Runner-ups

Jason Shapiro, Yale University
"The role of pleiotropy in horizontally transmitted mutualistic symbioses"

Rachael Bay, Stanford University
"Genomic differences reflect fitness over a small-scale thermal gradient"

Ruth Patrick Student Poster Award

Winner

David Harris, University of California, Davis
"Generating realistic species assemblies with a partially observed Markov random field"

Tweets of Asilomar #ASN2014

There was a hum of talk in person and on Twitter during the meetings, so I decided to capture a few for posterity--and to inspire the next one!

Rachael Bay (@RachaelABay)
Excited to be at the first ASN meeting in 12 years! #ASN2014

Holly Kindsvater (@HollyKindsvater)
Wow live tweeting a meeting is harder than I expected! Many interesting convos going on in life & online! #ASN2014

Ben Sheldon (@Ben_Sheldon_EGI)
Wi-fi problems during symp, but highlights tour-de-force presentations by de Meester and Nosil on microgeographic adaptation #ASN2014

Caitlin MacKenzie (@CaitlinInMaine)
#ASN2014 has pushed me out of my comfort zone -- both in model complexity & west-coast study systems. But, I love it!

Alex Perkins (@TAlexPerkins)
@cboettig thanks for #asn2014 debate coverage. sounds like a real brawl. we need these at every meeting!
  
Jenna Morgan Lang (@jennomics)
Some talks are SO hard to live tweet because the speaker is rapid-firing too much cool stuff. I'm looking at you, @TadashiFukami #ASN2014

Caitlin MacKenzie (@CaitlinInMaine)
Overheard at #ASN2014 "I'm definitely being stretched here, but stretched in a good way. Mental yoga."

Marc Mangel (@MarcMangel1)
Am Nat is my favorite journal and this ASN meeting is superb!
Do it again, Council
#ASN2014
@ASNAmNat

Luke Harmon (@lukejharmon)
Had a great time at #ASN2014. Thanks for all the zombie love. Shout out to Dan Bolnick for an amazing mtg and Trevor Price for the debate.


Marc Johnson (@evoecolab)
@ASNAmNat Regretting not going to #ASN2014. Sounded amazing! My type of meeting ...


Holly Kindsvater (@HollyKindsvater)
Andrew Beckerman: daphnia have locally adapted developmental plasticity. Sweet combo of common garden & genomics, much to chew on. #ASN2014


Jenna Morgan Lang (@jennomics)
.@boechera just gave THE most awesomest talk. I'm totally jonesing to read soil microbe/plant phenology paper now! #ASN2014

Holly Kindsvater (@HollyKindsvater)
Spent a satisfying afternoon working on #science with @dr_k_lo #ASN2014

Andrew Beckerman (@beckerhopper)
So long #ASN2014 and thanks for all the sunny science. Well done team @DanielBolnick

Holly Kindsvater (@HollyKindsvater)
Sorry to be leaving Asilomar today. Thanks for a great meeting @ASNAmNat ! #asn2014



Daniel Bolnick (@DanielBolnick)
If you didn't get a chance to buy one of Alex Wild's photo in the raffle / silent auction, visit alexanderwild.com & mention #ASN2014



January 31, 2014

The Debate at Asilomar--21st Century Naturalist Meeting, January 2014


One of the goals the ASN 21st Century Naturalists Meeting



was to experiment with different formats, one of which was the evening debate organized by ASN President Trevor Price, who told the debaters--NO COMPROMISE:


The proposition was “This house believes that species richness on continents is dominated by ecological limits.”

Proponent: Dan Rabosky. Seconded by: Allen Hurlbert.
Opponent: Luke Harmon. Seconded by: Susan Harrison.
Organized by Trevor Price, ASN President 2014.
As it was described in the program:
To what extent does regional and local diversity depend mostly on time and diversification rate (Wallace's old hypothesis for the latitudinal gradient), or is instead close to an ecological carrying capacity? These issues have recently become much more focused given our improved understanding of biological diversity through time and earth's history, notably paleoclimate. Nevertheless we are far from resolution, and researchers still have strong views. 
In this debate, Dan Rabosky and Allen Hurlbert will present the case for ecological regulation, while Luke Harmon and Susan Harrison will argue the non-equilibrium case. The format will roughly follow that of the famous Oxford University debates. Dan will present a prepared 20-minute summary, followed similarly by Luke. Then, there will be opportunity for alternating rebuttals from either side. While flexible, we expect the first rebuttal to last up to 20 minutes from each side, with a second response of up to 10 minutes again from each side. Following this, questions will be thrown open to the audience; each question can be addressed to one or other side, or both, but both sides will be given the opportunity to respond. This is an innovation for the ASN, and if successful we hope to refine the format in future meetings.
Dan Rabosky is Assistant Professor at the University of Michigan. He has worked on Australian lizards and comparative methods, and is well known for his investigations of diversity-dependence in the pattern of lineage splitting in phylogenies. Allen Hurlbert is Assistant Professor at the University of North Carolina, whose research explores broad-scale patterns of diversity and community structure, with an emphasis on North American birds. Luke Harmon is Associate Professor at the University of Idaho. He has worked on Anolis lizards and comparative methods, including evaluation of the correspondence between morphological diversification and lineage diversification, and causes of disparities in clade richness across vertebrates. Susan Harrison is Professor at the University of California, Davis. She works on regional, historical and local drivers of plant richness, focusing especially on he flora of California.

It was indeed lively and there were zombies...




Thanks to Luke Harmon there is a video of the debate posted on You Tube:


The participants have also agreed to turn their arguments into papers so that the debate will appear in the American Naturalist with an introduction by Trevor Price later this year. As Allen describes it in the interview (link below),
In terms of thoroughness, it seems we will be putting out two companion papers based on the debate that all share the same subheadings, thus providing a detailed point-counterpoint. In that way, it’s less important that the actual debate cover every topic or resolve any one issue, and the point of the debate can be focused on conveying to the audience the general areas of disagreement.
Thanks to Jeremy Fox, there's a great interview with the participants at Dynamic Ecology, "Resolved: debates at scientific meetings are a good thing"


As they say, it was a trial run so aspects were a bit rocky, but everyone that I spoke with agreed that the debate format was a great idea.