November 29, 2017

Transparency and Open Science


This fall I had an extended conversation with Tim Parker from Whitman College, who is Co-organizer of the “Tools for Transparency in Ecology and Evolution” group. Here’s a brief summary of the TTEE goals:

“In November 2015, representatives (mostly editors-in-chief) from more than 20 journals in ecology and evolution joined researchers and funding agency panelists to identify ways to improve transparency in these disciplines. This workshop (funded by the US National Science Foundation and by the Laura and John Arnold Foundation and hosted by the Center for Open Science) identified general principles and specific tools that journals can adopt to encourage greater transparency of the science they publish. Most of the ideas that emerged from the workshop rest within the recently developed Transparency and Openness Promotion (TOP) framework (https://cos.io/top/). At this time, TOP contains eight separate editorial guidelines for journals, each designed to be adoptable by any empirical discipline”

Tim and I discussed the eight TOP guidelines, whose details can be found here . Here’s a summary of where we stand on this and where I would like to go in the coming year. I would welcome thoughts on this process from you.

1.     Citation Standards: AmNat is already at the highest “Level 3”, which stipulates that “ Article is not published until providing appropriate citation for data and materials” (e.g., a doi address for the data on Dryad or elsewhere), with rare exceptions granted for cases where authors get waivers regarding posting data (e.g., for ongoing analyses of long-term datasets, or for human data where privacy requirements prohibit data sharing).

2.     Data Transparency: AmNat is at the basic “Level 1” (“Article states whether data are available and, if so, where to access them.”). The higher Level 2 requires that all data are available without exception, which I do not think is feasible (e.g., human genetic studies often prohibit data sharing for privacy concerns as part of human subjects regulations). We are 95% of the way there, but that last 5% I don’t think is likely to happen.

3.     Code Transparency: We will implement the basic “Level 1” (the article states whether code used in analysis or simulation is publicly available, and if so where). If the community supports the move, I would be okay with shifting toward Level 2 in the near future (“Code must be posted to a trusted repository. Exceptions must be identified at article submission.”). I find it odd, by the way, that Code transparency allows exceptions while data transparency does not. My own unfortunate experience (which I wrote about here as a means of catharsis) is that code can contain mistakes. Transitioning to code repository will help encourage authors to proofread their code as carefully as they proofread their prose, thereby (hopefully) catching any minor code mistakes that can have major effects on conclusions.

4.     Materials Transparency: the reagents and materials that we use are so heterogeneous as a set of disciplines, that I did not see particular merit to requiring that the “Article states whether materials are available and, if so, where to access them.” So we will not presently aim for this standard.

5.     Design and Analysis Transparency: We will soon institute Level 1 (“Journal articulates design transparency standards.”). This will entail a checklist (e.g., confirming that the paper clearly reports sample sizes, effect sizes, which statistical tests, etc) that authors fill out upon resubmission of a revised manuscript. Higher levels require these checklists before we even review a paper the first time, which I think is a disincentive for submissions. Having this happen upon resubmission is a good way to help guide authors through careful revision. The checklist is currently a work in progress, building on a TTEE template.

6.     Study Plan Pre-registration: Pre-registration is the practice of publicly reporting one’s experimental design in advance of executing the project and publishing. This allows readers to be confident that the published work was indeed designed for the stated goal, rather than some post-hoc reinterpretation. I see the value in this goa, but I also strongly believe that biology often surprises us with un-looked for insights. Consequently, I have no interest in requiring pre-registration. Neither do I see a down-side to allowing authors to state whether and where they pre-registered. TOP’s “Level 1” requires that papers merely state whether or not the study design was pre-registered. If so, they state where the pre-registered plan can be accessed. There would be no penalty for studies that were not pre-registered.

7.     Analysis Plan Pre-registration: As with plan pre-registration, authors have the opportunity to record their statistical plans before obtaining data. This reduces the temptation to ‘P-hack’, doing analyses until a significant result is found. Again, I see value in allowing our data to surprise us and lead us to new questions, so I do not intend to require pre-registered analysis. But, I am fine with papers reporting whether or not they were pre-registered. So, I am planning on adopting TOP’s Level 1 standard: papers state whether or not the analyses were pre-registered, and if so, where the pre-registration can be found. Post-hoc analyses are still allowed, and manuscripts without pre-registration would still have equal footing with those that did pre-register.

8.     Replication: TOP’s replication guideline states that the basic level is that the “Journal encourages submission of replication studies.” This would be a big break from The American Naturalist’s tradition, as our brand is based on conceptual novelty and so generally we do not accept studies that duplicate a prior result in a new species, let alone in the same species as a prior study. Consequently, at present I do not intend to adopt this guideline.

I am entirely open to feedback on whether and how to institute these TOP guidelines. In particular, if anyone wants to help me craft some of the details (e.g., pre-registration statement formats, checklists), just drop me a note.

An additional topic often comes up when we discuss open science, which is the topic of author page charges, publication fees, versus open access. The American Naturalist is a non-profit journal published by the University of Chicago Press, unlike many of the larger journals published by for-profit institutions. We have converged on a hybrid model of publication. Authors who have sufficient resources to pay for Open Access fees (that help defray the costs of production, see this blog post for an explanation), are welcome to do so. But many of our papers are written by graduate students or junior researchers who may not have substantial funds for publication charges. We seek to accommodate these authors as well, by allowing people to publish non-open access and pay very low page charges. People who truly don't have the financial means to pay even these small charges can get waivers. By taking this model, we remove barriers-to-entry for aspiring authors who maybe can't afford several-thousands-of-dollars publishing costs seen at some other journals. We also make it possible for authors with means to go Open Access to reach a broader audience.

- Dan Bolnick
Incoming Editor-In-Chief





An open letter from the incoming AmNat Editor, Dan Bolnick

I wrote following message to send Associate Editors of The American Naturalist, to introduce myself as the incoming Editor-In-Chief and to discuss some matters of journal policy. In the interests of transparency, I am making a lightly edited version of the message public to everyone. I hope this helps clarify aspects of journal policy, and as a result proves useful to aspiring authors and our readers.  -Dan
_________________________________________________________________________________


Dear Associate Editors,

On January 1, Judie Bronstein will formally step down as Editor-In-Chief of The American Naturalist, and I will do my best to fill her role. I’d like to take this opportunity to thank her for her many years of service to our journal. Judie began as an Associate Editor in 2004, became an Editor in 2010, and Editor-In-Chief in 2013.  She’s set a high bar for thoughtful and constructive decisions. She oversaw several big initiatives, including the transition to double-blind review, and a series of 150th anniversary articles highlighting the impact of past papers on our field today. Judie, thanks for your leadership, and enjoy a well-deserved break (once the papers you’ve been handling work their way through the system). I also want to extend a heartfelt ‘thanks’ to the staff at the University of Chicago Press who produce a wonderfully professional final product. You can read about who they are and what they do in a recent blog post (http://ecoevoevoeco.blogspot.com/2017/10/the-secret-lives-of-manuscripts.html ). And of course, thanks to all of you, the AEs who contribute valuable time and thought to identifying promising manuscripts, and working with the authors and reviewers to hone them into well-crafted papers.

            While I have met many of you in person over the years, or via email, I want to take a minute to introduce myself. I’ve been an AE of AmNat since 2008, and briefly also served as AE for Evolution. I like to think of myself as an “integrative biologist”, which is conveniently ambiguous. I tend to publish more in Evolution than Ecology, and go to the Evolution meetings more than ESA, though I occasionally go to the latter, so I suppose I’m an evolutionary ecologist with an emphasis on the former. In practice, my research at various times has touched on foraging behavior, mate choice, predator-prey dynamics, ecological causes of selection, speciation genetics, parallel evolution, biomechanics, genomics, parasitology, and immunology. I like to use a mix of research approaches, combining field observational natural history with field and lab experiments, genetics, meta-analysis, and theory. I suppose that’s why I’ve always gravitated towards The American Naturalist, for its interdisciplinary focus and its aspiration to fuse theory and data. On a more personal note, I’m a parent of two scientifically-minded young girls: the older is obsessed with the Big Bang and spectroscopy, the younger with lichens. My wife, Deborah Bolnick, is a human geneticist specializing in ancient DNA studies of North America. In the coming year we’ll be leaving Texas (where I’ve been at UT Austin for 13 years), for the cooler climate of Connecticut (University of…), to be closer to family and snow and mountains. I'm excited to join the strong EEB program at UConn, which also has a strong group of faculty working on microbiomes and on immunology. 

            While I have everyone’s attention, I also want to go over some important information that concerns all Associate Editors.  This is a bit long because there’s a lot I want to convey, so please bear with me:

Soliciting submissions: The number of journals serving ecology evolution and behavior has increased substantially in recent years. This trend has resulted in declining submission rates for many older society journals, including The American Naturalist. To combat this trend we need to be proactive about soliciting good submissions. I would like to ask each of you to proactively encourage colleagues to submit their work to The American Naturalist.  If you see a particularly good seminar, conference talk, or graduate student thesis defense, please consider suggesting our journal. I especially want to see us expand into fast-growing subjects where we have been a relatively minor player (e.g., genomics, among others).
            We don’t publish a large number of papers per issue (about 12 on average), so any given topic may be rarely covered. This gives our audience the false impression that “Am Nat doesn’t publish X anymore” (replace X with the researchers’ pet topic). To address this, Judie began a “Recent Articles in The American Naturalist” list, which is posted in the monthly ASN news letter and which gets a lot of hits online. I want to continue to fight these misconceptions by emphasizing that we remain conceptually broad, and welcome any high-quality paper in organismal biology broadly defined. You are a key part of spreading the word on this, by encouraging submissions in a wide range of subjects.  If you encounter a colleague who thinks their field is under-represented at AmNat, please point them to the Recent Articles lists (which have links to the cited articles). They’ll be surprised by the range of things we do publish. If they really don’t see their pet topic, then encourage them to try us anyway.
We especially encourage people to consider us for Syntheses and Perspectives; we get relatively few good proposals for these. The key is that an S&P needs to be more than a review, it needs to provide some novel insight(s) that emerge from a consideration of the existing literature.
            Many authors under-value their own work. In the past year I have colleagues say their  very-interesting results weren’t “AmNat-ty enough”. I fear we often miss outstanding papers because authors put our journal on a pedestal and don’t think their paper measures up. This self-deprecating bias may be especially severe for students. Please encourage authors who might otherwise self-select out of publishing with us.
           
Value added editorial decisions. A lot of journals send authors decision letters that basically rubber-stamp the reviews. I want to remind you that The American Naturalist aims to meet a higher bar. We want our AE and Editorial decision letters to add their own additional insights that add value to the paper. Whether it is published with us or not, we want to make the authors feel like we helped them out and made their eventual paper better. This helps us build a community of appreciative authors who are more likely to submit again, and more likely to read and cite our articles. This is more work, which is why we tend to send each AE fewer manuscripts than you might get as an AE at some other journals, but it is a key part of our brand.

Speed. I’ve seen multiple twitter threads suggesting that papers should take no more than 2 weeks from submission to decision.  I disagree. I think that papers should be returned as quickly as we can, but the priority is to render detailed and constructive commentary that improves the paper whatever the eventual decision. That takes time. I want to make it a hallmark of our journal that we prioritize careful constructive review, rather than sacrificing this on the altar of speed for its own sake. That said, the journal gets a very bad reputation (and loses submissions) when we take too long.  So, it is crucial that, when you are first assigned a paper, you look it over quickly (preferably within 1-2 days) and send it to review or write an editorial decline decision letter within one week (preferably less). Likewise, when reviews come back in, please write your decision letter within a week (preferably less). As AE, I typically sent a paper out to review (or not) within 48 hours, and tried to write a decision letter within 3 days of receiving reviews back.

 Editorial declines. If you judge that a paper just isn’t sufficiently engaging for our journal, please feel free to write an Editorial decline. But, there’s nothing more off-putting to an author than a half-baked editorial decline. If you opt for an Editorial decline, please write a serious review that gives the authors a road map for how they might improve the paper enough to be suitable for another journal (you may suggest other journals, also). We should aspire to add value to any manuscript that crosses our desk, and to be kind and thoughtful in our decisions. Our AEs are quite good at this, so we often get thank-you letters from authors after editorial declines, because our AEs and Editors put in a great deal of effort to be constructive even when declining, and because we render timely decisions this way. Editorial declines should take no more than a week. We have had some cases of editorial declines taking 3-4 weeks, which is unacceptably long and drives away authors. I know that detailed decline letters take time, and we are all busy.

Decline Without Prejudice. There is often some confusion about the boundaries between Decline, Decline without prejudice (DWOP) and Revise. Here’s a brief summary of my view on this.
We should decline when the paper has zero chance of being published with us. It may just not be novel or interesting enough, or may have fundamental flaws that we are confident cannot be fixed (e.g., an experimental design mistake that would require a complete re-do).
We should ask for revisions when the authors have failed to make a compelling case, but we suspect they can fix the paper through substantial rewriting, re-analysis, or by adding available data. When writing a ‘revise’ decision we want to make it very clear that we are not promising the paper will eventually be accepted. Rather, we should convey that if the authors can fix X, do Y, or add Z, to our satisfaction, then we will reconsider the paper.
I think most papers should really fall in either the ‘revise’ or ‘decline’ category. DWOP inhabits the grey area between these. I think DWOP should be used when we are very skeptical that the authors could successfully fix X, do Y, or add Z. Or, DWOP may be appropriate when we are demanding a fundamentally new element (new data, restructured model) to the paper whose outcome is very uncertain but crucial for acceptance. Conversely, we shouldn’t use DWOP as a substitute for “lots of revisions needed”. That can be made clear in how we write a ‘revision’ decision letter.

When to re-review? Journals are facing a lot of pressure to accelerate the time from submission to publication. One way to do this is to bypass a second round of reviews. When we receive a revision, it is the AE’s job to carefully examine the authors’ response to reviews, and read through the resubmitted manuscript. You have the right to decline the paper outright if they did not respond effectively to reviews, or to accept the paper if they did respond effectively. In fact, the default option should be for you to evaluate the resubmission. Sending a paper out for more review is needed only if  (1)  the authors have added new data or analyses that you are not qualified to evaluate, or (2) the authors have substantially rewritten a very large proportion of the paper (or fundamentally changed multiple figures), so that the text requires more than one pair of eyes.

Special Features. Until now, The American Naturalist has only published Special Issues arising from the American Society for Naturalists’ Vice Presidential Symposium that takes place at the Evolution meeting each year. We will be expanding to publish more Special Features (keep your eye out for one in the March 2018 issue, for instance). These will not be separately bound issues, but rather sections within a regular monthly issue. At first these will mostly arise from conference symposia sponsored by the American Society of Naturalists. In addition to the VP Symposium, ASN runs one extra symposium at the Evolution meeting, and at the Ecological Society of America meeting. We may also consider unsolicited proposals for special features. If you know of a potential special feature, talk to me about it to learn how we handle these. I want to emphasize a key point, however. In my experience reading Special Features in other journals, there are often papers that might not otherwise be good enough to see the light of day. I do not intend for us to compromise our standards for novelty or quality. Special Feature submissions may be rejected just like any submitted manuscript. If too few special feature articles pass through our usual filters at an acceptable pace, then we will publish the few that do as regular articles, reserving special features for batches of at least four papers on a coordinated topic. Unlike some journals, our special features will not be edited by the special feature organizers (who have a vested interest in the outcome).

Diversifying the AE pool. At present, 14 of our 64 AEs are female, and a large majority reside within the United States. This is, nevertheless, a big improvement over the distant past, because Judie and other recent co-Editors (Yannis Michilakis, Alice Winn, Troy Day, among others) have made a real effort to diversify our AE pool. One of my goals as Editor is to continue to improve the diversity of AEs. The good news is that women constitute 10 of the 19 new AEs who have started in 2017 or will start in 2018 (many appointed by Judie, before I came on the scene), and 8 of the 19 are non-US residents. I especially welcome suggestions for adding AEs from under-represented geographic regions (Latin America, Africa, Asia).
            Our AEs are typically drawn from people who have a proven track record of publishing and (more importantly) reviewing for us. We look for reviewers who give perceptive, in-depth, and timely comments. This means that the diversity of our AEs is partly constrained by the diversity of our reviewers (and authors). So, I want to encourage you to make an effort to recruit diverse reviewers (and solicit paper submissions from them). To help you with this, visit https://diversifyeeb.wordpress.com , and I especially encourage you to see the Blog post on this topic by your fellow AE Meghan Duffy: https://dynamicecology.wordpress.com/2017/09/06/choosing-reviewers-recognition-not-recall-and-why-lists-like-diversifyeeb-are-useful/

Easy Submission. We want to make it as simple as possible for authors to submit papers for review, which means stream-lining some of our requirements for review. The one thing I most want to bring to your attention is that we do not require authors format their manuscript to look like an AmNat paper (citation format, headings style, etc), at the initial submission. If the paper is worthwhile and its conceptual content satisfies us and the reviewers, there is time to bring it into nit-picky format later. So on initial review don’t worry about (for example), citations not being in exact AmNat style (though we request some form of (Author, year) format in the text.

Thanks for your attention, and I look forward to working with you all. I want you to know that I am very grateful for your help, and I am always open to questions and suggestions regarding the journal’s policies and directions.

Best wishes,


Dan Bolnick (incoming Editor-In-Chief)

October 5, 2017

Thought-Provoking Am Nat at 150 (part 2)

On January 6, 2017, future Editor-in-Chief @DanielBolnick was inspired by a tweet by Erik Svensson to ask, Dear evolution-ecology twitterverse: what is the most influential American Naturalist paper, for your own career?

There were a lot of responses!
Part 1 was here:
http://comments.amnat.org/2017/03/thought-provoking-am-nat-at-150-part-1.html

I just bumped back into my file of captures, so here's part 2:

Butch Brodie‏
Queller 1992 pulled together disparate world views for me

Sebastian Schreiber‏
Two for me:
JH Gillespie. 1977. Natural selection for variances in offspring numbers: a new evolutionary principle
RD Holt & JH Lawton. 1993. Apparent competition and enemy-free space in insect host-parasitoid communities 

Flo Débarre‏ @flodebarre 
@seb_schreiber it should go in par with your 2015 @ASNAmNat paper. The Appendix makes everything so clear!

Arvid Ågren‏
another favourite: Mayr (1983) How to carry out the adaptationist program?

Russ Corbett‏
Charlesworth, Coyne and Barton. 1987. The Relative Rates of Evolution of Sex Chromosomes and Autosomes.

Stephen De Lisle‏
Williams 1966 Natural selection, the costs of reproduction, and a refinement of Lack's principle. Short but sweet

jim mallet‏
Maynard Smith, J. 1966. Sympatric speciation. American Naturalist 100:637-650.
Vol. 100 was v. influential! "Pleiotropism," dismissed by JMS, now seems important in speciation

Brian J. Enquist‏
Too many good ones. - how about Janzen 1967; Brown 1984; Pullium 1988
daniel cadena‏
Janzen 1967 - why mountain passes are higher in the tropics.


Gregor Kalinkat‏
Yodzis & Innes 1992; Real 1977; Oaten & Murdoch 1975 - #predator #prey #functionalresponse & #foodwebs

Timothée Poisot‏
oh yeah Real 1977. Big times.
Polis & Strong 1996, Bolnick et al 2003, Futuyma 1976, Fry 1996. I'm a biotic interactions person.

Daniel Matute‏
Noor 1997. How Often Does Sympatry Affect Sexual Isolation in Drosophila?


Matthew Hahn‏
Lynch and Force.

Martha Muñoz‏
Huey et al. 2003. Behavioral drive and behavioral inertia. Brilliant paper!!


More to come!

September 21, 2017

Impact Factors and Original Research

Short Version:

Review articles increase Journal Impact Factor because authors cite them instead of the original research that underlies them. Too much original research used, for example, in meta-analysis, gets listed in online-only appendixes or author-supplied PDFs. Web of Science can’t find either. Google Scholar can’t find author-supplied-PDFs. In a quest to increase credit for the authors of original research, Am Nat is printing all the references in appendixes in the main Literature Cited list and will be encouraging authors to consider citing the original research.

Long Version: 

As anyone who has followed our Journal Impact Factor (JIF) knows, it’s been on a bit of a roller coaster. Since 2003, I’ve been trying to puzzle out, first, how the impact factor works and, second, why ours bounces around. All along, our journal’s Editors have insisted that we won’t change anything editorially with the JIF in mind. It was obvious early on that review articles and methods articles boost a journal’s JIF. Indeed, in a list of the 100 most cited articles, methods papers dominated (http://www.nature.com/news/the-top-100-papers-1.16224). So the Editors agreed that we would be consistent and publish the same conceptually driven, original research, that has always defined The American Naturalist. 

We do publish some papers with methods in them, of course (Felsenstein 1985 being our most-cited paper, for example), as well as papers with a review component in them—in particular, synthesis papers—but they have to be conceptual and provide new insights that move research forward. So, mysteriously (to me), over the years our editorial course has been steady while our impact factor has not. As a result, periodically, I’ve poked around in the JIF mystery.

I would look at Web of Science, and I would do what they say they do. So, for the number that came out in 2017, the JIF supposedly took the citations made in 2016 of papers published in 2015 and 2014. I can capture the trend but nowhere near the same values. Then I bumped into this blog entry that explained that the JIF is not based on Web of Science:
“The counting method used in the [Journal Citation Reports] is much less strenuous than the Web of Science, and relies just on the name of the journal (and its variants) and the year of citation. The JCR doesn’t attempt to match a specific source document with a specific target document, like in the Web of Science. It just adds up all of the times a journal receives citations in a given year.” https://scholarlykitchen.sspnet.org/2016/04/12/on-moose-and-medians-or-why-we-are-stuck-with-the-impact-factor/
I still don’t know why the denominators never make any sense, but it helps explain why scrolling through the Web of Science never adds up.

When looking at individual articles, it’s clear that most articles get a few citations in the short window that the JIF looks at, with a few high flyers that lift the mean. So just a few papers added into the mix and the JIF increases a lot. As these papers appear in (or fall out of) the 2-year window of time that defines JIF, the impact factor may jump up or drop down. Here’s a blog post about how this looked for some journals in Chemistry. https://stuartcantrill.com/2015/12/10/chemistry-journal-citation-distributions/

Another blog that looked at the problems with JIF calculations is Brian McGill’s post here: https://dynamicecology.wordpress.com/2016/06/21/impact-factors-are-means-and-therefore-very-noisy/

As a result, I started looking at our two highest years, and sure enough, there was a very highly cited paper that was lifting our average up—it was an introduction to an ASN Vice Presidential symposium. So, though the Editors don’t publish review papers, the invited ASN papers can act that way. As it turned out, in the two years we sharply dropped, we were missing a VP symposium introduction and there happened to be no ASN addresses. Since then, the VP introductions and the ASN addresses have returned on schedule and our impact factor has gone back up.

Though the high-flying introduction made a big difference, all the VP introductions get a healthy array of cites. This phenomenon is described in this post along with an explanation: http://ecoevoevoeco.blogspot.ca/2017/03/over-citation-my-papers-that-should.html where Andrew Hendry analyzes why some of his papers seem over-cited—and finds they are introductory papers:
Another choice for an over-cited paper might be the introduction we wrote to a Philosophical Transactions of the Royal Society special issue on Eco-Evolutionary Dynamics. The introduction simply pointed out that evolution could be rapid and that evolution could influence ecological process, before it then summarized the papers in the special issue. Again, nothing wrong with the paper, but a summary of papers in a special issue is hardly cause for (soon) 300+ citations, nor is that typical of such a summary. …. This is fine, but excellent papers that treat eco-evolutionary dynamics as a formal research subject, rather than a talking point, are out there and should be cited more. Indeed, several papers in that special issue are precisely on that point, and yet our introduction is cited more. Similar to this example of over-citation, I could also nominate the introduction to another special issue (in Functional Ecology) – which is my fourth most cited paper (437 citations). 
Why are these “OK, but not that amazing” papers so highly cited? My guess is that two main factors come into play. The first is that these papers had very good “fill in the box” titles. For instance, our PTRSB paper is the only one in the literature with Eco-Evolutionary Dynamics being the sole words in the title. Thus, any paper writing about eco-evolutionary dynamics can use this citation to “fill in the citation box” after their first sentence on the topic. You know the one, that sentence where you first write “Eco-evolutionary dynamics is a (hot or important or exciting or developing) research topic (REF HERE)” The Functional Ecology introduction has much the same pithy “fill in the box” title (Evolution on Ecological Time Scales) and, now that I look again, so too does the Conservation Biology paper (Evolutionary Response to Climate Change.) The second inflation factor is likely that citations beget citations. When “filling in the box”, authors tend to cite papers that other authors used to fill in the same box – perhaps partly because they feel safe in doing so, even if they haven’t read the paper. (In fact, I will bet that few people who cite the above papers have actually read them.) One might say these are “lazy citations” – where you don’t have to read anything but can still show you know the field by citing the common-cited papers.
He summarizes a point I was coming to realize--that it’s not just that review papers lift impact factors. Review papers take cites away from the original research. I had that demonstrated in one of my forays into Web of Science. I was following a paper that happened to be published early in the year. It was getting quite a few cites in that same year, the year that didn’t count toward the JIF. I checked it the next two years, but in the years that did count toward the JIF, the article got zero cites. It had been co-opted by a review paper that cited it. Another demonstration came to me when a reviewer wrote in asking how soon the paper he’d just reviewed would be published. He was writing a review paper and wanted to cite it. So a paper’s ability to be counted was getting co-opted before it was even in Production.

It became increasingly clear that the drive to lift JIFs with review papers has meant that the authors of original research are not getting their due. Therefore, we are philosophically committed to encouraging authors to cite original sources for results and ideas and will do what we can to get cites counted by Web of Science and Google Scholar.

--Trish (Managing Editor)

Updated to add another view of how the Impact Factor is damaging scholarship:
http://blogs.lse.ac.uk/impactofsocialsciences/2017/09/19/clickbait-and-impact-how-academia-has-been-hacked/

September 5, 2017

The Debate at Asilomar--21st Century Naturalist Meeting, January 2014


One of the goals the ASN 21st Century Naturalists Meeting



was to experiment with different formats, one of which was the evening debate organized by ASN President Trevor Price, who told the debaters--NO COMPROMISE:


The proposition was “This house believes that species richness on continents is dominated by ecological limits.”

Proponent: Dan Rabosky. Seconded by: Allen Hurlbert.
Opponent: Luke Harmon. Seconded by: Susan Harrison.
Organized by Trevor Price, ASN President 2014.
As it was described in the program:
To what extent does regional and local diversity depend mostly on time and diversification rate (Wallace's old hypothesis for the latitudinal gradient), or is instead close to an ecological carrying capacity? These issues have recently become much more focused given our improved understanding of biological diversity through time and earth's history, notably paleoclimate. Nevertheless we are far from resolution, and researchers still have strong views. 
In this debate, Dan Rabosky and Allen Hurlbert will present the case for ecological regulation, while Luke Harmon and Susan Harrison will argue the non-equilibrium case. The format will roughly follow that of the famous Oxford University debates. Dan will present a prepared 20-minute summary, followed similarly by Luke. Then, there will be opportunity for alternating rebuttals from either side. While flexible, we expect the first rebuttal to last up to 20 minutes from each side, with a second response of up to 10 minutes again from each side. Following this, questions will be thrown open to the audience; each question can be addressed to one or other side, or both, but both sides will be given the opportunity to respond. This is an innovation for the ASN, and if successful we hope to refine the format in future meetings.
Dan Rabosky is Assistant Professor at the University of Michigan. He has worked on Australian lizards and comparative methods, and is well known for his investigations of diversity-dependence in the pattern of lineage splitting in phylogenies. Allen Hurlbert is Assistant Professor at the University of North Carolina, whose research explores broad-scale patterns of diversity and community structure, with an emphasis on North American birds. Luke Harmon is Associate Professor at the University of Idaho. He has worked on Anolis lizards and comparative methods, including evaluation of the correspondence between morphological diversification and lineage diversification, and causes of disparities in clade richness across vertebrates. Susan Harrison is Professor at the University of California, Davis. She works on regional, historical and local drivers of plant richness, focusing especially on he flora of California.

It was indeed lively and there were zombies...




Thanks to Luke Harmon there is a video of the debate posted on You Tube:


The participants have also agreed to turn their arguments into papers so that the debate will appear in the American Naturalist with an introduction by Trevor Price later this year. As Allen describes it in the interview (link below),
In terms of thoroughness, it seems we will be putting out two companion papers based on the debate that all share the same subheadings, thus providing a detailed point-counterpoint. In that way, it’s less important that the actual debate cover every topic or resolve any one issue, and the point of the debate can be focused on conveying to the audience the general areas of disagreement.
Thanks to Jeremy Fox, there's a great interview with the participants at Dynamic Ecology, "Resolved: debates at scientific meetings are a good thing"


As they say, it was a trial run so aspects were a bit rocky, but everyone that I spoke with agreed that the debate format was a great idea.

Updated: The debate is in the journal at

Introduction
http://www.journals.uchicago.edu/doi/full/10.1086/680858 

Rabosky and Hurlbert
http://www.journals.uchicago.edu/doi/full/10.1086/680850

Harmon and Harrison
http://www.journals.uchicago.edu/doi/full/10.1086/680859

June 30, 2017

ASN Presidential Addresses

In looking up some things for #AmNat150 ( see http://amnat150.org/) I started assembling a list of presidential addresses in the journal.Not everyone gets around to writing up their addresses! When I realized that Hutchinson's Santa Rosalia was a presidential address, I thought it might be interesting to share all the ones I've found so far in the journal (some of them are a bit disguised).


June 15, 2017

Online-Only Material in Am Nat

If you are looking at online-only material and wondering how The American Naturalist might handle it, here is a quick guide:
  1. Is it essential to telling the article’s story clearly? Then it should be part of the article (or possibly be a separate print appendix). Papers can be as long as they need to be—but not any longer.
  2. Is it essential for replicating the paper? Is it something a reader will want to see 30 years from now? Then it should be in the appendix—in the source file, edited, and typeset.
  3. Is it nice but not necessary? Is it a format that can’t be printed on a piece of standard paper? Is it so extensive that it’s prohibitively expensive? Is it laid out in such a way that redoing it makes little sense? Then it should be in author-supplied file formats.
  4. Is it code? Then it’s in author-supplied files. Is it data? Then it’s in Dryad.

Here's why we do what we do...

We handle online-only material differently from many other journals, so it seemed like a good idea to explain why we do what we do. First though, it’s useful to look at the 20 years I’ve been managing editor and see the problems I've already witnessed. We had supplementary files in Excel back in 1998. Microsoft no longer supports Excel 1993 so proprietary software is a liability. We have moved every file (thousands of them) to new platforms three times. Files that aren’t integrated can be misplaced. We have systems to track them down and repost them, but over time, inevitably, this will be harder and harder. Data archiving policies have been developed, and alternative workflows for scientists, particularly with commercial sites, have developed, so the landscape is more complicated. Commercial sites (e.g., GitHub or FigShare), even if they aren’t particularly oriented toward profit, are financed by venture capital and are at the mercy of market forces and may not have the same kind of institutional commitment to long-term access and preservation, so they aren't a clear alternative.

So here is a breakdown of how we handle online-only material.

Appendixes

Back in 2003, we had a terrible backlog of papers. To try to squeeze in more papers into our print page budget, we decided to move the appendixes for papers from print, where they had always been, to be online-only appendixes. However, often these appendixes are essential for replication or for understanding the breakdown of the math. The University of Chicago Press was one of the first presses to have an electronic edition, and it had worked up its definition of the best practices for preservation. The Press determined that online-only appendixes had to be in the same source file as the main text, they should be edited and typeset for consistency with the article and for clarity, and they would be preserved for as long as the article itself would be preserved. The journal hopes to still be publishing for the next 150 years. The questions we ask about supplementary material, therefore, are, Does this material need to be preserved? What is the consequence if this material became unusable or lost 30 years from now? Will editing and typesetting this material make it less clear? Because figures are processed for the best quality possible, the tables are edited, and the math typeset, appendixes use up expensive and limited resources (people) and so there are page charges for appendixes (which can be waived with the other page charges). An appendix, if it is short or very useful, may still be printed in the journal. We may also print a short essential appendix (e.g., a table defining the variables) and put other appendix material online only.

Other Supplementary Materials

Supplementary materials generally are nice but not necessary. Separate files supplied by the authors, particularly separate files in proprietary formats, are detachable and “losable,” may not be usable as the owner of the proprietary software upgrades, and are not always well edited and well presented. However, in the time since 2003, authors have gotten used to the idea of putting all kinds of materials related to a paper online at no cost, so it’s sometimes not clear what needs very long-term preservation (i.e., appendix style) or what is informative but not as necessary. In addition, there are items such as massive excel tables, code that is better in its original file format, or files that are just not typesettable, like video or sound files. Material may also be at other locations (e.g., GitHub or FigShare) but the best practice is to have information, especially code essential for replicating the paper, together in one place with the article. Our goal is preservation that is as long-term as possible and that is not dependent on the vagaries of the venture capital marketplace. We post any author-supplied files (posted in the original format supplied) at no charge.

Data Archiving

In 2010, journals in ecology and evolutionary biology agreed to enforce the Data Archiving policy, originally proposed by the NSF and made possible by the development of Dryad. See the announcement of the policy here: http://www.journals.uchicago.edu/doi/full/10.1086/650340. I happened to attend the initial meeting launching the project in 2005—so I remember that the idea was two-fold: providing the necessary information so that a study could be replicated and preserving data into the future. The image they kept using in 2005 was the researcher who retired, turned off her computer, and lost a life-time of irreplaceable information. As a result, the journal requires that data be archived in Dryad or a noncommercial data repository that is open (or accessible) and that is designed for long-term preservation. Dryad can also host other materials related to the article. The journal article has an active link to Dryad and Dryad has an active link to the article so the connection is clear. Data files may also be stored with the article at UCP but they must be archived at Dryad. The American Society of Naturalists covers the costs (via a change in the page charge structure).

References in Appendixes

As of 2016, we are requiring that references cited in the appendixes be printed in the main body of the article. In pursuing a question about how the field’s reliance on the impact factor was creating its own impact on research, it became very clear that original research was not getting its just due. Lists of references in author-supplied files are not counted as citations by Google Scholar or by Web of Science. Google Scholar counts cites in our form of appendix (i.e., mining the html tagging) but Web of Science counts only the Literature Cited in the main PDF. We wish to support original research so we commit pages to printing the references. Occasionally a meta-analysis paper presents a reference list that becomes prohibitively expensive to print, but we’ve hit only one of those since this policy began.

Patricia Morse
Managing Editor