A "Best Practices" checklist to help authors and reviewers

A number of journals have introduced required or optional checklists in recent years, aimed at either reviewers or authors. The Editorial Board of The American Naturalist is adopting an optional (but recommended) checklist as well. The intent is to remind authors of items that reviewers (and readers) expect in a paper. It may also serve to remind reviewers of things they should look for, but which often get overlooked. Because we receive both a mix of theory and data and synthesis and meta-analysis papers, there is no one-size-fits-all checklist, so the list below is broken down into parts that pertain to different kinds of papers.

The following checklist is a semi-final draft, and we welcome feedback from both authors and reviewers and Associate Editors as we finalize this document. Send comments to daniel.bolnick@uconn.edu.

____________

Rationale:

A paper by Parker et al (2018. “Empowering peer reviewers with a checklist to improve transparency”, Nature Ecology and Evolution) advocated greater use of checklists in evaluating research publications. They argue that “Good checklists do not replace complex thought; they facilitate it. … by calling attention to essential elements that are often overlooked”. Here, we provide a set of checklists tailored to the diverse kinds of papers submitted to The American Naturalist. Authors and reviewers are not required to use this checklist, but it may help each identify common weaknesses that need to be fixed. Not all items in the following checklist pertain to all studies. It is the job of the authors and reviewers to judge what elements apply to any given study. The checklist is therefore not meant to be a straight-jacket, but rather a prompt to remind us what authors should aspire to do, and what reviewers should check for. The checklists below do not cover methods, syntheses, historical perspectives, and some other articles that are also welcome at this journal.

Authors:The following checklists are designed to remind you of key features that maximize transparency of your work, and that reviewers look for in evaluating your work. We encourage you to examine relevant parts of this checklist before submissionof a new manuscript, or during revision, to ensure that you are meeting our expectations. Using this checklist may help you pre-emptively avoid common reviewer critiques. Authors should also visit the journal webpage for formatting details:   https://www.journals.uchicago.edu/journals/an/instruct#data

Reviewers: You may find the checklist to be a useful reminder of manuscript features to comment on, including somewhat mundane details that authors frequently forget to include (and reviewers frequently forget to check for). 

Contents:

1:   General considerations

2:   Articles with empirical data

3:   Articles with meta-analysis

4:   Articles with theory

5:   Submission Formatting

Aspirational features that are recommended but not yet standard practice are marked with a ✪.

GENERAL CONSIDERATIONS

	SUITABILITY FOR THE AMERICAN NATURALIST
	The American Naturalist seeks papers that cause readers to consider new ideas, merge existing ideas in new ways, rethink familiar ideas, or which provide particularly compelling evidence for established ideas. Theory and data papers are welcome in equal measure.
	Methods papers enable researchers to ask new questions, or address existing questions in new and particularly effective ways.

	LITERATURE
	The manuscript gives sufficient coverage to relevant existing ideas, and does not inflate its own novelty by omitting mention of relevant published studies.
	Authors may cite reviews and syntheses, but when possible should also cite original sources for key new insights.

	WRITING
	Length: The American Naturalist does not have a specific page length maximum. Papers should be thorough but succinct: as long as they need to be, and not one word longer. Clear enough to describe the context, methods, insights fully.
	The prose is clear.
	Papers should be free from typographical errors, and spelling and grammar mistakes. Text should be structured in well-organized sentences and paragraphs. But, reviewers should refrain from commenting on the presumed linguistic qualifications of authors.

	FIGURES AND TABLES
	Figures avoid use of red/green that may be hard for color blind readers to see.
	Figure font sizes and lines are large enough to be easily readable when reproduced on a printed page at a typical figure size.
	Figures need not show all data broken down in all possible ways: figures should convey a result as simply and directly as possible, focusing readers’ attention on key trends.
	Color should be used where appropriate to clearly distinguish lines

	SUPPLEMENTARY MATERIALS
	Supplementary Materials enrich the results but are not essential elements to understand the core message of the paper. Imagine if the Supplement were lost to history, 50 years from now, because of a server failure. Would the paper still be understandable? The answer should be yes. If not, the Supplement has elements that belong in the main text.

EMPIRICAL PAPERS

	ETHICS
	For vertebrate animal research, authors describe animal care approval.
	For human subjects,  authors describe institutional research approval (e.g., IRB).
	The research does not pose a threat to population viability.

	FIELD STUDIES
	Authors report information about collecting permits (if needed)
	Authors list specific locations (e.g., coordinates) where samples were collected or experiments performed. Exceptions may be made, for instance to protect threatened populations.
	Dates of the field work are reported.

	INTENT
	✪ It is good practice to state whether the research was initially planned to address the topic reported in the paper, or whether the paper is reporting on post-hoc analyses. We are not averse to publishing post-hoc studies, because many biological discoveries arise through surprising and un-looked for results. But, we encourage authors to be open about the relationship between their initial intent and final findings
	✪ If a study design was pre-registered, we encourage authors to state whether the study design was pre-registered. We do not require this, nor do we penalize the absence of pre-registration, or deviations from pre-registered plans.

	SAMPLING
	Authors report how study subjects were allocated to experimental treatments.
	Authors state how sample sizes were chosen (e.g., describe a power analysis, or logistical constraints).
	Where applicable, sources of genetic stocks are identified.

	DATA COLLECTION
	Authors state the magnitude of experimental treatments (e.g., food rations, pH levels), and describe how these magnitudes were chosen.
	Observers recording data were blind to the experimental treatment imposed on the research subjects.
	If voucher specimens are available, authors identify where they are deposited.
	Authors provide detailed protocols for non-standard methods as supplementary materials, to aid readers in exactly replicating these methods, including reagents, concentrations, plasticware, vendors, etc.)
	DATA ANALYSIS
	Authors report sample sizes for all data, including subsets of data (e.g., each treatment group, other subsets), and sample size used for all statistical analyses.
	Authors identify the level of replication relevant to a given analysis.
	Authors provide the precise details of data analysis (software, packages with citations to authors, and descriptions of each statistical test).
	Authors provide estimates of effect size, and uncertainty in these estimates. Tests of statistical significance that omit effect sizes are omitting the biology.
	If exploratory analyses were performed, then only a subset of models reported in the paper, all exploratory analyses should be acknowledged, and the choice of final focal models clearly explained.
	In reporting model results, estimated effects for all model variables should be reported (this may be in Supplementary materials).
	✪ Code to replicate data analysis should be made available for both review and for readers after publication.
	✪ It is good practice to pre-register your planned statistical analyses. We do not require this, nor do we penalize departures from pre-registered plans. But, if your analysis was pre-registered, we encourage authors to state where the pre-registration is recorded.
	✪ Even if you do not pre-register, it is good to acknowledge when an analysis is a post-hoc test arising from an observation of the data, versus a pre-planned analysis.

METAANALYSIS

	DATA COLLECTION
	Authors identify the scope of the meta-analysis: range of dates, geography, taxonomy.
	Authors  describe the search criteria used to obtain information, including databases and keywords.
	Keyword searches include antonyms to catch negative effects.
	Authors describe criteria for including or excluding studies.
	Authors describe how data was extracted from relevant studies. Include software citations where relevant, and state whether duplicate observers obtain replicate data extracts to evaluate repeatability.
	✪ Authors are encouraged to acknowledge whether the study design was pre-registered. There is no penalty for lack of pre-registration.

	DATA ANALYSIS
	Authors state measures of effect size.
	Authors state methods for weighting studies by confidence, or justify the lack thereof.
	Authors state how analyses accounted for pseudoreplication arising from multiple effect size estimates within one study population or species.
	Were phylogenetic corrections were applied to adjust for non-independence of study species?
	Authors clearly explain statistical model choices.
	Authors state whether any data points were excluded after data acquisition, and why.
	How was publication bias evaluated?
	Authors explain how many entries suffer from missing data, and how these missing data were handled.
	✪ Upon submission, we encourage authors to provide a copy of the statistical code used to analyze the data, along with necessary data files (see Submission Formatting)

	RESULTS
	Authors state the number of studies that passed various filters for inclusion.
	Authors provide measures of both effect size, variance in effect size, and confidence.
	Authors describe possible sources of bias in the available data.

THEORY

	BIOLOGICAL MOTIVE
	Too often, theory is developed on the back of other prior theory, without clear reference to an empirical biological motivation. All theory published in The American Naturalist should aim to explain or anticipate a biological phenomenon, and so must clearly describe the underlying biology with suitable references to empirical results.
	Authors should describe specific examples of the biological phenomenon being studied.
	Authors should relate model assumptions to biological examples, where possible.
	Authors put results into biological context, comparing model output to relevant empirical data when possible.

	MODEL CONSTRUCTION
	Authors explain rationale behind choosing analytical versus simulation methods.
	Authors explain rationale behind choosing a deterministic versus stochastic model.
	Authors justify choice of continuous versus discrete time as relevant to the underlying biology.
	Authors define all symbols where they are first used. Tables of symbols (in the main text or supplement) are also useful, but if readers must refer repeatedly to a table it interrupts their reading.
	When possible, the choice of a function for a biological relationship should be based on empirical trends.
	Assumptions and simplifications are clearly stated, and where possible are justified with reference to empirical data.
	To reach an empirical audience, the meaning of each equation must be clearly summarized in writing.

	MODEL ANALYSIS
	The range of parameters is clearly stated and, when possible, justified with reference to biological information.
	When multiple parameters are involved, parameter space is explored sufficiently widely to evaluate interactions among key parameters.
	Analytical solutions are clearly derived.
	Approximations and simplifications are clearly identified, and their consequences are discussed (e.g., when assuming very weak selection to obtain an analytical result, consider what the consequences are if selection is strong).
	Simulation code should be made available to reviewers, and subsequently shared on a public repository. Note that sharing your Github page can reveal your identity, so submitting a code file as an attachment with your manuscript is preferred

SUBMISSION FORMATTING

The American Naturalist places few constraints on initial submission formats, but be aware that reviewers are often pickier than the journal office. Reviewers’ opinions can be skewed by careless formatting that clearly deviates from journal style. See http://www.journals.uchicago.edu/journals/an/instruct#prepfor additional details.

	MANUSCRIPT FORMATTING
	REQUIRED
	Continuous line numbers or page numbers and line numbers are required for peer review.
	Manuscripts must be double-spaced (some reviewers are very opinionated about this).
	Fonts for math should be embedded in the pdf.
	RECOMMENDED
	We do not require precise American Naturalist-style citation formatting for initial submission, but we do require this for a final accepted version. However, authors should be advised that reviewers can be critical when citations clearly deviate from journal format (e.g., may criticize numbered in-text citations).
	We do not adhere rigidly to Introduction-Methods-Results-Discussion formatting. Sometimes other structures are warranted, for instance when a paper includes a model, a mix of model and data, or a series of successive experiments that build upon each other. However, methods should always be described before Results.
	We encourage authors to put figures and tables in the main text, where they are first mentioned, with captions on the same page. A majority of reviewers tend to prefer this arrangement. Preferably, figures should be on the same page as the caption that describes them.

	DATA AND CODE
	Proof-read your code as carefully as you proof-read your manuscript prose.
	Code should be carefully annotated to state dependencies, and the purpose of each step. For examples of good practices, see online summaries of Best Practices in R (https://swcarpentry.github.io/r-novice-inflammation/06-best-practices-R/)
	Code should be self-contained so a user with the relevant input data files can re-create your results, without extraneous code.
	✪ We strongly encourage authors, when feasible, to submit data tables and analytical code (e.g. an R markdown file) alongside the manuscript, so editors and reviewers can check the code with the data. These may be submitted with the paper, or provided via a link to a github repository or equivalent.  Data shared through a website that might reveal reviewer identity will not be accepted, and the paper may be returned.
	✪ We encourage authors to have their Dryad data package (data files, metadata, and ReadMe) prepared before submission, and to make these files available for review.