A not-for-profit Q/A site to enable community peer review
Motivation & Goals
I.e. problems real people have that the community review project (CR) will strive to alleviate, and the method by which CR plans to help these people. This section is meant as a general overview and includes a bit more than the minimum viable product (MVP).
- Problem: Academic and technical publications, even those that passed peer review, often contain mistakes or areas which could benefit from clarification. Currently, readers are forced to work through these issues themselves, email authors, or post questions to general purpose forums where the posts 1) may not necessarily be found by other experts, readers, or the paper’s authors and 2) may not be editable/updatable by anyone other than the OP or site admins.
Related Goal: Provide a repository for open questions, clarifications, and fixes to technical and academic publications (in particular papers that have already passed peer review). All posts/threads will be initiated by a question (which can, e.g. in the case of a found mathematical mistake, be immediately answered by the author). [Terminology Note: in this doc we’ll refer to a post/thread related to a specific problem/question as a “QA thread” or simply a “QA”.] All QA threads will be required to reference at least one paper by DOI or ArXiv #.
- make it simple for readers and authors to find all QA related to a specific paper. E.g. searching by DOI or ArXiv # brings up all Q/A from all versions of paper.
- encourage experts and authors to provide high quality answers and readers to post high quality questions. E.g. through reputation based incentives, high-rep-user editing/administration, and clear community guidelines -- similar to stackexchange)
- encourage visibility of high quality answers E.g. through upvoting and vote-based ordering -- similar to stackexchange.
- provide mechanism for removal and/or improvement of low-quality answers. E.g. give reputation incentives for editing, only allow medium- or higher-rep users to edit, and be clear about when posts should be edited in community guidelines -- similar to stackexchange.
- Problem: Reviewers and authors in traditional peer review generally share the same field/department. Many papers are however interdisciplinary and thus contain content which the reviews may not have expertise in. This allows papers with questionable content to pass peer review and in some cases even gain a high citation count. The result can be the spreading of pseudoscientific knowledge or “bad science”.
Related Goal: Provide a mechanism for multidisciplinary review (e.g. to allow a statistician to comment on a paper published in an ecology paper or vice versa).
- Area-specific reputation -- a credible statistician posting about an ecology paper should appear as just that: credible regarding stats but not necessarily regarding ecology. E.g. users have CR rep (+ can link stackexchange account to get those community reps?)
How to Contribute
Two ways to get started
- Post an issue here saying you'd like to contribute and a sentence or two about your skill set.
- join the Community-Review club on Clubhouse and come to the weekly general meeting (Sundays @ 6:30p PST) -- yes, we're holding our general meetings on CH in an effort to recruit volunteers, and yes it's working well so far =)