A while back, I noted some disturbing facts about citation rates in the humanities. For instance, according to this study:
More than 90% of published humanities papers have received no citations two years after publication.
About 82% have never been cited five years after publication.
Citation rates in the humanities have been consistently this low across the past several decades.
Kieran Healy's recent research on citation rates in philosophy broadly indicates similar trends. As he put it, "on the average hardly anyone is getting cited, be they man or woman." These trends are in marked contrast to the sciences, where (according to the first study linked to above):
Over 50% of natural science and social science papers receive at least one citation within two years.
About 70% receive at least one citation after five years.
Citation-rates in both areas have climbed dramatically and steadily over the past several decades.
Why are citation rates so low in the humanities, and in philosophy specifically? And is this a problem? If so, what should be done about it? Obviously, these are very big questions, and we cannot hope adequately address them here. Instead what I would like to do is focus on a particular issue I have some concerns about: restrictions on "replies" to articles in other journals.
I have sometimes heard people float the hypothesis that citation and discussion rates are low in philosophy because comparatively few papers actually warrant discussion. This is something I have heard surprisingly often (i.e. I have heard more than a few people say things like, "Most published papers in philosophy are bad - so it is no wonder they are not discussed"). Although I deeply disagree with this suggestion--from my vantage-point, there appear to be too many good papers that are underdiscussed, not too many bad papers that are discussed--I am not sure we can evaluate the hypothesis without considering possible confounds: that is, other possible explanations of why so few papers are discussed. I want to suggest that the common editorial practice that many journals have of not accepting "replies" to articles in other journals may be just such a confound.
For those who may not know what a "reply" piece is (sometimes they are also called "discussion notes"), a reply is a publication that primarily, or exclusively, addresses another paper, aiming (for instance) to refute or otherwise undermine its argument. This is in contrast to what are sometimes called "free-standing" or "stand-alone" papers--papers that focus primarily on developing an original positive argument of their own.
My first two publications were replies, and occurred in one of the few philosophy journals out there (JESP) that actually accepts replies to papers in other journals. However, most journals do not accept replies to papers in other journals--which basically means that if you write a reply, you have approximately one venue, and one venue only, that you can submit it to for publication. Allow me to explain why I think this plausibly erects several related barriers to philosophical discussion.
1. The False-Negative Problem
The first concern I have about the practice of journals not considering replies to articles in other journals concerns the prospect of "false negatives"--that is, good/important replies never being published. Here is how I think this problem arises. If one can submit a reply to only one journal, whether the reply is ever published all comes down to the judgments of one set of editors and/or reviewers. Yet, as we all know, the peer-review process can be "noisy." What one set of reviewers or editors at one journal judge worthy of rejection, reviewers or editors at another journal may judge to be publishable and important.
Consider for instance some remarks by Jason Stanley reported a while back at NewAPPS:
I'm reviewing Kieran Healy's citation data, and it reminds me again how weird journal acceptance is. My book *Knowledge and Practical Interests* is the fifth most cited work of philosophy since 2000 in Phil Review, Mind, Nous, and the Journal of Philosophy (book or article). Yet the book itself is the result of three revise and resubmits, and finally a rejection from Phil Review. One of those drafts was also rejected from Mind, and also from Nous. All of those journals have accepted papers discussing, in many cases very centrally, a work those very journals have deemed unpublishable.
I have heard many stories like this, of influential papers were evidently rejected (and in some cases, desk-rejected) at multiple journals before being published (I seem to recall hearing somewhere this was true of David Chalmers' and Andy Clark's famous article in Analysis on the extended mind).
Because the peer-review process is so noisy/inconsistent, I think it is plausibly critical to the integrity of the process that authors have multiple chances to submit their work at different venues. If one can only submit a given piece of work to only one venue (as in the case of replies), then it makes the fate of the piece (i.e. whether it is published) come down to the (potentially mistaken, biased, etc.) judgments of one set of individuals, who may well reject a perfectly good/important paper. It is only by allowing authors to submit pieces to multiple journals sequentially that "false negatives" (good papers never being published) can be reliably avoided.
And indeed, it is generally recognized in many different areas of human life--in politics, corporate book-keeping, etc.--that transparency is important. In order to ensure that a system is working as it should, there must be some transparent way to check how the system is working. When it comes to the normal process of submitting articles for review, the process is relatively transparent in this regard. Although sometimes our papers are desk-rejected (a non-transparent decisionmaking process), over the course of multiple submissions to multiple journals the process becomes increasingly transparent. For example, if your manuscript is desk-rejected at many journals, that is some relatively transparent reason to think the manuscript is unpublishable in its present form--as it shows that multiple editors at different journals all came to the same decision. Conversely, if your manuscript is not desk-rejected at some journals--and particularly, if one receives reviewer comments--then, at least over time, one typically gains some insight as to the quality of one's paper: one may get negative reviewer reports explaining why one's paper isn't publishable, or perhaps some positive referee reports suggesting that it is publishable (or at least may be with some revision).
In contrast, the "no replies to articles in other journals" rule makes the publishing process for replies decidedly non-transparent--as something like the proverbial "black box" that one cannot see into. After all, if your reply is rejected without explanation, and you can only send it to one journal, you basically have to give up on it, and may be left with no idea why the paper was rejected. This is for the simple reason that, in contrast with normal articles--which again, one can send many places--with any given reply one can only get one data-point: the decision of the single journal to which one sends it. But one data-point gives very little information, especially if (as is often the case with replies) one's paper is rejected without comment.
2. The Disincentivization Problem
In addition to the false-positive problem, the rule against publishing replies to articles in other journals plausibly raises a second barrier to philosophical discussion: it disincentivizes people from writing replies to being with.
As I mentioned above, my first two publications were replies. Not only that: I very much enjoy writing replies. Like many philosophers, I "enjoy me a good refutation"! And indeed, critically evaluating arguments is one of the most central and important skills we learn as philosophers. Philosophical discussion only benefits from critical engagement with people's arguments. It provides a clearer picture of which arguments and theories are good, which are bad, which can be improved, etc. Yet, although I very much enjoy writing replies, I basically stopped writing them.
Why? Answer: given the rule that one can only submit them to basically one journal (the journal the original article appeared in), I found that writing replies mostly isn't worth my time. For a while, I wrote a few replies that I thought were strong. But, when they were desk-rejected without explanation, I couldn't help but think to myself, "Why both doing this? Why bother writing refutations that I have only one shot of publishing, and which may be rejected without any explanation?" Writing replies takes time. Given that stand alone articles can be submitted to many journals, once I started publishing standalone articles, I simply figured my time would be better spent on them, not replies. Hence, I don't write replies. I have been disincentivized from writing them.
I suspect I am probably not alone here. What incentive is there for anyone to write replies when the costs of writing them are significant, but one only has one shot to publish them? This problem is only compounded, I think, by the problem of non-transparency mentioned above. Given that replies are sometimes (often?) desk-rejected without comment, getting a reply rejected can be a very frustrating process--one that leaves an author with no real idea why their paper wasn't published, and without any real recourse (such as submitting the paper elsewhere) for determining whether the decision arrived at was correct (i.e. whether their paper really isn't publishable). If anything is a dispiriting disincentive, it seems to me, that is. Why write and submit a reply when (A) chances are, it will be rejected (since most papers are rejected!), (B) one can only submit it one place, and (C) one may get no transparent explanation for whatever decision is reached at the one place you can actually send it?
3. The Misapplication Problem
I also think there may also be some reasons to worry whether the editorial rule against publishing replies to articles in other journals may be misapplied, in a manner that plausibly erects a third barrier to philosophical engagement.
Generally speaking, "replies" are very short (2-3K words) and purely negative in nature, critiquing a particular paper. However, I have some experience with papers that I do not think are properly considered replies being desk-rejected. Here is such a case: one writes a very long and involved paper that (A) begins with a refutation of an article in a rival journal, and then (B) uses the refutation as a springboard for a novel positive argument. Suppose, furthermore, that step-(A) is critical to the new paper--that is, that due to its dialectical structure (i.e. it aims to use problems with the earlier argument to motivate a novel positive argument), the paper basically has to begin with a refutation of an article in a rival journal. I have some experience with papers like these being desk-rejected "because we do not consider replies to articles in other journals." Further, although in cases like this one may attempt to recast the paper in a different way so that it "appears less like a reply", in my experience another problem can arise here: recasting the paper in this way can actually undermine the paper's dialectical effectiveness (since the primary motivation for the paper's novel, positive argument may be to correct what one takes to be a very big mistake in another person's argument/theoretical framework, i.e. the argument being "replied" to). And indeed, my experience is that the disincentive problem arises here yet again. For although I myself have written a few papers like this over the years (i.e. full-length papers critiquing someone else's framework, and then attempting to provide a better framework of my own), I have mostly abandoned them, preferring once again to write and publish "stand-alone" papers. But this is just to say, once again, that I have been disincentivized from doing something that the citation statistics outlined above indicate: I have been disincentivized from actually engaging with the work of my fellow philosophers. And once again, I suspect I am not alone.
4. The Chicken-and-Egg/'Foot in the Door' Problem
Finally, to return to an issue we discussed last week, I think there are plausible reasons to worry whether the current rule against replies to articles in other journals might make it more difficult for early-career philosophers to "get their foot in the door" in terms of getting their work discussed in the literature. As this study, "Does Science Advance One Funeral at a Time?", there appears to be a good deal of preexisting "inertia" in the scientific literature. Academic literatures generally seem to revolve around the ideas and arguments of people who already "have their foot in the door." Once a person's work receives "uptake" in an academic literature, discussion of their ideas may generate some momentum, or further discussion in the literature. Conversely, as 'Perplexed Marketeer' noted here, when a person's work hasn't had "uptake" yet, one may have to convince reviewers that the work in question warrants discussion. But, of course, that may be difficult to do! All of which brings us back to the false-negative and disincentive problems. If replies are already disincentivized, and it can be difficult to get reviewers to publish discussions of articles that have no "uptake", the very fact that replies can only be submitted to one journal plausibly makes it all the more difficult for an article to get the kind of "uptake" in the first place necessary for reviewers to be amenable to papers discussing one's work.
So then, or it seems to me, there are several related reasons to worry about whether the current rule against replies to papers in other journals plausibly erects several barriers to philosophical discussion. Given the current status of philosophical discussion today--where, again, 80+% of papers are never cited, let alone discussed--I think these things should worry us. If anything, our discipline should be trying to encourage more cross-discussion, not squelch it. Why, then, do so many journals have such a rule? As far as I can tell, there are two plausible reasons for it:
Volume: considering replies to articles in other journals might overwhelm journals with submissions, adding to an already saturated peer-review system with long wait-times and not enough reviewers.
Self-interest: as 'Insightful diner' noted here, journals have self-interested reasons against promoting discussion of articles in other journals (viz. increasing another journal's citation-rate/impact factor can undermine the prestige of one's own journal!).
For reasons I explained in response to 'Insightful diner' (and which Steven French seconded), I am unpersuaded by the self-interest concern. First, opening up one's own journal to replies to articles in other journals promises to improve the content of one's own journal. It would plausibly provide readers with refutations and constructive engagement with the rest ofthe literature that (for reasons given above) are currently disincentivized. Second, critiques, refutations, and improvements of work in other journals appearing in your own journal promises to increase the prestige of one's own journal relative to others (as it would show readers that one's own journal publishes stronger work than the work in other journals being critiqued).
I am much more sympathetic with the volume concern, as indeed I think allowing replies to articles in other journals plausibly would increase journal submissions substantially. Nevertheless, I am optimistic about whether the volume concern can be surmounted. First, some journals (such as JESP) already consider replies to articles in other journals, and JESP has some of the quickest turnaround times around! Second, it seems to me that journals could handle volume problems creatively--for instance, by merely asking for a "thumbs-up/thumbs-down" verdict from reviewers (as opposed to the standard written justification for reviewer recommendations), or perhaps placing a cap on how many reply submissions they can accept for review annually. Although when it comes to standard articles I am against these approaches--as I think authors generally deserve some justification for why their articles are rejected, etc.--it seems less problematic to me in the case of replies (or, at least, a legitimate tradeoff to accept given the total costs and benefits involved of journals publishing more replies).
In any case, although I think the volume concern is a serious one, I think there are a number of reasons for our discipline to have a more open conversation about whether philosophical engagement might be improved by removing or modifying the standard editorial restrictions on replies, and if so, how journals might do so effectively and feasibly.