It seems that they might.
Matti Eklund, one of the editors of Philosophical Review, emailed me asking for some more detailed number crunching on the Journal Surveys section with respect to Comment Quality Ratings. He wanted to test the hypothesis that philosophers who have papers rejected tend to rate the quality of comments lower. So I did that. The numbers seem to confirm this.
The chart below compares quality of comment ratings for some of the mainstream general journals. The column labeled “All” is the average from all respondents. The column labeled “Initial Reject” averages just the ratings from respondents who received an initial verdict of “Reject” and the column labeled “Initial Accept” averages just the ratings from the respondents who received an initial verdict of “Accept” (Keep in mind the sample sizes for that latter column are low – these journals rarely give out an initial verdict of “Accept”).
What’s interesting is that, while it seems that rejected philosophers tend to rate quality comment lower, there is still marked variance among these journals among rejected philosophers. Also interesting is that Australian Journal of Philosophy and Philosophical Quarterly are both above 3 among rejected philosophers.
Also interesting is the major shift for Philosophical Review. They seemed to be at the bottom in terms of comment quality, but if you look at their average among rejected philosophers and you factor in PR’s very low acceptance rate…you’ll see that looking at the average among ALL respondents for PR would be misleading.