The RePEc Blog

The RePEc rankings are a popular feature of RePEc. As we gather more information about the profession, we can also refine the criteria that are used for those rankings, as well as add more of them. With this post, we seek from our users their opinion about a few potential changes. For each proposed change a poll is attached, which we hope will help in deciding what to do.

Regarding citation counts

Citation counts are the basis for a series of criteria used in ranking authors and departments. Our citation project, CitEc, uses references from open access material, or those offered directly by publishers in their metadata, or from user contributions. But citations may also appear elsewhere and count be counted. One source is Wikipedia, which has a little less than 3000 references pointing to items in RePEc. Another is the RePEc Biblio, a curated bibliography of the most relevant research in an increasing number of topics (1100 references, but it is just starting). And a third one are blog post indexed on EconAcademics.org, the blog aggregator focusing on economic research (8000 blog mentions so far). The question here is whether references listed on Wikipedia, the RePEc Biblio or EconAcademics.org should count as citations for ranking purposes. All these citations are already listed on the relevant IDEAS pages, but they have so far not counted towards any statistic. As usual, self-citations would not count, as much as possible. For this poll, we want to distinguish whether they should count for the ranking of economists and institutions on the one hand, and journals and series impact factors on the other hand, or both.

Regarding breadth of citations

Citation clubs bias how we try to measure the impact of someone’s research. We have already some criteria that try to measure the breadth of one citations, the number of citing authors, and the same weighted by the citing author’s rank. Another way to measure breadth is to measure how widely an author has been cited across fields. For this, we can measure in how many NEP fields an author is cited. To this effect, the analysis is of course limited to working papers that have been disseminated through NEP, which has currently 92 fields. Again, self-citations are excluded, and this new criterion would only apply to author rankings.

Doctoral advisors and programs

The RePEc Genealogy is a crowd-sourced project that gathers informations about who studied under whom, where and when. From this information, one could determine who is the best dissertation advisor and which doctoral programs are the strongest. Some preliminary rankings are already available in this regard, based on the data that has been gathered so far: 1869 advisors and 499 programs at the time of this writing. It is expected that these numbers would significantly increase once a ranking would be incorporated. Instead of the h-index currently computed, it would be calculated in the same way that institutional rankings are determined: by adding up the scores of the relevant student for each criterion, and ranking within each criterion and then aggregating criterion ranks. As one can expect that only a fraction of authors and institutions can be ranked this way, all the others would be ranked right after the last author or institution with a student. It is to be expected that this ranking would matter mostly for the top ranked authors and institutions. Note that a ranking of economists by graduating cohorts is going to be first released in early May.

Exclusion of extremes

For author and institution rankings, the various criteria are aggregated after excluding the best and worst criterion. This was introduced when there were about 25 criteria. Now, there are 33 for authors and 31 for institutions. Depending on the outcomes of the votes above, there may be even more. Thus one may want to exclude even more extreme scores to avoid taking outliers into account. How many extremes should be excluded on each end? The status quo is one.

Questions or concerns?

Feel free to post a comment below!