Proposed changes to RePEc rankings up for vote

April 29, 2013

The RePEc rankings are a popular feature of RePEc. As we gather more information about the profession, we can also refine the criteria that are used for those rankings, as well as add more of them. With this post, we seek from our users their opinion about a few potential changes. For each proposed change a poll is attached, which we hope will help in deciding what to do.

Regarding citation counts

Citation counts are the basis for a series of criteria used in ranking authors and departments. Our citation project, CitEc, uses references from open access material, or those offered directly by publishers in their metadata, or from user contributions. But citations may also appear elsewhere and count be counted. One source is Wikipedia, which has a little less than 3000 references pointing to items in RePEc. Another is the RePEc Biblio, a curated bibliography of the most relevant research in an increasing number of topics (1100 references, but it is just starting). And a third one are blog post indexed on EconAcademics.org, the blog aggregator focusing on economic research (8000 blog mentions so far). The question here is whether references listed on Wikipedia, the RePEc Biblio or EconAcademics.org should count as citations for ranking purposes. All these citations are already listed on the relevant IDEAS pages, but they have so far not counted towards any statistic. As usual, self-citations would not count, as much as possible. For this poll, we want to distinguish whether they should count for the ranking of economists and institutions on the one hand, and journals and series impact factors on the other hand, or both.

Regarding breadth of citations

Citation clubs bias how we try to measure the impact of someone’s research. We have already some criteria that try to measure the breadth of one citations, the number of citing authors, and the same weighted by the citing author’s rank. Another way to measure breadth is to measure how widely an author has been cited across fields. For this, we can measure in how many NEP fields an author is cited. To this effect, the analysis is of course limited to working papers that have been disseminated through NEP, which has currently 92 fields. Again, self-citations are excluded, and this new criterion would only apply to author rankings.

Doctoral advisors and programs

The RePEc Genealogy is a crowd-sourced project that gathers informations about who studied under whom, where and when. From this information, one could determine who is the best dissertation advisor and which doctoral programs are the strongest. Some preliminary rankings are already available in this regard, based on the data that has been gathered so far: 1869 advisors and 499 programs at the time of this writing. It is expected that these numbers would significantly increase once a ranking would be incorporated. Instead of the h-index currently computed, it would be calculated in the same way that institutional rankings are determined: by adding up the scores of the relevant student for each criterion, and ranking within each criterion and then aggregating criterion ranks. As one can expect that only a fraction of authors and institutions can be ranked this way, all the others would be ranked right after the last author or institution with a student. It is to be expected that this ranking would matter mostly for the top ranked authors and institutions. Note that a ranking of economists by graduating cohorts is going to be first released in early May.

Exclusion of extremes

For author and institution rankings, the various criteria are aggregated after excluding the best and worst criterion. This was introduced when there were about 25 criteria. Now, there are 33 for authors and 31 for institutions. Depending on the outcomes of the votes above, there may be even more. Thus one may want to exclude even more extreme scores to avoid taking outliers into account. How many extremes should be excluded on each end? The status quo is one.

Questions or concerns?

Feel free to post a comment below!


RePEc in March 2013

April 4, 2013

What is new this month? Nothing much to report, for once, except that as MyIDEAS usage is expanding, we are now going to alert series and journal maintainers, editors and authors about how many people are following them. But as promised, we will not be revealing who it is. And, of course, we started fantasy leagues.

During the month, we counted 668,431 file downloads and 3,065,876 abstract views. Almost 500 authors registered, proving that there is still a large potential for growth in the RePEc Author Service. And we welcomed the following new archives: Institute of Accounting and Finance Kiev, Grantham Research Institute, Akroasis, Valahia University, Harvard University Press, Kobe University (II), Universidad de Guadalajara, University of Primorska, Ecological University of Bucharest, Duke University (II), Masaryk University, School of Oriental and African Studies, Gdansk University of Technology.

Finally, we reached the follow thresholds:
1000000 cumulative abstract views through Socionet
150000 cumulative downloads through Socionet
15000 listed books
6000 listed online books


Introducing the RePEc fantasy league

April 1, 2013

Fantasy (or rotisserie) leagues allow player to manage team members from a real-life competition to compose optimal teams. Fantasy leagues organize their own competitions and are popular in a variety of sports throughout the world. RePEc now has its own fantasy leagues, which allow players to manage economics departments. Specifically, there are two leagues.

Scratch league

Department owners in this league pick faculty members from the pool of authors registered in the RePEc Author Service and create departments from scratch. They need to keep a topical balance and all start with the same budget. Simulated departments are evaluated using the same criteria that underly the ranking of real life institutions.

Extant league

Here, department owners start from an existing department and then trade its faculty to improve it. Simulated departments are evaluated by comparing them to the existing ones they started from. Only economics departments are available for this league.

Markets

Both leagues work with separate markets for economists. Initial prices are based on current ranking scores of the registered economists. All currently registered economists are part of this market, and one can ask to be excluded from this market after log in.

Rules

Rules follow standard fantasy league rules. A few specificities for economists:

  • Departments in the scratch league require that economists work in a balanced set of fields. These are based on which topical NEP mailing lists disseminated their working papers.
  • One cannot short an economist.
  • Newly registered economists are added to the pool with an initial price corresponding to their ranking after the release of new monthly rankings (between the thrid and fifth day of the month.
  • Fantasy league rankings are not public.
  • A department owner cannot add him/herself to a department. If member of the original department in the extant league, he/she has to immediately trade him/herself.
  • One can play at most one department in each league.
  • Legues do not follow a seasonal schedule, they are continuously open.
  • Rules may be adjusted if the need arises.

Enjoy!


Follow

Get every new post delivered to your Inbox.

Join 250 other followers