RePEc rankings: adjustments and novelties

March 26, 2010

Given the increased interest in the various RePEc rankings, as well as thanks to comments made by users, I have made various adjustments over the last months. Some are barely perceptible, while some give some new features. Here is a short list.

Aggregate ranking for series and journals
Thanks to popular demand, there is now a ranking that aggregates the four impact factors, the h-index, download counts and abstract views. This ranking is available for journals, working paper series and all series.
Linked series
By linked series, I mean the situation where a series ceased to exist and continues under a new name. This happens in particular for journals that switch publishers, or journals that merge. Various statistics are now merged for ranking purposes. Thus, there is now a unique impact factor for the various series or journals. This was more complex to pursue for the h-index, and obviously all affected series gained from that. However, this increased the scope of self-citations that could be removed for the impact factor calculations.
Note to RePEc archive maintainers: you can link your series or journals to others by adding a new field in your series templates, Followup: or Successor:, followed by the handle of the other series.
Expanded listings
Following a poll a few months ago, the portion of the rankings that is public has been expanded. This will be visible with the ranking released early next month. The ranking of authors and institutions within countries or US states passes from the top 20% to the top 25%, within fields from the top 5% to 10%. The big ones, the top 5% authors and institutions still includes the top 5% in much detail, but now also the next 5% in five 1% “bins.”
US Economics departments
While there is already a ranking of Economics departments, one of the most frequent requests is to have one specifically for the United States. There will be one starting with next month’s update. Link.
Lost authors
We sometimes lose track of some authors when their monthly messages bounce back. Typically this is because they have moved or died. In both cases, they should not be counting towards the ranking of the institutions they are affiliated with. For a few months now, they have not. By the way, you can help rectify their status or their address by alerting us. The list of lost authors is here. See also the known deceased authors.
Peer authors
Registered authors receive every month an email with an analysis of their rankings. This now also includes a list of about 20 peers that are similarly ranked.
There is no reason an erratum or a correction should count as an additional publication for an author. We now try to drop them from ranking considerations and also to link them to the original article.

Poll on internal disclosure of author statistics

January 14, 2010

The statistics compiled by RePEc are used for various rankings, for example for authors. While we still consider these statistics to be experimental, in particular those pertaining to citations, these numbers are increasingly used for evaluation purposes. We value the privacy of authors and only disclose the statistics for the top ranked ones. Authors get each month a link to their statistics, a link with a code valid only for a month. This avoids a link that may have been disclosed once to be visible forever.

We get more a more requests from department heads to obtain the data for members of their department. Our typical response is to have them ask directly the members of their department to forward them their monthly RePEc emails. But of course, we could also provide directly all the relevant information. The purpose of this poll is to see whether participants in RePEc would favor such a disclosure. The conditions would be:

  1. The request must come directly from the head of the relevant department, by email to Christian Zimmermann
  2. The request contains a link allowing to verify that this person is indeed head.
  3. Statistics would be disclosed only for the members of the department who are currently affiliated with the department, as indicated in their RePEc profiles.
  4. Those with invalid email addresses would be excluded from the analysis, under the presumption that their affiliation may not be current.
  5. The department head would be provided with a link containing the analysis, a link that expires with the next monthly update of the rankings.
  6. The department head needs to be a registered author.
  7. A department would receive at most one disclosure a year.
  8. We reserve the right to refuse disclosing statistics.
  9. By department, we mean any unit with a separate entry in EDIRC.

If you have an opinion about thus please vote below and/or offer a comment. Given the nature of the question, we would require significant more than a majority (two thirds) to offer this service. The poll closes on February 21, 2010.

Update (22/2/2010): This poll is now closed. There was little interest, with only 89 votes, which fell 52-37 in favor of the disclosure. With an approval rate of 58%, it fell short of what I consider a substantial majority for an implementation. Thus, there will be no disclosure to department heads.

Polls on ranking disclosures

October 15, 2009

Rankings have become an important part of RePEc and we regularly get request about non-published rankings. Indeed, depending on the ranking in question, only the top 5%, 10% or 20% among authors or institutions are displayed, depending on the geographic or field aggregation. Given the insistence of some requests, I am now considering whether RePEc rankings should be disclosed in a more extensive fashion. Before making any changes, I am seeking the opinion of users.

But first, let me expose the reasons of the limited disclosure so far. Our interest is to have as many institutions and people participate in RePEc, and keep their data there current. Rankings provide the right incentives for this. Thus RePEc participation is our focus, and rankings are an accessory (and we still consider them to be experimental, as the data is still far from complete). We know, however, that at least some people do not like their poor rankings exposed and would thus remove their registration in RePEc if this were exposed. Thus, too extensive ranking disclosure would defeat their purpose. But I have no idea how widespread this would be. The second reason for limited disclosure is that rankings become less reliable as one goes further down the list. Consider, for example, that 28% of all authors have no recorded citation. Third, full disclosure will create a lot of large files and tables. We have about 22000 authors and 4500 institutions to rank…

The following polls are not binding. There results will help to define what users want. Feel free to discuss aspects that go beyond the options of the polls in the comment section (of this post, not of the individual polls). I will then decide what to do. For both author and institution rankings, the options are: 1) keep things as is, 2) disclose all the way to the top half, 3) keep things as is, but provide rankings for the following one in clusters. For example, rank the top 5% as now, then have a list of the top 6-10%, another for the top 10-15%. 4) Provide full rankings. Polls will be open until November 21, 2009.

Update: Polls are now closed. A post soon will discuss results as well as various adjustments to rankings.

About RePEc impact factors

July 27, 2009

Impact factors have always been a popular way to measure the influence of academic journals. They have been popularized by ISI, now part of Thomson. RePEc also provides impact factors, and this post is about explaining the differences between the two.

ISI takes a sample of journals and analyzes the citations across those journals. To be eligible, a citations has to appear within two years of the publication of the cited article, the cited article must be printed (not forthcoming, a working paper or a manuscript), and the cited article must be among the analyzed journals (286 in Economics). ISI is currently experimenting with a five year window, in addition to the existing two-year window.

RePEc considers all publications listed in its bibliographic database. Thus, it also considers other publication forms than journal articles: close to 1000 journals and 2600 working paper series. It imposes no time window, citations of any age qualify. In most cases, a citation of a working paper will count towards its published form once the article is included in RePEc, possibly after the original citation (condition: at least one author has both versions in his/her RePEc profile). This implies that working paper series and book series can also have impact factors. RePEc is thus more comprehensive.

However, the pool of citations RePEc is drawing from is different. It relies very much on working papers (who can later be published), as they are typically openly accessible. Some publishers also provide references in the bibliographic metadata, but not all. One implication of this is that RePEc is more current as it includes citations to and from research that is not yet published. As research gets published, this data gets updated. But as references from many journals are missing, RePEc citation data must still be treated as experimental. Whether these omissions matter remain to be seen. After all, impact factors always have to be considered in relative terms, not in absolute terms, and if omissions were not biased, they would not matter.

Another major difference is that RePEc excludes self-citations. This is an important issue as some journals, explicitly or implicitly, encourage authors to cite other articles published within the two year window in the same journal. Thus, just as self-citations are excluded for authors, they are excluded for journals. And this can matter a lot.

Finally, the impact factor is determined by divided the eligible citations by the number of eligible articles. ISI determines itself what articles are eligible for the denominator, and this can even be negotiated with the publisher. In RePEc’s case, if an article (or a working paper) is listed, it counts without adjustment.

RePEc also publishes variations on the “simple” impact factor: recursive impact factors, where every citation counts with the impact factor of the citing publication, this favors impact over numbers; discounted impact factors, where the impact of a citation decays with time (regardless of the age of the cited item; and a combination of the two, discounted recursive impact factors. Finally, there is now also the h-index. All variations have a different story to tell about the publication, and RePEc offers the reader the choice.

The best top level institutions in Economics

June 21, 2009

RePEc rankings are surprisingly popular, despite their experimental status, in fact this is the most read topic on this blog. So to cater to the interest of our users, let us add another ranking… RePEc has been ranking institutions for quite a while now, using the institutions listed in EDIRC. This ranks, say, at the department level, not at the university level. This is detrimental to institutions where economists are scattered in various departments, in particular in departments that are not listed in EDIRC, for example law, political sciences and statistics. A new ranking is now computed that assembles all authors within the top level institution for their affiliation(s), say a university, a government, etc. Current results are here.

The methodology is the following. For affiliations listed in EDIRC, the top level is used. That would typically be a university. For affiliations not listed in EDIRC, the homepage domain of the institution submitted by the author is matched with any institutions listed in EDIRC. If no match is found, it is taken as is. Finally, as usual with multiple affiliations, a weighing scheme is used to distribute the author’s score across all affiliations.

Note a few particularities. All components of the University of London (LSE, Imperial College, etc.) are all merged into one. All subdivisions of a national government are also merged. US Federal Reserve Banks, however, are not merged, as they are top level in their respective states.

Tips for authors to improve their RePEc ranking

April 16, 2009

By far the most popular topic on this blog is material about rankings. People love to know who the best are and how they fare. This post is about optimizing one’s ranking within RePEc, and doing so in a way that does not trigger our safeguards against cheating. It turns out all the following points are points we actually want to encourage anyway so as to improve the quality of the data collected in RePEc.

As an author, here is what you can do once you logged into the RePEc Author Service:

  1. Make sure all your works listed in RePEc are actually in your profile. Thus, do not remove from your profile working papers that have been published. Some working paper series have higher impact factors than many journals, and working papers are much more downloaded than articles. In addition, if all versions are in your profile, we can link between them. (If you previously refused items that were yours, you can recuperate them by clicking on the “refused” tab in your research page, unrefuse the relevant items, and then redo the search)
  2. Make sure the name variations listed in your profile really encompass all possible ways a publisher may have listed your name. The automatic search is only going to find works with such names.
  3. There may be additional citations waiting for your approval. These are those for which we have less confidence that they pertain to the right work. Click on the “citation” tab in your author account.
  4. Link to your profile on EconPapers or IDEAS from your homepage or email signature.
  5. When refering to your works on a web page, put the link to EconPapers or IDEAS. We cannot count downloads that do not transit through RePEc services.
  6. Make sure all your works are listed on RePEc. For the missing ones, encourage the publisher to list them, or get your department to open a working paper series, or upload your works on the Munich Personal RePEc Archive.

As an institution, you can optimize your ranking by making sure your registered authors follow the advice from above and:

  1. Make sure everyone is registered and maintains his/her profile.
  2. Make sure everyone gives the proper affiliation. You can check who is listed with you by finding your institution on EDIRC.
  3. Have your working paper series listed on RePEc. Instructions are here.

If everyone optimizes like this, RePEc data will be more complete, current and useful. Help us make it better!

The best young economists?

March 25, 2009

Who are the best young economists? RePEc publishes all sorts of rankings based on its data, but has so far been missing one that highlights the best young economists. Indeed, they are typically invisible from the general rankings as it takes many years to build up the required body of work and citations to be featured among the top economists.

Unfortunately, authors registering with RePEc do not supply their year of birth or the year they obtained their last graduate degree. However, RePEc has information about the date of most publications, and it is then possible to determine (roughly) when a career started. Here, we do not make the type of publication (article vs. working paper, for example), as the goal is to try to approximate when the economist started being active in research.

Based on this criterion, two groups of economists are selected: those with their first publication, whatever the medium, less than five years ago, and those less than ten years ago. Quite obviously, there is considerably more measurement error compared to that already present in the general ranking, first because of the imperfect measure of the start of the career, second because the body of work is typically much smaller. But we hope people will still find these rankings useful.


Get every new post delivered to your Inbox.

Join 250 other followers