Two important changes regarding rankings:
- There are now rankings of authors and institution by field. As discussed before, the procedure is the following: identify relevant authors by the number of papers announced in NEP field reports. Past a threshold (currently 25% of all announced papers or 5 announced papers), the author is considered specialist of her field (see discussion. For institutions, each affiliated author contributes with a weight corresponding to the proportion of papers announced in the field, irrespective whether the threshold was met (see discussion).
- For rankings within regions (and within fields, described above), authors and institutions are not anymore ranked by picking them from the “grand list”, i. e., the ranking of all authors or institutions. Rather, the ranking is performed within the respective subgroup: for example, authors are ranked with the others of the same region according to all criteria, and then ranking points are aggregated. This had already been done a few months ago for women, which led complaints about this particular ranking to virtually disappear…
Soon, we will also add citation rankings for recent papers. An announcement will be made on this blog in a week or two. Note that all these rankings are experimental and subject to changes. We welcome discussions and suggestions about them in the comment section.
The RePEc Blog 
Christian,
I’m not sure that I agree to the new national rankings. In any ranking that is based on the harmonic mean of the ranks within a group, the weight of the performance indicators is based on the behaviour of the reference group. This implies that someone may be an excellent researcher in Country A, but a lousy researcher in Country B. I am rather old-fashioned, and still believe that academia is universal.
Best
Richard
I don’t agree with Richard. Academia may be universal, but standards for prominence in the profession certainly differ across countries. In the USA and UK, most evaluations of prominence come from publication in internationally-ranked journals. In some other countries, many authors may only tend to publish in journals in their native language. It is not surprising that in such an environment a ‘big fish in a smaller pond’ who occasionally appears in highly ranked international journals may be considered one of the best researchers in that country, but with modest visibility internationally.
I will point out one way in which Christian’s change looks sensible to me. I am now ranked virtually the same at my primary workplace in USA and my research affiliation in Germany. In prior rankings I was ranked much higher in the German listings, which was amusing but I did not take it seriously. I think the current scheme is delivering more sensible results.
Kit Baum
Hi Kit,
You’re 14 out of 4053 in the USA, and 13 out of 1372 in Germany. You’re much better in the USA than you are in Germany.
Alan Krueger outranks you in Germany, but you outrank Krueger in the USA.
While I understand that traditions and expectations are different between countries, I do think this is the way forward.
Richard
should have written: I do NOT think this is the way forward
My view is that researchers in small countries, which can often be inward-looking, should be encouraged to compare themselves against their peers worldwide, and that their worldwide ranking is thus what should matter, including when comparing themselves against local colleagues.
The fact that Richard is the 3rd ranked Irish author worldwide, but comes 8th in the Irish ranking, is surely very odd.
On the other important innovation, field rankings, I have two comments.
First, this is a great idea.
Second, looking at my own field, economic history, there are problems with th classification as it stands.
There are two sorts of errors that a classification scheme could make. It could include people as economic historians who aren’t, or it could exclude people who are. The first doesn’t bother me, since it is good that non-economic-historians write papers on history, and the economic history profession should be grateful for a little competition on their own turf. The second error does bother me however. The current list does include Brad DeLong, Michael Bordo and Alan M. Taylor, all prominent economic historians. But it excludes other economic historians in the top 5% of authors worldwide, including Barry Eichengreen, Douglass North , Paul David and Bob Margo. Presumably if one is in the top 5% of all authors worldwide one should also be in the top 5% of economic historians.
Let me offer a reply for the discussion about the regional (or national) rankings. Just extracting from the world ranking does not offer any additional information, except for countries that would have no representatives in the worldwide top 5%. So computing rankings from a regional pool provides added value, as long as results are somewhat different.
Now, how can we explain these “oddities” where the ranks can be different from the worldwide ranking? Say economist A is much better than compatriot B in some dimension, but slightly worse on others. In the worldwide ranking, A gets a better score and possibly rank because he managed to get many non-nationals between him and the others for his strong criteria. In the ranking from the national pool, he is just one rank ahead, and B is just ahead on the others. On aggregate, B may be ahead. Richard is like economist A in that he is very prolific and thus garners many bonus points from this in the worldwide ranking. But not in the national ranking.
Agreed, national rankings from the national pool add information — and comfort too: Klaus Zimmermann is a better German economist than I am, but I do better on the US ranking. I’d rather be American than German, and I guess Zimmermann is proud to be German.
There is a post-normal whiff here.
The national ranking is more important, because it contains my immediate peers, and because it is one that national funders and assessors will look at. People are known to select a strategy that improves their rank. (My Irish rank would improve if I get the many second rate publications of my compatriots on IDEAS/RePEc.) In a small country, the national definition of excellence may diverge from the global definition. There is already a deviation between Irish norms and international norms — because the mediocre majority in this country has very peculiar ideas on what a good economist should and should not do.
Therefore, I think that IDEAS/RePEc should display both national/national and national/global ranks, and the national/global ranking should determine the order in which people are displayed.
Richard just made some interesting points I want to address once I have received some more input (other readers, do not hesitate to write…).
Now regarding misclassification in the field rankings: Kevin asks why some prominent economic historians do not show up. They are all registered with RePEc, so this is not the problem. All but one qualify for the field. The one who does not is Douglass North, because only one of his five papers announced in NEP made it to the NEP-HIS report. Why are then the others not among the top 5%? It may have to do with the fact that the ranking is polluted with the other type of misclassification: authors that are only marginally in the field, but overall rank higher. This is a consequence of the choice I made of making an unweighted ranking of everyone who qualifies. Maybe I should weigh each by the proportion of papers announced in NEP that are in the field. Bob Margo then would certainly make it, as 14 of 19 papers were listed in NEP-HIS. Bob Lucas, currently on top, probably would rank lower, as only one of his three NEP papers qualify.
To see how this kind of misclassification may matter, see how Tom Cooley is the top agricultural economist. Currently, he is dean of the business school at NYU. Busy as he must be, he has not refreshed his profile for some time. Thus he has just three papers in NEP, one of them in agricultural economics (on social security with an agricultural sector). But he is very highly ranked overall, and this makes him the top ageconomist. Similarly, Bob Barro is tops in every field he qualifies for, by virtue of being the top ranked economist overall. I think we need to find a good way to weigh the scores by field.
For the area rankings released today January 2, 2008, I have amended the procedure in the following way: an author’s score is now multiplied by a field specific weight, this weight being the number of papers announced in the field NEP report divided by the number of all papers announced in NEP. I hope this will make the field rankings more credible.
I have a suggestion. When giving the area rankings, why not give both sets of rankings: i.e. those using ‘world relative prices’ and those using ‘domestic relative prices’?
Kevin,
I have incorporated your suggestion in the rankings released today.
Thanks Christian!
I think the “Top 20% authors in Germany” ranking is indeed amusing and cannot be taken seriously. No one in the Top 10 is a German citizen, no one works in Germany. Frankly, I would not be surprised if some of the listed authors have never been in Germany for a single day. I think it makes absolutely no sense to list economists who are “affiliated” with IZA or CESifo in the German ranking. This kind of affiliation means nothing, IZA and CESifo just give away these affiliations. In principle, they could give it to every economist in the world ranking, then the world ranking and the German ranking would be identical after all. I think that only the university at which the economist is employed should determine to which country the economist belongs.
I would not argue with Melanie’s logic, but as #12 in that ranking, I must comment. I am not a German citizen (although I am named after a great-grandfather who was) but I do have a research professor appointment with a German research institution. That affiliation is not given without serious consideration. I am co-principal investigator on a grant from a German foundation at that research institution. I spent 10 days working in Berlin earlier this spring on that project. So I would not agree that every non-German on that list lacks any meaningful connection with German scientific activity.
It can be somewhat difficult to determine ‘the university at which the economist is employed’ from the information provided when self-registering at RePEc Author Service. Can you tell me which affiliation we should consider for P.C.B. Phillips?
Melanie makes a comment that has been made to us several times already. The problem is that the current system is not set up to allow authors to identify a principal affiliation, or to rank them or to attribute different weights to each. We simply did not think about that when designing the system. Now it is quite difficult to change that, especially as our volunteer resources are rather thin. But it is certainly something I will keep in mind once with find a volunteer who would want to help out (perl, and probably javascript, competences a great plus).