Call for comments: modifications in the rankings of institutions

October 19, 2008

One feature of RePEc is its ability to rank researchers and the institutions they are affiliated with. Researchers create a list of affiliations when they register in the RePEc Author Service. However, this system was devised before rankings started to be computed, and some unforeseen consequences have emerged for authors with multiple affiliations. As there is no way to determine which affiliation is the main one, or what percentage economists would allocate to each, we are forced to treat each affiliation equally for ranking purposes. This leads in several cases institutional rankings to be “hijacked” by organizations that offer secondary affiliations. See, for example, the overall ranking of institutions. Another consequence can be found in the regional ranking, where individuals with a main affiliation from outside may take the place from legitimate insiders. Prime examples are Massachusetts, the United Kingdom and Germany.

What are the solutions? The obvious one is to modify the RePEc Author Service scripts to allow the declaration of a main affiliation or of affiliation shares. We have pondered that for some time now but find it very difficult to implement, especially as the main resource person for this project is not with us anymore. Thus we need to find some way to proxy the affiliations shares. I want to propose here one way to do this, open it for discussion, with the goal of having a formula in place for the January 2009 rankings.

The logic of the proposed formula is that there are many people affiliated with a particular institution, then it must be that most of them have courtesy or secondary affiliations. If person A is affiliated with institutions 1 and 2, institution 1 has many people registered and institution 2 few, then the ranking scores of person A should count more toward institution 2 than 1. Of course, such a distribution scheme pertains only to authors with multiple affiliations.

To be precise, let I be set set of affiliations of an author. For each i in I, let Si be the number of authors affiliated with institution i. Compute S as the sum of all Si. The weight of each affiliation is Ti=S/Si. These weights are then normalized to sum to one.

Take the following example. Economist A is affiliated with the Harvard Economics Department (46 registrants), the NBER (324 registrants) and the CEPR (262 registrants). The respective Ti would be 632/46=13.74, 632/324=1.95, and 632/262=2.41, given that 46+324+262=632. After normalizing the T‘s to one, Economist A’s ranking scores would count to 13.74/18.10=75.9% for the Harvard Economics Department, 1.95/18.10=10.8% for the NBER and 2.41/18.10=13.3% for the CEPR. For regional rankings, 86.7% (75.9% + 10.8%) of his scores would count in Massachusetts and 13.3% in the United Kingdom. Under current rules, scores are distributed fully to affiliated institutions and count fully in each region.

This is much simpler than I can manage to explain here… But a few additional details are in order: some variations in definitions can be discussed: Si can represent the number of registrants, the number of authors (registrants with works) or the numbers of works of authors. The latter would be to avoid institutions to discourage (erroneously) young faculty with few works to sign up. I favor the number of authors. Also, we need to deal with affiliations that are not listed in the database (EDIRC) and thus do not have a defined number of registrants. One solution is to just ignore such affiliations. The drawback is that the relevant authors may not get ranked in some regions where they are genuinely affiliated. Thus I propose to apply for those institutions the average Si of the other affiliations. If no affiliation is in the database, all get the same weight.

I now welcome comments on how to proceed and hope to implement the new scheme for the January 2009 rankings, which are released in the first days of February 2009.

January 18, 2009 Update: The new ranking method for institutions has now been programmed and is ready for the early February release. The formula discussed above has been adopted with two amendments. The first was discussed in the comments: 50% of the weight is allocated to the institution with the same domain name as the author’s email address. The remaining 50% is allocated over all affiliated institutions by the formula given above. The second amendment pertains to the weights of institutions that are not listed in EDIRC. As there is no author count for them, I put the default at the average number of authors per listed institution, currently 4.55.

February 3, 2009 Update: I am receiving many questions about the sudden changes in the rankings within countries. As authors with multiple affiliations do not count fully in each location any more, their ranking has worsened. Similarly, institutions that have many members with multiple affiliations now look worse. Note also that a few small errors have crept in, and they will be corrected for the February ranking.


October 14, 2008, Open Access Day

October 14, 2008

October 14, 2008, has been declared Open Access Day to increase the awareness of Open Access. RePEc, and its predecessors, have been promoting open access for 15 years now, by enhancing the dissemination of preprints, which in Economics are usually called working papers or discussion papers. A quarter million of them are now listed, with many of them being close versions of published articles that are hidden behind a publisher’s paywall. Whenever possible, we link the two versions. The conditions are that the titles be very similar and the author be registered in the RePEc Author Service, having claimed all version in the research profile. RePEc also indexes numerous open access journals, with their article labeled to recognize free downloads.

In this respect, it is important to note that the vast majority of publishers allow authors to publish working papers, in many cases even as post-prints (after publication of the journal article). Through the linking between versions we do in RePEc, this essentially comes to make pay-journals open access. For a list of publishers and there policies, see SHERPA/RoMEO.

Are Open Access works popular? We have not systematically studied this so far, but consider the following. A working paper available online has been downloaded on average 1.77 times in September 2008 (after numerous corrections to eliminate robots and multiple downloads), while the figure stands at 0.97 for journal articles (including those that are open access). Also many working paper series have impact factors superior to many journals, highlighting that researchers in Economics do not hesitate to cite pre-prints.


RePEc in September 2008

October 3, 2008

The big news for this monthly feature is that we have now topped a quarter million working papers listed in RePEc. In terms of traffic, we have recorded 675,205 file downloads and 2,704,001 abstract views on the reporting RePEc services. Note that these numbers are, as always, the results of heavy adjustements in order to count legitimate human readers.

New contributors to RePEc for the month are: Queen’s University (II), Universität Giessen, World Bank (II), Bangladesh Development Research Center, Nottingham Trent University, World Institute for Development Economic Research (UNU-WIDER), Emerald Insight, Universidad de Antioquia, Watson Wyatt Worldwide.

For the various thresholds achieved during this month, we have:
350,000 items listed in profiles of registered authors
320,000 abstracts listed
250,000 working papers listed
160,000 items with analyzed references
120,000 working papers with analyzed references
17,500 registered authors


Volunteer recognition: Christopher Baum

September 24, 2008

Christopher (Kit) Baum is Associate Professor of Economics at Boston College and one of the early volunteers in RePEc, gradually taking important responsibilities. He opened at Boston College one of the first RePEc archives, first with the department’s working papers, soon complemented by a large collection of Stata routines. Once commercial publishers started to get interested in having their publications indexed on RePEc, he took the initiative to negotiate with several of them data transfer protocols, many of which he still maintains and hosts. He is also the person answering to the central RePEc email, which in particular corresponds with maintainers of new RePEc archives. He is also the administrator of the RePEc home page and of this blog.

Quite obviously, Kit has become an essential, if not overburdened part of the RePEc team. Without him, RePEc would not have grown, both in the number of archives and also in terms of the coverage of the large commercial archives. He is also very active and influential in the internal discussions among RePEc volunteers, where policy questions are argued and emergencies are resolved.

Kit is an atypical volunteer in RePEc in that he does so much. There are many other opportunities for volunteers to get involved in RePEc, large and small. Just ask or propose.


RePEc as a bibliographic tool

September 14, 2008

RePEc is a scheme to collect bibliographic information about publication and pre-publications in Economics. Publishers provide all the relevant information, which is then displayed in various ways by RePEc services. This allows users to have access to this data. While it is useful to find items of research while browsing or searching through these services, it is even better when one can upload the relevant bibliographic data directly into one’s bibliographic tool.

Every abstract page on IDEAS has links that allow to download such bibliographic information in various formats: as a HTML citation, a plain text citation, the BibTeX entry familiar to LaTeX users, the RIS format used in various software like EndNote, and the ReDIF format used by RePEc. For registered authors, it is also possible to obtain these records for all their publications in one download. If other formats are used in the research community, they can be provided as well. Just ask.


RePEc in August 2008

September 4, 2008

What is new at RePEc? We have introduced RSS feeds for our NEP mailing lists, now giving an alternative to the emails for the dissemination of new working papers. Econlit has opened a RePEc archive collecting bibliographic information about some of the top US working papers series that were not yet listed on RePEc. While traffic was low, as usual for Summer, with 575,686 file downloads and 2,316,727 abstract views, we got a steady stream of authors registering, about 10 a day.

The new archives who joined RePEc were: Institute of Agricultural Economics (Romania), University of Waterloo, Pontificia Universidad Católica de Chile, Society for Judgment and Decision Making, Economic Statistics e-Center, Universidade Federal do Paraná, Economic and Social Review, University College Dublin, Suffolk University, Centro de Estudios de las Finanzas Públicas, Econlit.

Finally, RePEc passed the following thresholds:
500,000 cumulative book abstract views
400,000 cumulative abstract views at Socionet
190,000 online workig papers
90,000 cited working papers
40,000 articles with references
1,500 books listed


Keeping contact with authors

August 26, 2008

One crucial aspect of RePEc are the regular mailing that participants get. Indeed, it is easy to forget that one participates in some initiative if one is not reminded about it from time to time. Thus, authors registered in the RePEc Author Service get an email every month with statistics, a list of new citations that were discovered, and some news about RePEc. Since we have started these emails, we noticed that authors have become much more diligent in making sure their profiles were up-to-date and that they have responded to suggestions made in the emails.

One issue that arises over time is that emails become obsolete, as authors move or institutions normalize email addresses. When their email address changes, authors are asked to log in under their old address (which is the username), then change their contact details to the new address (which becomes the new username). Creating a new profile with the new address leads to duplicates.

But some authors forget, as they do not check their old mailboxes. Eventually, the monthly emails bounce, and these authors are placed on the list of lost authors, along with a mention in their online profiles on EconPapers and IDEAS. This list has been very useful in keeping the number of bad emails down, currently 243 out of 17,300. as they and others come forward with updates. Sometimes, we also learn that some authors have unfortunately passed away. Their profile is kept online, hopefully in perpetuity.

Apart from the 17,300 authors with works listed in RePEc, the RePEc Author Service has also about 5500 registrants with no listed works. Either they chose not to claim the listed works, oversaw this feature while registering, or simply registered erroneously. These people get a reminder every six months. Emails that bounce are promptly removed from the service, as these profiles serve no purpose. Indeed, the RePEc Author Service is about assembling publication, affiliation and contact information of authors.


Promoting academic blogs in Economics

August 16, 2008

Previous discussion on this blog (1, 2) covered the issue of peer review of new research, in particular now that the Internet has become such a predominant medium of communication. One way in which peer-reviewing could work is through blogs, wherein in a blogger discusses a paper and lets others comments on the paper (and the discussion).

Currently little of this is happening in Economics, as blog are much more focussed on discussing current events and each other. To promote the more academic blogs, and to encourage others to open such academic blogs, I just created the Econ Academics Blog, which aggregates those blogs that fit somewhat these criteria at this point. The Econ Academics Blog does not have blog discussions about research, but rather links to them. I hope people will find it useful as a portal and use it on a regular basis to find interesting discussions of academic nature, and that more appropriate blogs will be created.


NEP alerts now available through RSS

August 13, 2008

NEP (New Economics Papers) is an email service that alerts subscribers to new online working papers in their area of interest. About 80 fields are currently available, and the roughly weekly emails are sent free of charge. While the RePEc team thought email dissemination was sufficient, there also appears to be demand for RSS feeds as for this and other blogs. This is now available, and the RSS feeds can be subscribed to by clicking on the relevant field report on the NEP home page.

This new feature was added in typical RePEc fashion: David Hugh-Jones inquired with Marco Novarese why there was no RSS feed, Thomas Krichel encouraged David to set it up, and two days later, it was up.

If you think new features should be added to RePEc, we always welcome suggestions, especially if you are willing to do it yourself… much like many of the available NEP editors have been volunteers who just wanted a particular field to be covered.


RePEc in July 2008

August 3, 2008

While the Summer is usually calm, RePEc has just passed some historic benchmarks in terms of content and traffic: 750 journals listed, 2000 working paper series, half a million works available online, 20 million working paper downloads recorded, 200,000 works that are cited within RePEc. This all reflects the tremendous growth that RePEc continues to enjoy.

New contributors to RePEc during the month on July were: University of Petrosani, University of Sussex, University of Trento, Hanseatic University, Institute of National Economy (Romania), University of Wisconsin Press, TOBB University of Economics, Asociación Española de Contabilidad y Administración de Empresas, European Commission (DG Taxation), and University of Chile.

This month’s traffic numbers were 595,349 file downloads and 2,407,502 abstract views for those RePEc services that provide statistics: EconPapers, IDEAS, NEP and Socionet. These numbers are filtered for multiple downloads, robots and other “unusual” behavior.

Finally, let’s see all the thresholds that have been beaten, an impressive list for a single month:
20,000,000 cumulative working paper downloads
2,000,000 cumulative software component abstract views
500,000 works available online
500,000 cumulative chapter abstract views
333,333 works listed in registered author profiles
200,000 works with citations
125,000 article abstracts
125,000 JEL coded items
17,000 registered authors
2,000 working paper series
1,250 books listed
1,000 books available online
750 journals