75% of the top 1000 economists are now registered with RePEc

January 8, 2008

The RePEc Author Service recently surpassed 15,000 registered authors, and the post relating this mentions the high coverage among top ranked economists. To document this, take one popular ranking, the one by Tom Coupé that is based on publications from 1990 to 2000. Tom Coupé has two rankings, one where publications are weighted by the impact factors of the journals, the other where citations are counted. According to the “publications” ranking, 75% of the 1000 economists are now registered with RePEc, according to the other 65%. The difference comes from the fact that the latter also includes non-economists (political scientists, statisticians, demographers, law scholars, and sociologists) that are cited in Economics journals.

One particularly interesting aspect of these rankings is how the proportions of registered authors decline with rankings:

Ranks registered,
publication ranking		 registered,
citation ranking
1-100 93 77
101-200 81 72
201-300 78 69
301-400 73 76
401-500 77 66
501-600 71 61
601-700 73 54
701-800 77 55
801-900 62 62
901-1000 65 60
Total 750 652

How can we explain this pattern? Are registered authors more likely to publish well or be cited? This may be true for more recent measures of visibility, but in 1990-2000, the RePEc Author Service was not yet functional. Are then better ranked authors more likely to care more about their visibility and thus more likely to register?

1 Comment | RePEc rankings | Tagged: | Permalink
Posted by Christian Zimmermann

RePEc in December 2007, and what we have done over Year 2007

January 2, 2008

Every month, a short summary of what happened with RePEc is sent to the RePEc-announce mailing list. I will also put that message, slightly adapted, on this blog.

The major event this month is that we passed to three important thresholds: 15,000 authors, 80% of the material now online, and 1/8 billion abstract views. For some hints at what 15,000 authors represent in the Economics profession, see elsewhere on the blog. Also, we have now released rankings for the most cited recent papers and articles.

As year 2007 is now over, we can reflect on what RePEc has achieved over that year. 158 archives were added, and the total of currently 844 archives have added 108,000 bibliographic items to RePEc, a 24% growth, with 240 new working paper series and 130 new journals. 105,000 new items are online, a 31% growth. 3,500 authors registered, almost ten a day, a 30% growth. Citation analysis coverage increased by 39%.

In 2007, we added also a few new features:

  • Compilations by institutions of all publications from affiliated and registered authors (find institutions on EDIRC)
  • Customized publication compilations: by defining a list of authors or by creating a reading list
  • Registered authors can now manage citations at the RePEc Author Service: delete erroneous ones and approve citations that were deemed dubious matches.
  • Rankings have been improved with more criteria, with rankings within fields and with citation rankings for recent items only.
  • The RePEc blog was inaugurated.

Finally, RePEc celebrated its 10th year in its current form. I think this was an impressive year, and I am looking forward to an even better year 2008!

In terms of traffic, December is expectedly calmer, but we still managed record numbers for the month: 1,822,061 abstract views and 504,315 downloads. This leads us to the thresholds we have passed this month:

125,000,000 cumulative abstract views
275,000 online articles
130,000 items with references
15,000 registered authors
1,900 working paper series
80% of all items available online

Leave a Comment » | Workings of RePEc | Permalink
Posted by Christian Zimmermann

What are the most cited recent papers in Economics?

December 22, 2007

RePEc has been publishing for several years now a list of the most cited papers and articles cataloged in its database according to three criteria, recently expanded to six. By popular demand, we now publish also a list of the most cited recent papers and articles. The selection criterion here is that the last know version has been published five or less years ago. That may sound like a long period, but considering the publication lags we suffer, I think it is reasonable. Thus, currently, articles (and papers) published in 2002 or thereafter qualify. Within a few days, those from 2002 will be dropped, so enjoy them while you can…

At the same time, the list of the most cited items has been expanded. Previously, only the top 200 were released, now we show the top 1‰. This list thus gets longer as RePEc expands and stands currently at 559. Again, the list is available according to six different criteria. So, check out whether your favorite papers are listed. And remember, all this citation data is still experimental as we try to improve on its quality, but still quite informative.

1 Comment | RePEc rankings | Permalink
Posted by Christian Zimmermann

Citation Accuracy

December 19, 2007

Open Access News pointed out a very interesting article in the Journal of Cell Biology, Show Me the Data. Written by that journal’s executive editor, the executive editor of Journal of Experimental Medicine, and the Executive Director of The Rockefeller University Press, it first reiterates many quality issues with journal impact factors that seem to be well-known among biologists, but I suspect that they are news to many economists. Many of these issues also hold for citation rankings for individuals. Beyond that, there are other issues that make citation data suspect. Fortunately, there are potential solutions to many of these problems.

First, it helps to describe impact factors as they are calculated by Thomson Scientific (previously the Institute of Scientific Information, or ISI). An impact factor in year t is the mean number of cites to all articles in that journal in years t-1 and t-2 divided by the number of number of research or review articles. Criticisms include

  • the data in the denominator and numerator are not consistent
  • Thomson is unclear on what exactly defines a research or review article
  • some journals have negotiated with Thomson on exactly what defines the article type
  • retracted papers are not excluded
  • of course, the mean is inflated by a few star papers
  • editors can game the system; apparently some do and some don’t (I’ve even seen this in the Wall Street Journal)

The authors go on to say that they contacted Thomson and received some of their data. They found numerous errors in how article were categorized. Further, “The total number of citations for each journal was substantially fewer than the number published” as reported by Thomson. When they requested further data from Thomson, the data still didn’t add up. They conclude “It became clear that Thomson Scientific could not or (for some as yet unexplained reason) would not sell us the data used to calculate their published impact factor.”

Their bottom line is even more clear: “If an author is unable to produce original data to verify a figure in one of our papers, we revoke the acceptance of the paper. We hope this account will convince some scientists and funding organizations to revoke their acceptance of impact factors as an accurate representation of the quality—or impact—of a paper published in a given journal. Just as scientists would not accept the findings in a scientific paper without seeing the primary data, so should they not rely on Thomson Scientific’s impact factor, which is based on hidden data.”

Besides the points reiterated and brought up in the Journal of Cell Biology, there are further accuracy issues with Thomson data. For example, to identify authors, they only use initials for the their first and middle name. As they pool papers from all fields, this is a more severe error than one might first guess. Thomson reports that Kit Baum (known to Thomson as CF Baum) has publications in the Fordham Law Review (on nuclear waste) and the Sociology of Education (on group leadership).

A further issue is Thomson’s coverage; EconLit lists some 1,240 journals in our field while the last time I checked Thomson covered but a fraction of these. I don’t have recent data for their coverage, but in total Thomson covers 8,700 journals encompassing all academic fields, so it seems doubtful that Thomas has substantially changed its economics coverage.

A further problem plaguing all citation analysis is simply extracting citation data with software. After all, citations are written for people, not machines. I haven’t seen data for Thomson on this (one wonders if it is public), but I do know that CitEc has faced a very real challenge here.

There would seem to be several solutions to these problems. First, all of us should treat impact factors and citation data with considerable caution. Basing journal rankings, tenure, promotion, and raises on uncritical acceptance of this data is a poor idea. In the extreme, one could imagine legal action in a tenure case.

Second, as the authors of the Journal of Cell Biology argue, this data should be public, just as research findings should be. One initiative here is a Petition for OA [open access] to bibliographic data. My understanding is that through a “RePEc service” like EconPapers or IDEAS, raw CitEc data can be accessed by the public. Further, CitEc works with RePEc Author Services to correct citations. Here’s one more reason to join those 15,000 who have registered with it!

Third, we should investigate putting unique identifiers into each reference so that software can easily read it. That is, besides listing the journal, its volume, and so on, it would also include a unique identifier to the cited paper. DOIs are one possibility, but it is prohibitively expensive to get a license to dispense DOIs. However, “RePEc handles,” which identify papers in RePEc, are permanent and also cover working papers. Thus, we might start including them in each reference. This highlights a further issue: there is little incentive for authors to add this to their citations as it aids others. Perhaps one step in this direction would be for sites like IDEAS, which provide references for papers in different formats like BibTeX or EndNote, could include the RePEc handle along with the current author, title, journal, etc.

7 Comments | Economics Profession | Permalink
Posted by Bill Goffe

15,000 authors on the RePEc Author Service

December 15, 2007

The 15,000th author registered recently on the RePEc Author Service (which also has another 5,000 registered, but without any works in their profile). See a list of all those registered at EconPapers or IDEAS. This give us the opportunity to reflect on the coverage of this service: what proportion of academic economists is covered? Let me offer a few suggestions.

Assume that the works listed in RePEc provide a representative sample of all the works written by economists. Then determine how many of these works are listed in the profile of a registered author. By that account, about 40.1% have been claimed, and thus about 40% of the profession would be registered with RePEc. This latter number is in reality higher, due to several biases: a) some authors are not alive and cannot register; b) some registered authors have the unfortunate habit to remove from their profile working papers once they are published; c) some works listed are not written by economists, and these authors are less likely to register with RePEc.

Alternatively, estimate the number of authors in the world from the membership in academic societies. I guess the three largest societies are the American Economic Association (18,000 members), the European Economic Association (2,300 members) and the Econometric Society (5,500 members). Obviously, their membership overlaps, and not every of their members is an author. But not every economist is member either. Assume that adding their membership numbers corrects for all mismeasurements, then the RePEc Author Service covers 58% of the profession.

One can also observe a specific subsample of economists, those listed among the top 1000 by Tom Coupé. There, the RePEc Author Service covers 75% of the top 1000 by publications and 65% of the top 1000 by citations (which includes quite a few non-economists). But we have good reasons to believe these proportions are higher than for the whole population. Indeed the proportion is significantly higher for the better ranked within this sample, and we can extrapolate that those outside the top 1000 are less represented in the RePEc Author Service.

In summary, the RePEc Author Service covers between 40% and 75% of the profession. Possibly less, possibly more, likely in between.

1 Comment | RePEc features | Tagged: , | Permalink
Posted by Christian Zimmermann

Blog disruption

December 14, 2007

The RePEc blog was offline for a few days due to a hardware failure, along with a few other websites at Boston College, our host. Everything seems to be running well now, but please contact us if you see any remaining issues.

Leave a Comment » | RePEc blog policy | Permalink
Posted by Christian Zimmermann

The end of print journals?

December 8, 2007

The (US) Association of Research Libraries released a few days ago a report entitled “The E-only Tipping Point for Journals: What’s Ahead in the Print-to-Electronic Transition Zone” (pdf). It makes the argument that sooner or later every publisher will turn to an electronic-only format in the face of rising (relative) costs of print formats. Currently, we are in a transition period where most journals went from print-only to print and electronic, and it is predicted that with 5 to 10 years, the printed journals will be only from the most specialized and small ones who cannot afford the fix cost of setting up the electronic editions. Another feature of the transition is the large proportion of new journals that do not even bother with a print edition.

This discussion largely pertains to university press publishing, but can probably be extended to commercial publishing. Indeed, commercial publishers show signs that they want to discourage print editions, either through their subscription price structure or by modifying subscriptions to be by default electronic-only. In Economics, the dissemination of research, in terms of readership, is dominated by pre-prints (working or discussion papers) that have gone all electronic for some time now, with only few exceptions. As far as I know, nobody regrets the period of the all printed working papers: they were difficult to obtain unless you were in the “club”, only few institutions had a systematic (but costly) way to disseminate them, and only established researchers had any chance of being read through this medium. People would even travel to some libraries to consult their working paper collections. Today, research is much more widely disseminated and researchers from outside the elite institutions have a better chance to follow and contribute to the research frontier. We hope RePEc has contributed to this democratization. Never has been the use of electronic pre-prints as widespread as now, possibly at the cost of reducing journals to historical records of research. Well, journals also act as gateways through peer-review, but you sometimes have to wonder about this as well when hearing all the complaints about this process.

A few interesting numbers from the study: 60% of 20,000 per-reviewed journals are available in electronic format, library-provided electronic editions are at least ten times more read than print ones, only 30% of library subscriptions are print only.

Leave a Comment » | Dissemination of research in Economics | Tagged: , , | Permalink
Posted by Christian Zimmermann

Important changes in RePEc rankings

December 2, 2007

Two important changes regarding rankings:

  1. There are now rankings of authors and institution by field. As discussed before, the procedure is the following: identify relevant authors by the number of papers announced in NEP field reports. Past a threshold (currently 25% of all announced papers or 5 announced papers), the author is considered specialist of her field (see discussion. For institutions, each affiliated author contributes with a weight corresponding to the proportion of papers announced in the field, irrespective whether the threshold was met (see discussion).
  2. For rankings within regions (and within fields, described above), authors and institutions are not anymore ranked by picking them from the “grand list”, i. e., the ranking of all authors or institutions. Rather, the ranking is performed within the respective subgroup: for example, authors are ranked with the others of the same region according to all criteria, and then ranking points are aggregated. This had already been done a few months ago for women, which led complaints about this particular ranking to virtually disappear…

Soon, we will also add citation rankings for recent papers. An announcement will be made on this blog in a week or two. Note that all these rankings are experimental and subject to changes. We welcome discussions and suggestions about them in the comment section.

16 Comments | RePEc rankings | Permalink
Posted by Christian Zimmermann

RePEc in November 2007

December 1, 2007

Every month, a short summary of what happened with RePEc is sent to the RePEc-announce mailing list. I will also put that message, slightly adapted, on this blog.

The new feature of the month has to do with rankings: There are now rankings of institutions and authors by field. Also, there have been procedural changes in the rankings within region. See the discussion of these items elsewhere on the blog.

We continue with fast paced additions of material. This year alone, over 100,000 items have been added to the bibliographical database. Accordingly, the RePEc web services are more popular than ever, establishing new traffic records: 697,596 file downloads and 2,516,310 abstract views within a month. This leads us to the thresholds we have passed this month:

2,500,000 abstract views in a single month
550,000 items listed
500,000 downloaded software components
325,000 articles listed
275,000 items with abstracts
225,000 papers listed
125,000 items with references
100,000 papers with references
90,000 articles with citations

Leave a Comment » | Workings of RePEc | Permalink
Posted by Christian Zimmermann

Thanksgiving to Volunteers: Ivan Kurmanov

November 21, 2007

Ivan KurmanovAs the United States are celebrating Thanksgiving, it is time to celebrate our volunteers. With this post, we hope to start a regular feature that highlights the work that our volunteers do, sometimes unseen from the general public. RePEc is all built on volunteer effort, and we hope this feature will help these crucial people to get the recognition they deserve.

Today, we want to recognize Ivan Kurmanov, who has just left the RePEc team after being on board for over 10 years. As a undergraduate Economics major at the Belarussian State University in 1996, he noticed the work being led by Thomas Krichel at the now defunct NetEc, the precursor of RePEc. Thinking it was a great initiative, he volunteered to help out. Thomas quickly found something to get him busy: Writing ReDIF-perl, a perl module that validates the data contributed to RePEc by the participating archives and then massages the data for uses by RePEc services. ReDIF-perl has proven to be tremendously useful. Then, Ivan tackled the RePEc Author Service (then called HoPEc) that needed a lot of work, especially to iron out various bugs and performance issues. This was no easy task, as HoPEc was programmed in C++, while all other components of RePEc run with perl. Eventually, it became clear that a complete code rewrite became necessary.

Thomas managed to find a grant from the Open Society Institute to provide an open source author registration system, and Ivan started working full time on it. This is how the current RePEc Author Service was created, based on ACIS, which is now open source under a GPL license. ACIS performs quite complicated tasks, like pattern matching of names, which may include accents and other marks, or citation analysis with surprising efficiency. Another remarkable aspect of this project is that it is extremely well documented, unlike many other RePEc projects, unfortunately.

While technically Ivan was paid for part of his time with RePEc, we should still consider him a volunteer given all the tremendous work he has performed that went well beyond what would have been expected from the little money the grant provided. Also, he had to cope with often shaky Internet connections in Belarus. Ivan now works full time as a programmer, and we hope he will still listen in on RePEc and give his advice, and occasional fixes. Of course he leaves a void, and while Thomas Krichel is currently providing interim coverage, we are looking for a new volunteer to maintain and expand the code behind ACIS and the RePEc Author Service.

Leave a Comment » | RePEc volunteers, Workings of RePEc | Tagged: , , , | Permalink
Posted by Christian Zimmermann

« Previous Entries
Next Entries »