Thanksgiving to Volunteers: Ivan Kurmanov

November 21, 2007

Ivan KurmanovAs the United States are celebrating Thanksgiving, it is time to celebrate our volunteers. With this post, we hope to start a regular feature that highlights the work that our volunteers do, sometimes unseen from the general public. RePEc is all built on volunteer effort, and we hope this feature will help these crucial people to get the recognition they deserve.

Today, we want to recognize Ivan Kurmanov, who has just left the RePEc team after being on board for over 10 years. As a undergraduate Economics major at the Belarussian State University in 1996, he noticed the work being led by Thomas Krichel at the now defunct NetEc, the precursor of RePEc. Thinking it was a great initiative, he volunteered to help out. Thomas quickly found something to get him busy: Writing ReDIF-perl, a perl module that validates the data contributed to RePEc by the participating archives and then massages the data for uses by RePEc services. ReDIF-perl has proven to be tremendously useful. Then, Ivan tackled the RePEc Author Service (then called HoPEc) that needed a lot of work, especially to iron out various bugs and performance issues. This was no easy task, as HoPEc was programmed in C++, while all other components of RePEc run with perl. Eventually, it became clear that a complete code rewrite became necessary.

Thomas managed to find a grant from the Open Society Institute to provide an open source author registration system, and Ivan started working full time on it. This is how the current RePEc Author Service was created, based on ACIS, which is now open source under a GPL license. ACIS performs quite complicated tasks, like pattern matching of names, which may include accents and other marks, or citation analysis with surprising efficiency. Another remarkable aspect of this project is that it is extremely well documented, unlike many other RePEc projects, unfortunately.

While technically Ivan was paid for part of his time with RePEc, we should still consider him a volunteer given all the tremendous work he has performed that went well beyond what would have been expected from the little money the grant provided. Also, he had to cope with often shaky Internet connections in Belarus. Ivan now works full time as a programmer, and we hope he will still listen in on RePEc and give his advice, and occasional fixes. Of course he leaves a void, and while Thomas Krichel is currently providing interim coverage, we are looking for a new volunteer to maintain and expand the code behind ACIS and the RePEc Author Service.

Leave a Comment » | RePEc volunteers, Workings of RePEc | Tagged: , , , | Permalink
Posted by Christian Zimmermann

More on peer review and blogging

November 15, 2007

A community of research bloggers tired of being confused with “news, politics, family, bagpipes, and so on” blogs has started its own blog at BPR3 (Bloggers for Peer-Reviewed Research Reporting). The idea is to encourage peer review bloggers to certify themselves and use an icon on their blog. We discussed earlier whether such blogs would be appropriate in Economics, along with some past experiences. Other fields seem to have some active research blogs. Do we have any blog in Economics that would qualify?

Inside HigherEd has two articles on academic blogging, one discussing how painful it is, the other how great it is. Both authors are graduate students, but they can still offer interesting perspectives to more seasoned researchers tempted by academic blogging. A few excerpts:

Over the past three years, I’ve learned what it’s like to write in a way most academics never have: namely, for an audience. If this seems like a simple point, that’s because it is. Nor is it one of those profoundly simple points, either: it’s straight simple. When a blogger sits down to slave on her dissertation, article, or book, she doesn’t turn her back on the public sphere. Because in the end, the public sphere is us.

I’m talking about the communities we currently have, only five years in the future, when we’re scattered around the country, unable to communicate face-to-face, but still connected, still intellectually intimate, because we’ll still regularly be engaged with each other’s thoughts. But I’m not only talking about us. There’s no reason our community needs to consist solely of people we knew in grad school. Why not write for people who don’t already how you think about everything? Why not force yourself to articulate your points in such a way that strangers could come to know your thought as intimately as your friends from grad school do?

More than formatting issues, however, I think that everyone needs to realize that having a productive conversation in an online format is very hard work, which is why it happens so rarely. Many bloggers can point out online conversations in which they were pushed to think in a new direction or got genuinely valuable feedback on a question, but as with all human endeavors, there is a high percentage of dross to go along with the occasional gold. Policing comments is a difficult job, and efforts to keep conversations on-topic or ensure that contributors have some substantial knowledge to share will often cause resentment in light of the “democratic” leanings of online communities. All this is on top of the obvious problems with online interaction as opposed to in-person conversations.

As more and more academic resources become available online, hopefully academic blogs will begin to fill a role analogous to the political blogs that link to and comment on particular news stories — that is, bringing new scholarly research to the attention of an interdisciplinary audience. I hope that events like this will help to push more journals toward open-access electronic formats. Failing that, however, academic blogs seem to me to be best-suited as a social outlet for academics who would otherwise feel isolated, creating camaraderie and supplementing the social aspects of disciplinary conferences.

There is also a discussion on this topic on the blog of the Association of College & Research Libraries.

The RePEc blog does not consider itself to be part of the research blog community, indeed our focus is not research but the dissemination of research. Hence, we are interested in understanding the feasibility and the interest in research blogs in Economics. And if any research blogs appear (or already exist) in Economics, RePEc would be more than happy to feature them on this blog and possibly elsewhere.

3 Comments | Dissemination of research in Economics | Permalink
Posted by Christian Zimmermann

Ranking Institutions Within Fields

November 9, 2007

In previous posts, we discussed how to categorize authors by field and then how to rank them within fields. These discussions are still open and I can still be convinced to change the procedure. Today, I would like to make a proposal regarding the ranking of institutions within fields.

We have several options regarding how to count people from an institution for a specific field. In the examples, I assume that author A has 10, or 50%, of his works in macroeconomics, B 3 or 30%, C 10 or 20%, D 1 or 10% and A 0%. I also assume that the 25% and 5 rule applies, as discussed in the post on categorizing authors. Thus, under these rules, authors A, B, and C are considered macroeconomists. The options are:

  1. Count fully all authors considered within the field: A+B+C.
  2. Count all authors considered within the field proportionally to their involvement in the field: 0.5A+0.3B+0.2C.
  3. Count all authors, irrespective whether they qualify as specialists, proportionally to their involvement in the field: 0.5A+0.3B+0.2C+0.1D.

My preference is for option 3. The reasons are the following. The first option fails to properly differentiate between strong specialists and marginal ones. This may also have been a concern when ranking authors, but the issue there was the high volatility of weights at the author level. At the institutional level, this is less of a concern as several authors are aggregated. Note also that only the top 20% institutions will be listed anyway, thus I expect all of them to have several authors within the field. Thus I prefer option 3 over option 1. Then I prefer option 3 over option 2 because it allows to count for authors that may not be specialists but still may contribute to enriching the field. Think for example when a prospective graduate student compares programs. While she cares about the specialists of her field of interest, she may also care about those faculty on the fringes of the field.

Of course I am open to suggestions and can still be swayed to to change my opinion. I plan on implementing this for next month.

1 Comment | RePEc rankings | Permalink
Posted by Christian Zimmermann

Further Thoughts on “New Peer Review Systems”

November 5, 2007

Several thoughts on various points raised in New Peer Review Systems and the comments that followed.

  • In a sense, the lag in the review process might be optimal. A publication of most any sort is valuable to the author and one in a leading journal of course has a very substantial return. Journals thus have a good reason to deter papers that aren’t at all appropriate; this was pointed out by Ofer Azar, “The Slowdown in First-Response Times of Economics Journals: Can It Be Beneficial?,” Economic Inquiry, 2007, 45 (1). The constraint here would seem to be editor’s and referees’ time. In economics, the most common cost that journals impose on authors is a lengthy review process. I’d hazard a guess that bepress gets around this by their ranking of papers into different tiers; they don’t have to deter less than stellar papers as they’ll likely get a home there. This is combined with their system where authors who submit there agree to review two papers quickly (a nice example of a virtuous cycle). Another way to speed up the referee process is a system where any reader can submit comments on a paper. But, as Christian points out, this doesn’t seem to attract many comments. It turns out I’ve looked a bit at this and found 5 journals that have tried a reader rating system and none have attracted a sufficient number of comments to make it fly. From here, one option is something that ranks papers after they’ve been out, such as citations. Paul Ginsparg has some thoughts on one approach, as does Hal Varian (now the chief economist at Google). But, these might take years to generate sufficient data to render a judgment on a paper. I think many of us want something that is quicker.

    Another possibility is something like Faculty of 1000 in biology and medicine where a level of review beyond journals takes place. I very much like the summaries in their sample web pages; you don’t see that in economics. One could imagine it working on top of our working paper culture. But, I wonder if some of their success comes from the grant culture in these fields as this is a fee-base service that seems to pay the reviewers? How might one set up something similar in economics with our working paper culture? Journals would likely see it as preempting their role.

  • I agree with Preston that perhaps the most interesting part of Economics E-Journal is the open review system (all can read reviews) and the feature that allows authors to publicly respond to referee reports. Both would seem to give referees correct incentives. I would think that journals could implement this quickly and easily at low cost. Also, while deep thinking and working through a paper you are writing is extremely valuable, so is getting feedback and discussing a paper and the ideas in it. I have a first draft of a paper on these topics (see below) but in writing this blog entry I have developed some new insights. I wouldd add that prompt discussion is something that the Internet can aid for those of us without local colleagues in our fields.A very minor point on his post: if you count economics journals by the number in EconLit, there are about 1,240. In short, most any paper should certainly be published!
  • I certainly second Christian’s point about blogs and research. All the economics blogs I know of do not discuss serious research that much if at all. Much more common are discussions of current economic events and policy that members of the public find interesting. I don’t know of a one where someone might say, “Say, that paper by Sam Jones in Computational Economics” is interesting because the algorithm he used to calculate…” Also, debates by papers take years given the time to write one and respond. This seems rather silly given today’s technology; after all, journals in the their current form came about when information traveled at the speed of a horse or ship. Perhaps a blog is too quick for complete works, but I understand in law their use is leading to a relative decline in the importance of law journals. One example (first paper found in a Google search) is Guest Blogger: The Start of the Supreme Court’s 2007-08 Employment Discrimination Docket: Federal Express Corporation v. Holowecki. Yes, it discusses current events, but the topic is one that only specialists would seem to care about. You do not seem to see such in economics blogs that I am aware of.

Where does this leave RePEc? Well, I’m not really sure. It is hard to change norms in a field and I am not sure that RePEc could swing it. But, I do not think that a comment system on individual papers would get that much traction. Somehow you want to get the judgment of peers (for promotion, tenure, and annual raises) in a speedy manner. Those two criteria seem to be at odds with each other.
It turns out I have done some thinking on these issues; at the risk of self-promotion they can be found in Next Steps in the Information Infrastructure in Economics. Note that this draft was written for a conference of non-economists, so some parts will strike a very obvious cord to an economist’s ear. I have also have yet to incorporate some very useful comments that Christian kindly wrote. In a nice mix of blog and papers, further comments on the paper are greatly appreciated.

9 Comments | Dissemination of research in Economics, Economics Profession | Permalink
Posted by Bill Goffe

Ranking Authors Within Fields

November 4, 2007

As discussed earlier on this blog, we are trying to categorize authors by fields. The current proposal is to consider someone in a field if either 5 or 25% of her papers announced in NEP where announced in the relevant field report. These parameters are still open for debate, and the reader is welcome to weigh in. Once this is done, the next step is use this methodology to rank authors within fields. There are several ways in which this can be done.

  1. Take the unweighted list of authors in each field. This means the following: Take the list of authors as categorized by the parameters mentioned above. Just rank them within the list assuming the same weight to each.
  2. Do the same, but with weights. Those weights would correspond to the share of the field within each author’s work. So an author who has 4 of 12 papers announced in NEP announced in the NEP-MAC report would have one third of his scores count towards the ranking among macroeconomists.
  3. Do the same, but not restrict to those have passed the threshold to qualify for a field (currently 5 papers or 25% of papers announced in NEP).

My preference would go for the first option. I do not think that the field weights are that precise to allow using them for authors. Also, the ranking of author in a field may drop with the other options for the sole reason that he has published a working paper in another field.

I intend to have the first field rankings available in a month, so voice your opinion before then.

4 Comments | RePEc rankings | Permalink
Posted by Christian Zimmermann

Seeking Volunteers

November 3, 2007

The RePEc project is entirely supported by volunteer effort. Indeed, no one is paid in any way for working on RePEc, in fact RePEc does not even have a budget. A core team is heavily involved, on spare time, and many others participate in smaller, but nonetheless important ways. We are always interested in renewing the blood in the team, so if you wish to help out and provide useful services to the profession, join the team in whatever capacity you can provide. Several of the tasks require familiarity with programming (in particular perl) and with Linux systems, but not all tasks. And like for everyone else on the team, this can be learned on the spot.

Watch this space as we announce calls for volunteers for specific projects. More generally, we are interested in anybody who could help provide some redundancy. Indeed, many projects rely currently on one person, and if this person were not able to help anymore, we would have to scramble. Thus, if you are particularly interested in one or the other aspect of RePEc, why not get involved?

We also want to start a few projects and are looking for people to help out there. Again, watch this space for announcements. Or make your availability known either in the comment section or through repec@repec.org.

Leave a Comment » | RePEc volunteers | Tagged: | Permalink
Posted by Christian Zimmermann

RePEc in October 2007

November 2, 2007

Every month, a short summary of what happened with RePEc is sent to the RePEc-announce mailing list. I will also put that message, slightly adapted, on this blog.

Two new items this month: first we now have a blog, at https://blog.repec.org/ where we discuss issues relevant to RePEc and the Economics community in general. Second, we now attempt to classify authors by fields. See results at http://ideas.repec.org/i/e.html.

Coverage has tremendously increased this month fuelled by 15 archives and the old archives: over 34,000 new items are now listed. Traffic was also very high, setting records on several servers. Total traffic was 642,109 file downloads and 2,280,187 abstract views.

Quite obviously, we passed a few thresholds under these circumstances:

120,000,000 cumulated abstract views
400,000 items with download
300,000 articles listed
250,000 articles with download
90,000 distinct papers announced through NEP
75,000 papers with citations
17,500 NEP reports
1,750 chapters listed

Leave a Comment » | Workings of RePEc | Permalink
Posted by Christian Zimmermann