“The Closed Marketplace of Economic Ideas,” a Rebuttal

January 8, 2016

In a Project Syndicate column, Federico Fubini makes the argument that the intellectual leaders in economics from ten years ago are still the leaders of today, and this despite the fact that we have had a financial crisis that was not predicted by the profession. I do not agree with this column on several fronts. As the argument was made using RePEc data, I feel obligated to set the record straight.

One may discuss whether the economics profession has really not seen the crisis coming. But even if this crisis was unforeseen, it is wrong to argue on principle that the best economists from ten years ago should not be considered to be the best today. Indeed, it is not the case that the whole profession is focussed on predicting financial or economic crises. Economics has much more to offer, just see the list of recent Nobel Prize winners or the large variety of fields covered by NEP, RePEc’s research alert service. Most of those fields have nothing to do with crises. For example, the leaders in auction theory from ten years ago are likely to be leaders now, the same applies to development economics, empirical labor economics, or environmental economics. Ten years is short in the evolution of scientists.

But beyond this mischaracterization of what the economics profession does, there is the issue with the use of the data to prove the point. Fubini uses the ranking of economists provided by IDEAS/RePEc, taking the December 2006 and the September 2015 rankings. But before looking at them, one has to understand what they measure. They are an aggregation of 23 (2006) or 35 (2015) criteria, almost all of which pertain to the lifetime output of the economists. Those that are not are four looking at readership statistics for the last months, and in the recent rankings six criteria discounting citations by their age. Basically, the accumulated research and citations that were considered in 2006 are also considered in 2015. There is obviously going to be high persistence among the best economists. And keep in mind that critiquing someone will earn him a citation.

What you want to do is using two datasets that do not overlap, one that considers publications until 2006, and one since that year. One can get very close by using another ranking, the one that considers only the publications from the last 10 years. In the following I will use the one that was published today and pertains to December 2015. We have thus only one year of overlap in the publication dates. And the results look quite different (I did this in a couple of hours, I cannot vouch the numbers are totally correct).

Incumbency rate by cohort
Cohort Fubini Better
Top 10 90% 70%
Top 20 95% 75%
Top 50 98% 72%
Top 100 94% 61%
Top 200 65% 42%

The rest of the arguments in the Fubini column also change quite a bit once you look at better data.

You must not be an economist. In fact, Lucas and Fama both moved up in the RePEc rankings during the period I examined, from 30 to nine and from 23 to 17, respectively. And the persistence at the top is striking across the board. Among the top ten economists in September 2015, six were already there in December 2006, and another two were ranked 11 and 13.

Lucas (Robert E. Jr.) and Fama actually dropped, Lucas from 30 to 33, Fama from 23 to … not being ranked in the top 10%. And in the top ten for December 2015, only three where there in 2006, another two ranked 11 and 13 in 2006 (the same).

Mobility in the RePEc rankings remains subdued even after widening the sample. For example, of the top 100 economists in September 2015, only 14 were absent from the much wider top 5% in 2006, and only two others had advanced more than 200 spots over the previous decade. Among those recently ranked from 101 to 200, just 24 were not in the top 5% in 2006, and only ten others had moved up by more than 200 places. The rate of renewal among the 200 most influential economists was as low as 25% – and just 16% among the top 100 – during a decade in which the explanatory power of prevailing economic theory had been found severely wanting.

Again with better data, this looks different. From the 2015 top 100, 41 were absent from the top 5% in 2006. Nine advanced more than 200 spots. For 101 to 200, 45 were not in the top 5%, and seven moved up more than 200 spots. Using Fubini’s definitions, the renewal rate is thus 51% in the top 200 and 50% in the top 100. Hardly a stagnation.

In the rankings of economists, by contrast, criteria such as gender or geographic origin confirm the overall inertia. Only four women made the RePEc top 200 in September 2015, compared to three in December 2006, and two were included on both lists. Likewise, emerging countries – which represent more than 90% of the world’s population, three-quarters of global GDP growth over the last decade, and nearly half of total income in current dollar terms – supplied just 11 of the top 200 economists in September 2015, up from ten in December 2006. And ten of those 11 – three Iranians, four Indians, two Turks, and one Chinese – have lived and worked in the US or the United Kingdom since their student days.

With better data, this changes as well. There are now seven women. Still too few, though. There are 18 economists from emerging countries: two Turks, one Egyptian, seven Indians, two Iranians, two Pakistani, one from Cameroun, two Chinese, one Bangladeshi. Eight of them live in an emerging economy.

I’ll let the reader judge whether there is still a “closed market place for ideas in economics.” But the picture certainly looks different from what Fubini seems to imply.

Economics Replication Wiki now on IDEAS

July 16, 2014

A major part of the scientific process is the replication of previous studies, something necessary to confirm that things were done right, that they are not sensitive to details and that results have not changed with the passage of time, either because the methods got better or the data has evolved. Unfortunately, there is little replication in economics, and if there is some, it is difficult to publish it. One can theorize why this may be the case, but it is clear replication studies are little valued and not particularly welcome in journals. It is also quite difficult to determine whether a particular study has been replicated.

To help with all that, the Center for Statistics at the University of Göttingen (Germany) has launched a Wiki to index replicated and replicating studies in economics, with funding from the Institute for New Economic Thinking. As it is a wiki, it is crowd-sourced in the sense that any registered person can amend the records, and in particular add replication studies. One can also add to a list of articles published in top journals that should warrant replication and vote (anonymously) from that list (current winners).

The listings on this Replication Wiki are now indexed on IDEAS as well. The principle is similar to the indexation of Wikipedia articles: if a study on the Wiki has a link to IDEAS (or EconPapers, IDEAS will link back. Those adding or amending entries on the Wiki are thus encouraged to link to the IDEAS abstract page to create the backlink on IDEAS.

As any crowd-sourced project, the Replication Wiki will only live from the participation from the public. If you know of replication studies, consider spending a few minutes and add to this wiki.

The Job Market Paper archive

September 19, 2013

A graduating economics PhD or doctoral student who is looking for a job in academia or policy circles is typically doing so with a “job market paper.” The JMP is the one that many recommendation letters from faculty focus on, it is the one that is mostly talked about in job interviews, and it is presented during campus visits. It is thus fair to say that the JMP is the best this student has done so far, and a lot of effort goes into this paper. Shouldn’t this work then be more widely disseminated than a few recruiting committees?

We are thus introducing the Job Market Paper archive on RePEc. Job candidates can upload their paper, which gets the standards treatment of any new working paper in RePEc: it gets listed on the many services using RePEc data, including the websites EconPapers and IDEAS, as well as the email notification service NEP. In addition, the papers are hosted by a RePEc server for posterity. This is important, as job market candidates tend to find jobs and often move their web page as a consequence, resulting in broken links. Finally, the presence of the papers in this series clearly identifies the author as a new economist one may want to look at for a hire. Recruiters can simply follow what is new in this archive.

As expected, certain restrictions apply. To learn more, see here.

Note for that for those who are not on the job market and do not have access to a local working paper series that participates in RePEc (instructions), MPRA is still available.

The Purpose of Journals

February 14, 2013

The editor of the Economics Bulletin, John Conley, has noted that many things go wrong with economic journals. Here is the abstract of his letter:

This letter calls attention a recent trend in economics publishing that seems to have slipped under the radar: large increases in submissions rates across a wide range of economics journals and steeply declining acceptance rates as a consequence. It is argued that this is bad for scholarly communication, bad for economics as a science, and imposes significant and wasteful costs on editors, referees. authors. and especially young people trying to establish themselves in the profession. It is further argued that the new “Big Deal” business model used by commercial publishers is primarily responsible for this situation. Finally it is argued that this presents a compelling reason to take advantage of new technologies to take control of certifying and distributing research away from commercial publishers and return it to scholarly community.

According to Conley,

The purpose of academic journals is to facilitate scholarly communication, filter for errors, and maintain the record of scientific advance.

This is, in my opinion, an idealized conception that does not reflect  the purpose of economic journals anymore. For economic research, the current economic journals are largely redundant. Conley himself notes this:

I seldom actually read journals  any more. I research topics using Google Scholar, RePEc, SSRN, and so on. It is inconvenient to sign up  with publishers to get tables of contents emailed to me or to login to my university’s library web portal to  search a journal issue by issue. I find it adds very little value over a more general search in any event. In  short, certification remains important to help people gain tenure and promotion and to get a sense of the  quality and centrality of individual scholars. However, neither certification by a journal, nor the collection  of similar papers within the bound or even electronic pages of a specific journal has very much meaning to  me when I am trying to understand where the debate in a subfield is at any given moment. As a result, I  was beginning to come to the conclusion that while they are irritating, commercial publishers are “mostly  harmless” to the research enterprise itself as publishing itself is becoming mostly irrelevant.

This coincides with my own observation: researchers don’t need journals. The main purpose of the journals is currently to ease the work of hiring committees. People publish in order to get a job. The wish to communicate new findings appears secondary in most cases.

Journals could serve worthier aims, however: they are needed by students, college teachers, and others who would like to obtain reliable information but can not as easily  separate the wheat from the chaff as active researchers can.

The important point Conley is making is, however, that the current journal system, although largely irrelevant for research, is nevertheless

bad for scholarly communication, bad for economics as a science, and imposes significant and wasteful costs on editors, referees. authors. and especially young people trying to establish themselves in the profession.

I fear, however, that John Conley’s suggestion to increase the number of journals would not improve the situation very much. As long as hiring committees use the reputation of journals, rather than the reputation of individuals,  a useful system of  “communication, filter for errors, and maintain the record of scientific advance” is practically blocked.

What can be done besides increasing the number of journals? Here some further suggestions.

1. Hiring committees can restrict the number of papers to be considered for judging an applicant to, say, three and disregard all other writings. This may help to reduce the number of publications and thereby reduce the need for further journals; it would also tilt the quality-quantity trade-off in favor of quality. (I think this has been a practice in Berkeley.)

2. Hiring committees that feel incompetent to judge the substantive quality of a contribution and have to resort to statistics of some sort may turn to citation counts of individual authors, as obtainable through  Google Scholar, Web of Science, or RePEc). This is a better solution than the the current practice of relying on the prestige of journals and would take account of the fact that  many papers in top journals are not so good, and medium-quality journals publish excellent articles.

Introducing the RePEc Genealogy

September 28, 2012

A new RePEc service is born, the RePEc Genealogy, which collects and displays the academic family tree for economics. This is a crowd-sourced initiative, which means that any person registered with the RePEc Author Service can contribute information about oneself and others: institution and year where the terminal degree was obtained, advisor, and possibly students.

The collected data will be used in various ways. Currently, author profiles on IDEAS link back to relevant genealogy pages. The directory of institutions, EDIRC, has lists of alumni and their publications. In the future, when we have critical mass, we can use this data to properly rank young economists. Currently, we infer there start in the profession by dating their first publication. A graduation year would be more appropriate. Also, the genealogy data would also allow us to evaluate graduate departments.

Help make this service useful. You can add information by logging in using your RePEc Author Service credentials here. Thank you!

Plagiarism in Economics

February 16, 2011

We are all aware that plagiarism exists, and RePEc has helped expose quite a few cases through its open bibliographies. But sanctions for plagiarism are rather limited. An offended party may complain with the administration of the accused offender, in some cases without consequences, and in others with sanctions that can lead to dismissal. But the now convicted offender may simply take a new job as if nothing happened, the new employer being oblivious to what happened.

Economics does not have an ethics board that could deal with such cases beyond the current employer of an accused offender. There is now a proposal to create a committee dedicated to plagiarism. This committee would examine cases and vote on sanctions which may go all the way to publicly exposing the plagiarist. A group of volunteers have are discussed a simple set of procedures. Over the next month, the plan is to solicit comments from the public through this blog and call for further volunteers to participate in the committee. After that the committee would become active and deal with any new plagiarism cases that come to its attention. Please contact any current member to participate.

To view the current proposal and committee members, see a simple and bare bones site at plagiarism.repec.org. The committee awaits your reaction. Beyond comments, you can also vote your reaction below.

The FRED Network, a social network for economists

June 12, 2010

Guest post by Richard Anderson

On June 3, 2010, the Research Division of the Federal Reserve Bank of St. Louis introduced the first Internet social networking web site for economics and business. The new web site <www.thefrednetwork.com> is a namesake of the Bank’s popular FRED data service, the most widely used free source of United States economic data on the Internet.

The FRED Network will permit economists and the public to communicate more easily with the data analysts that support FRED, as well as with economists both at St. Louis and elsewhere in the world. The web site is a “dual-threaded” design, meaning each user can select both the topics of interest and the site’s users with whom they wish to communicate. Unlike unfiltered Internet blogs, users will not have to sort through commentary of little interest to locate useful information.

Social networking web sites help people find others with similar interests, exchange knowledge about both data and research projects, solve problems, and develop new ideas. Also, companies learn from their customers by reading customer comments.

“The FRED Network will improve communication between the Bank staff and our customers,” said one staff member. “In the past, we answered most questions via email and only one person saw the information. Now, we can answer within The FRED Network and all customers with that interest will see the information.”

Social networking brings together like-minded people, whether users of FRED data or top-level professionals pursuing complex research.

A unique feature of The FRED Network is the ability for each user to write their own blog. Each user’s blog is available to all other users who sign up to receive that user’s commentary. The unique design of The FRED Network allows each user to read blogs from selected users while excluding ones they do not care to read. This feature allows users who are passionate to write about their interests and expertise, while allowing users who do not wish to receive that commentary need not do so.