Economics Replication Wiki now on IDEAS

July 16, 2014

A major part of the scientific process is the replication of previous studies, something necessary to confirm that things were done right, that they are not sensitive to details and that results have not changed with the passage of time, either because the methods got better or the data has evolved. Unfortunately, there is little replication in economics, and if there is some, it is difficult to publish it. One can theorize why this may be the case, but it is clear replication studies are little valued and not particularly welcome in journals. It is also quite difficult to determine whether a particular study has been replicated.

To help with all that, the Center for Statistics at the University of Göttingen (Germany) has launched a Wiki to index replicated and replicating studies in economics, with funding from the Institute for New Economic Thinking. As it is a wiki, it is crowd-sourced in the sense that any registered person can amend the records, and in particular add replication studies. One can also add to a list of articles published in top journals that should warrant replication and vote (anonymously) from that list (current winners).

The listings on this Replication Wiki are now indexed on IDEAS as well. The principle is similar to the indexation of Wikipedia articles: if a study on the Wiki has a link to IDEAS (or EconPapers, IDEAS will link back. Those adding or amending entries on the Wiki are thus encouraged to link to the IDEAS abstract page to create the backlink on IDEAS.

As any crowd-sourced project, the Replication Wiki will only live from the participation from the public. If you know of replication studies, consider spending a few minutes and add to this wiki.


The Job Market Paper archive

September 19, 2013

A graduating economics PhD or doctoral student who is looking for a job in academia or policy circles is typically doing so with a “job market paper.” The JMP is the one that many recommendation letters from faculty focus on, it is the one that is mostly talked about in job interviews, and it is presented during campus visits. It is thus fair to say that the JMP is the best this student has done so far, and a lot of effort goes into this paper. Shouldn’t this work then be more widely disseminated than a few recruiting committees?

We are thus introducing the Job Market Paper archive on RePEc. Job candidates can upload their paper, which gets the standards treatment of any new working paper in RePEc: it gets listed on the many services using RePEc data, including the websites EconPapers and IDEAS, as well as the email notification service NEP. In addition, the papers are hosted by a RePEc server for posterity. This is important, as job market candidates tend to find jobs and often move their web page as a consequence, resulting in broken links. Finally, the presence of the papers in this series clearly identifies the author as a new economist one may want to look at for a hire. Recruiters can simply follow what is new in this archive.

As expected, certain restrictions apply. To learn more, see here.

Note for that for those who are not on the job market and do not have access to a local working paper series that participates in RePEc (instructions), MPRA is still available.


The Purpose of Journals

February 14, 2013

The editor of the Economics Bulletin, John Conley, has noted that many things go wrong with economic journals. Here is the abstract of his letter:

This letter calls attention a recent trend in economics publishing that seems to have slipped under the radar: large increases in submissions rates across a wide range of economics journals and steeply declining acceptance rates as a consequence. It is argued that this is bad for scholarly communication, bad for economics as a science, and imposes significant and wasteful costs on editors, referees. authors. and especially young people trying to establish themselves in the profession. It is further argued that the new “Big Deal” business model used by commercial publishers is primarily responsible for this situation. Finally it is argued that this presents a compelling reason to take advantage of new technologies to take control of certifying and distributing research away from commercial publishers and return it to scholarly community.

According to Conley,

The purpose of academic journals is to facilitate scholarly communication, filter for errors, and maintain the record of scientific advance.

This is, in my opinion, an idealized conception that does not reflect  the purpose of economic journals anymore. For economic research, the current economic journals are largely redundant. Conley himself notes this:

I seldom actually read journals  any more. I research topics using Google Scholar, RePEc, SSRN, and so on. It is inconvenient to sign up  with publishers to get tables of contents emailed to me or to login to my university’s library web portal to  search a journal issue by issue. I find it adds very little value over a more general search in any event. In  short, certification remains important to help people gain tenure and promotion and to get a sense of the  quality and centrality of individual scholars. However, neither certification by a journal, nor the collection  of similar papers within the bound or even electronic pages of a specific journal has very much meaning to  me when I am trying to understand where the debate in a subfield is at any given moment. As a result, I  was beginning to come to the conclusion that while they are irritating, commercial publishers are “mostly  harmless” to the research enterprise itself as publishing itself is becoming mostly irrelevant.

This coincides with my own observation: researchers don’t need journals. The main purpose of the journals is currently to ease the work of hiring committees. People publish in order to get a job. The wish to communicate new findings appears secondary in most cases.

Journals could serve worthier aims, however: they are needed by students, college teachers, and others who would like to obtain reliable information but can not as easily  separate the wheat from the chaff as active researchers can.

The important point Conley is making is, however, that the current journal system, although largely irrelevant for research, is nevertheless

bad for scholarly communication, bad for economics as a science, and imposes significant and wasteful costs on editors, referees. authors. and especially young people trying to establish themselves in the profession.

I fear, however, that John Conley’s suggestion to increase the number of journals would not improve the situation very much. As long as hiring committees use the reputation of journals, rather than the reputation of individuals,  a useful system of  “communication, filter for errors, and maintain the record of scientific advance” is practically blocked.

What can be done besides increasing the number of journals? Here some further suggestions.

1. Hiring committees can restrict the number of papers to be considered for judging an applicant to, say, three and disregard all other writings. This may help to reduce the number of publications and thereby reduce the need for further journals; it would also tilt the quality-quantity trade-off in favor of quality. (I think this has been a practice in Berkeley.)

2. Hiring committees that feel incompetent to judge the substantive quality of a contribution and have to resort to statistics of some sort may turn to citation counts of individual authors, as obtainable through  Google Scholar, Web of Science, or RePEc). This is a better solution than the the current practice of relying on the prestige of journals and would take account of the fact that  many papers in top journals are not so good, and medium-quality journals publish excellent articles.


Introducing the RePEc Genealogy

September 28, 2012

A new RePEc service is born, the RePEc Genealogy, which collects and displays the academic family tree for economics. This is a crowd-sourced initiative, which means that any person registered with the RePEc Author Service can contribute information about oneself and others: institution and year where the terminal degree was obtained, advisor, and possibly students.

The collected data will be used in various ways. Currently, author profiles on IDEAS link back to relevant genealogy pages. The directory of institutions, EDIRC, has lists of alumni and their publications. In the future, when we have critical mass, we can use this data to properly rank young economists. Currently, we infer there start in the profession by dating their first publication. A graduation year would be more appropriate. Also, the genealogy data would also allow us to evaluate graduate departments.

Help make this service useful. You can add information by logging in using your RePEc Author Service credentials here. Thank you!


Plagiarism in Economics

February 16, 2011

We are all aware that plagiarism exists, and RePEc has helped expose quite a few cases through its open bibliographies. But sanctions for plagiarism are rather limited. An offended party may complain with the administration of the accused offender, in some cases without consequences, and in others with sanctions that can lead to dismissal. But the now convicted offender may simply take a new job as if nothing happened, the new employer being oblivious to what happened.

Economics does not have an ethics board that could deal with such cases beyond the current employer of an accused offender. There is now a proposal to create a committee dedicated to plagiarism. This committee would examine cases and vote on sanctions which may go all the way to publicly exposing the plagiarist. A group of volunteers have are discussed a simple set of procedures. Over the next month, the plan is to solicit comments from the public through this blog and call for further volunteers to participate in the committee. After that the committee would become active and deal with any new plagiarism cases that come to its attention. Please contact any current member to participate.

To view the current proposal and committee members, see a simple and bare bones site at plagiarism.repec.org. The committee awaits your reaction. Beyond comments, you can also vote your reaction below.


The FRED Network, a social network for economists

June 12, 2010

Guest post by Richard Anderson

On June 3, 2010, the Research Division of the Federal Reserve Bank of St. Louis introduced the first Internet social networking web site for economics and business. The new web site <www.thefrednetwork.com> is a namesake of the Bank’s popular FRED data service, the most widely used free source of United States economic data on the Internet.

The FRED Network will permit economists and the public to communicate more easily with the data analysts that support FRED, as well as with economists both at St. Louis and elsewhere in the world. The web site is a “dual-threaded” design, meaning each user can select both the topics of interest and the site’s users with whom they wish to communicate. Unlike unfiltered Internet blogs, users will not have to sort through commentary of little interest to locate useful information.

Social networking web sites help people find others with similar interests, exchange knowledge about both data and research projects, solve problems, and develop new ideas. Also, companies learn from their customers by reading customer comments.

“The FRED Network will improve communication between the Bank staff and our customers,” said one staff member. “In the past, we answered most questions via email and only one person saw the information. Now, we can answer within The FRED Network and all customers with that interest will see the information.”

Social networking brings together like-minded people, whether users of FRED data or top-level professionals pursuing complex research.

A unique feature of The FRED Network is the ability for each user to write their own blog. Each user’s blog is available to all other users who sign up to receive that user’s commentary. The unique design of The FRED Network allows each user to read blogs from selected users while excluding ones they do not care to read. This feature allows users who are passionate to write about their interests and expertise, while allowing users who do not wish to receive that commentary need not do so.


Is there an alphabetical bias in citations?

March 11, 2010

In Economics, there is a tradition to list multiple authors in alphabetical order, unless exceptional circumstances call for a different order. This implies that “alphaberically challenged” authors like me often get forgotten, either because they disappear in “et al.” or because they become also-rans. RePEc manages to correct for “et al.” in citations, but it is possible that because later co-authors get less name recognitions, they also get less cited when sole authors. Using data from authors registered in RePEc, here is are some simple statistics that could shed some light, or raise some new questions.

I split the over 23,000 author into 26 bins, one for each letter of the alphabet corresponding to the initial of their last name. First, see how the average number of distinct works authors have written for each of those bins. The graph below runs from A on the left to Z on the right. While this is not a straight line, it does not appear to be obviously trending up or down. The correlation is -0.27.

Distinct works per author, by author's initial A-Z

Now look at the number of citations per author. This time, if you blink a little, you can see a little downwards trend. The correlation is -0.45 and note also that the spread is much larger.

Citations per author, by author's initial A-Z

Now try again with citations per work. A downward trend is now more visible, and the correlation is -0.53, with a different of about one citation between start and end of alphabet. Note that these are just simple averages, without any control for anything else that could correlate with the alphabet and lead to lower citation counts. But it is not obvious what such a control could be. Conclusion? Is there a bias in citations against alphabetically challenged authors? Possibly, but it is not a large one.

Citations per work, by author's initial A-Z


RePEc and the democratization of research

December 9, 2008

In the last issue of the American Economic Review, the following article caught my eye: Restructuring Research: Communication Costs and the Democratization of University Innovation by Ajay Agrawal & Avi Goldfarb. In short, it documents who gained in electrical engineering faculties from the reduced cost of collaboration through the introduction of Bitnet, in the early Internet days. The basic result is that the middle-tier universities benefited the most. Indeed, the top ones were already well connected with each other, and the middle ones took advantage of collaborating with the top ones.

The main goal of RePEc is precisely the democratization of research. Given publication delays in Economics, if one wants to stay abreast of developments at the frontier of research, one needs to read working papers. Before the Internet, the only way to get hold of them was either if you were already at a top ranked Economics department, or if you were somehow within a club of well connected researchers. Just being aware of the most current research was a challenge for anybody outside these circles. This is what motivated Thomas Krichel, as a research assistant in 1991, to find ways to learn about new working papers, and share what he found. This initiative evolved into RePEc in 1997.

Are Elite Universities Losing Their Competitive Edge? by E. Han Kim, Adair Morse & Luigi Zingales documents that Economics faculty in elite universities where more productive at least in part due to their location in the 1970s, and that such a location effect has disappeared by the 1990s. While it is open whether RePEc has contributed to such democratization, we have always favored it: everybody should be able to learn about current research, and everybody should be able to contribute to it.


RePEc in June 2008

July 3, 2008

June was a surpisingly busy month, especially in terms of content expansion. We have now reached 600,000 works listed on RePEc, and it took only 10 months to add the last 100,000. Traffic was also heavy for the season, reaching 584,843 downloads and 2,803,705 abstract views.

The following institutions joined RePEc with an archive: World Scientific Publishing, Queens College (CUNY), GEFRA, Kobe University, Institut für Angewandte Wirtschaftsforschung (IAW), Université d’Auvergne, Universtät Freiburg, Società Italiana degli Economisti. Finally, here are the thresholds we reached this month:

140,000,000 cumulative abstract views
100,000,000 cumulative abstract views on IDEAS
45,000,000 cumulative abstract views for articles
600,000 listed works
350,000 articles listed
300,000 online articles listed
240,000 working papers listed
180,000 working paper abstracts
150,000 items with references
120,000 article abstracts
20,000 NEP reports


Where are the women?

June 19, 2008

Women have always been underrepresented in Economics. For example, regarding US faculty, the Committee on the Status of Women in the Economics Profession (CSWEP), a subcommittee of the American Economic Association, determines in its latest annual report that women represent 28% of assistant professors, 21% of associate professors and 8% of full professors in PhD granting Economics department. As a whole, they represent 19% of all Economics faculty.

The point of this post is not to complain about the low proportion of women in the profession, or about their dwindling share up the ladder, but about the lack of involvement on women in RePEc. Currently, their share is at 14.5%. It is clearly below the 19% mentioned, although it is slowly increasing (it was 13.6% a year ago and 12.7% two years ago). Why this underrepresentation?

It is of course possible that their a bias in those numbers, because the CSWEP numbers pertain only to the United States and the RePEc Author Service covers the whole world. So, let us analyze the top 1000 economists from Tom Coupé’s list. Of the men, only 22.9% are not registered taking the ranking by publications, and 32.4% with the ranking by citations (which includes quite a few non-economists). For women, the numbers are 37.2% and 44.4%. We see that top female economists are less likely to be signed up with RePEc.

Therefore, encourage women to register at the RePEc Author Service!

PS: You may wonder how these numbers are determined, as gender is not indicated when registering with the RePEc Author Service. It is inferred from first names, using a database of gender likelihood by name. For the more uncertain cases, an exception table was created using additional information, in particular from pictures on personal web pages.


Follow

Get every new post delivered to your Inbox.

Join 194 other followers