The archives listed in RePEc differ in their policies regarding withdrawal of items, or replacement of an old item by a newer one. Some archives, like NBER, permit withdrawals and replacements, while others, like IZA or MPRA do permit neither withdrawals nor replacements. (ArXiv, the leading archive for physics, has adopted a no withdrawal policy as well.)
I am managing MPRA, which publishes unrefereed discussion papers in economics. In the following, I detail the reasoning underlying MPRA’s policy choice. As the case for prohibiting withdrawals seems to be strong, it is hoped that other RePEc archives adopt a similar policy if they have not done so already.
Discussion papers are preliminary versions of articles that may appear in their final form in the future. Discussion of these preliminary versions serves to improve them.
Discussion of a discussion paper requires that it can be cited. Citation requires that you can find the cited item, and even the cited phrase at the page given in the citation. In short: The cited item must remain reliably unchanged and retrievable.
In the old days, you mailed typed manuscripts to colleagues, and successively revised your papers in response to their suggestions and criticism. This entailed the problem that your colleagues would refer to different versions. In order to correctly grasp their points, you had to keep track of the different versions you had mailed around. (I never managed.) With a stable Internet address for each version, this tracking can be done over the Internet with ease. Permitting substitution of old versions by new version under the same Internet address would invide confusion and would make citations unreliable.
So the alternative seems to be: Either you keep your papers private and have your discussion in form of private correspondence, or you put them on the Net for public discussion. The second alternative is implied by placing the paper in a discussion paper archive, and this seems to require that identifiable versions remain accessible concurrently.
In addition, there are further reasons for favoring a “no withdrawal” policy by archive maintainers.
— If the final version of a paper ends up in a toll-gated journal, this excludes the majority of economists from reading the final version. The presence of a preliminary version mitigates the problem.
— If the preliminary version is referred to by a hyperlink, the reference becomes largely useless. NEP reports will, for instance, show dead links in such cases. This is a nuisance.
— If problems about priority of findings arise, these may be settled more easily if all versions are available on the Net.
— For archive maintainers, the manual handling of withdrawals requires considerable work. This speaks against the possibility of withdrawals as well. (For large archives, this reason is overwhelming. At MPRA we initially permitted withdrawals, but this proved impracticable and provided the proximate cause for adopting the no-withdrawal policy.)
— Further, the fight against plagiarism is eased by adopting a non-withdrawal policy. Typically, plagiarizers ask for removal of their contribution if detection is imminent. This tends to shade the case. If a plagiary remains in the archive, the case remains transparent. If an item is identified as a plagiary, it is to be marked as such, and the original source indicated. This has additional advantages:
— the interested reader is referred to the original source
— the plagiarizer cannot make his plagiary undone, thereby hiding the offense from scrutiny by potential future employers
— because of that threat, plagiarism becomes more risky and is discouraged.
— problems with plagiarism may be settled more easily and be handled more transparently if all versions are available on the Net. Otherwise, a paper may be plagiarized, the original paper substituted by a revised version, and priority will go to the plagiary, while the revised version will be counted as a result of plagiarism! This ought to be avoided.
The common objection against a no withdrawal policy is that authors would prefer readers to read the newest version. Yet RePEc provides information about all versions, and the metadata at IDEAS or EconPapers provide alerts about other existing versions. So the readers may choose the most recent one. (Such problems occur all the time, but it would be impractical to introduce the possibility of withdrawing everything, including published papers. For example, I have recently updated a paper published in a journal in 2008 and would like to refer the reader to the new version in the format of a discussion paper which contains important improvements and new material, but there is no way to do that, other than hoping that the reader searches through RePEc or sees the different versions in Google.)
There is, thus, a conflict between the interest of the author to have only his or her favorite version on the Net, and the public that is interested in transparency and unmanipulated documentation. At MPRA, we try to take account for that by indicating if a paper is superseded by a newer version. Further, we offer the possibility to watermark papers as withdrawn by the author, but leave them in the archive.
The RePEc Blog 
At the NBER we have had 16,000 working papers in 32 years, and I only recall one withdrawn, so that has not been a clerical burden. In any case it is just a matter of changing the bibliographic entry to missing and removing the world read permission on the .pdf. If it happened every day, we would make a cgi-script and let the author do it himself.
If the author believes the paper is incorrect – should we compel him to flaunt his error? Would that discourage submissions? Doesn’t that rule have very bad incentives, in that an author denied the ability to withdraw a paper may be tempted to defend it?
We do save the original paper and could produce it should some controversy require so. We also keep all revisions, and while we only offer the latest version through links on the website, the early versions are still available through a URL we provide to the author, if he wishes to disseminate that or if some priority dispute requires it. But that has not happened yet that I am aware of. A revision date is added to the title page so that readers are informed that the working paper number does not reflect the final revision..
When I was young I worked in a laboratory whose discussion papers were watermarked “Not for citation or attribution”. I wouldn’t want our authors to revert to that.
Ekkehart summarizes very well the reasons why a working paper should not be withdrawn. Yet we routinely see this happening, and often it is not because of an error or a revision, but because of a mistaken belief that once a paper is published in a journal, its discussion paper version should disappear. Many authors also remove working papers from their profiles under these circumstances. This is bad because it cuts the links between the versions. Hence someone who cannot read a gated article will not know about a readable working paper version, and somebody looking at a working paper will never know it was published.
It should be noted that most publishers now allow pre-prints (discussion papers) to remain online, and many even allow post-prints. For a list of publisher policies in this regard, see SHERPA/RoMEO.
We also occasionally see archive maintainers “cleaning up” old working papers. Authors then complain to us that their rankings drop, in particular for downloads. Indeed, working papers are downloaded much more frequently than articles. But beyond the statistics, this again a wrong policy, as Ekkehart has shown above.