Note that this is not a secret. Every IDEAS abstract page has a statement to this effect under “Corrections.”

The citation analysis is performed by José Manuel Barrueco Cruz, who can certainly use some help.

Now as to what RePEc brings compared to other services. One is that we unify structured data from many publishers (currently 1218), and we have a focus on Economics. This reduces noise. Given that we have structured data, we can offer better search results (and improve Google Scholar results). Also, we offer email services for new working papers (NEP), links between works, authors and institutions. And there are more projects in the works.

The current non-trivial user-generated data are, to my knowledge:

(1) being able to claim or reject ownership of works in the database and thus disambiguate name conflicts.

(2) being able to reject citations that don’t match.

(3) Download counts — which are by nature highly manipulable without risk of reputational consequences. (Unlike specific reforms I suggested).

That’s not much.

If you want a volunteer who will help implement ways to gather and validate user-generated info, I have the skills and I’ll do it. However, that seems to conflict with your stated principles. I’m not interested in volunteering under the current principles because there is little value added by the project. Except rankings, which are now poorly done and unreliable and thus harmful because, as you state, publishers care little about author/rankings and they provide all the data.

You say “if you find something is incomplete, find a way to make it complete.” Well, that’s why I took the time to make specific suggestions in the suggestion box. I’m even willing to do them myself, but they seem to conflict with your principles. It’s easy enough to point the finger at publishers and say they should fix it. But, they didn’t put up a web-site and issue rankings, Repec did.

As a side-note, if I understand your formula description right, it probably does quite a bit of harm for 25 papers from one school to be placed in the wrong NEP while being missed in the right one. No human editor “recognized” those papers as Post-Keynesian; it’s obvious that some automated tool mistakenly spit those out and the human editor didn’t fully read the list. That’s not a frivolous complaint from a fragile ego. Many students now look those rankings up when considering schools, and that has real implications right down the line.

I’ll not reply further, because I understand this is not a chat forum. But, while I appreciate your replies, my frustration with the site has not been reduced at all and seems unlikely to ever be reduced, given your current principles.

I can understand that people have complaints. The way RePEc has always worked is that if someone has a complaint, he should always provide a way to fix the problem. Plenty of improvements in RePEc services came that way. Thus if you find something is incomplete, find a way to make it complete. Do not wait for us to do it, we have already our hands full (and we also do this as volunteers with other priorities).

1) Every abstract page on IDEAS and EconPapers has a contact email for such corrections. Series and journal pages as well. These contacts are the ones that feed the data to RePEc. They hold the keys.

2) The principle in RePEc is that the information about papers is supplied by the respective publishers. This reduces considerably the need for monitoring. Now, publishers unfortunately rarely use JEL codes (40% for working papers, 7% for articles). This makes them of little used for this. I explored automatic JEL code discovery, but the computational power required for this is overwhelming, and there would be errors. Also, to allow author amendments to existing material requires some serious changes in current software, and current volunteers have no time (or desire) for this. But we are always looking for new blood…

Note that one advantage for using NEP data is that it is generated by a human, who is a specialist of her field.

3) This is an idea I have had in mind for a while, but the major problem here is monitoring for abuse. It would be easy for anyone to increase citation counts by “expanding” bibliographies. And yes, people are petty, we already had to deal with various attempts of illicit citations. I am open to suggestions on how this could be done efficiently for all parties.

4) Email us with the RePEc handles, and we will link them. No need to do so if titles are close to identical, and both works are in your profile. They will be linked at the next refresh.

5) Believe me, we are trying to do our best. Contact the people who can change the data, and I hope they will oblige. Pressure content providers to let us get to the reference sections. And I hope you understand that we want to limit opportunities for abuse to give credibility to our data.

It is my intention to remove the “experimental” label once we have more citing than cited works. The score is currently 248223 to 308223.

Examples:

1) One of my better-cited publications is missing, while every other paper in that issue is listed. Clearly just a clerical error — but there is no contact information to fix it, and no incentive for some clerk at some journal to fix one paper from 7 years back anyway. Why not just let _me_ fix it, by providing the necessary information? Sure, I could lie, but the reputational consequences of something like that would be severe. I could also make up data for publications, but I don’t.

2) You provide rankings of departments by field, and you allocate fields by NEP reports. Well, my NEP reports are warped. I’m included in things that have nothing to do with my field. I am not alone in that, and it has a major influence on several people in my department. Why not just let _me_ tell you what I do? I can, for example, give you JEL codes by paper. For those who don’t want to, just stick with NEP.

3) Why not let users just report papers that cite them (by repec handle), since some journals won’t let you automate it? I can’t imagine anyone so petty or risk-loving as to lie on that, but you can combat the possibility by alerting the citing authors (if registered) in some non-intrusive fashion.

4) Sometimes my working papers change titles before publication. Let me tag two papers with different titles as the same paper.

5) Labeling rankings as “experimental” is a cop-out, because you know they are widely used anyway. If you’re going to do them, at least let the people and departments affected correct the data issues, as above. That is almost surely the quickest and soundest way to improve the quality of this site.

best,
philip