Medical Trove PubMed Dips Toe Into Scariest Internet Realm: Comments

PubMed Commons

PubMed’s pilot project to allow reader comments.

Since you’re here reading CommonHealth, chances are you’re such a savvy navigator of medical information that you’re familiar with PubMed, that monumental database of 23 million medical research papers at latest count.

(Just in case you’re not, check it out here — it’s a priceless treasure trove containing the nearest thing we’ve got to the publicly accessible sum of all humanity’s knowledge on medicine and the life sciences.)

If you are already a PubMed fan, you might be interested to learn that the great repository has just taken a bold step into the 21st century: It has begun to accept comments from readers, in a pilot project called PubMed Commons.

The move is being widely hailed as a significant step forward, but in this time of Internet trolls and spambots, the decision is also not without controversy. It means a step into one of the more problematic aspects of the Internet — the comments section.

Yikes. You know how nasty and crazy those can get. To the point that some Internet venues just turn them off. Sometimes, after I kill out the rare nasty comments we get here on CommonHealth, I feel as if I’ve just cleaned a toilet.

Back to PubMed. First, the positive side: The scientific papers on PubMed go through a process of peer review in advance — meaning that usually a handful of scientists assess the research and decide whether it’s worth publishing in a given scientific journal.

You could argue it’s only fair: If you trash my paper, I can trash yours.

But there hasn’t been a great process for evaluating research after it’s been published. Science aims to be “self-correcting” — findings need to be checked and replicated in order to become accepted as correct. Until now, though, if you find something in a paper that seems wrong, there has only been a cumbersome process of sending in a letter to a journal’s editor and hoping it will be published.

Now, papers posted on PubMed may be subject to a sort of instant, post-publication peer review.

“The general public will be able to watch scientists debate, argue, critique one another’s papers in real time,” said Ivan Oransky, global editorial director of Medpage Today.  “I happen to have some issues with how open it will be and who can comment, which are my particular issues, but I do think that in general, it is a step forward and it’s a big deal; it’s people being able to see what’s going on.”

Now to the bit of controversy Oransky alludes to: PubMed isn’t allowing everyone to comment. At this initial stage, they’re mainly allowing only people who have already authored papers that are in PubMed or have received certain government grants to comment on other people’s papers. (My rough literary analogy: Only people whose books have been featured in the New York Review of Books may comment on the current edition’s reviews.)

You could argue that these limits on commenters are only fair — if you trash my paper, I can trash yours.

The counter-argument is that they’re not very democratic, and you could lose many highly valuable commenters if you only allow published authors to comment.

Of course, you could also lose a whole lot of mean trolls, too. Popular Science recently made the radical decision to just turn off its comments section. The editors cited research that found that uncivil comments are actually bad for science; they can make readers discount work even when they shouldn’t.

Oransky of Medpage Today is also the co-founder of one of my favorite blogs: Retraction Watch, which covers retractions in scientific journals as a window into the process of science, aiming to making the whole endeavor more transparent. (Confession: I don’t rubberneck at highway accidents but do take guilty pleasure in reading about how and why science goes wrong.)

In a Retraction Watch post, Oransky welcomes Pubmed’s addition of comments, but questions the decision to restrict commenters so tightly.

When we spoke, Oransky said he and Retraction Watch co-founder Adam Marcus believe that “It is still better, despite all the problems we’ve seen with commenting recently, to have a free and open exchange for everyone.”

“There are a lots of people who have never and will never publish on PubMed, who’ll never get grants from any major institution, who have a lot of really valuable things to say about scientific literature,” he said. “We’ve seen a lot of what you might refer to as sort of laypeople have comments on papers that have led to significant — if not changes in the papers, if not corrections and retractions, even just moving the research forward and having ideas for that.”

Oransky also questions the PubMed decision not to allow anonymous commenters. We all know anonymity tends to bring out the worst in people, but he points out that science may be a special case:

“People are not willing to come forward if they have to name themselves because there’s such a real hierarchy in science,” he said, “and there can be such revenge exacted by senior people if you are someone who questions other people.”

All of this is still in play; PubMed will be seeing how these comments work and gathering feedback. For more on the PubMed Commons decision-making process, read Stanford’s Rob Tibshirani’s blog post here.

Oransky’s Retraction Watch post spurred a lively debate of 70 comments and counting, and if you want to send in feedback to PubMed, you go to the PubMed Commons FAQ page here and click on the teeny tiny “Write to the Help Desk” link near the bottom right. (My first feedback: Maybe make feedback easier?)

Readers, thoughts? We welcome them, even if you’ve never published a single scientific paper or won a government grant…

 

Please follow our community rules when engaging in comment discussion on this site.
  • dratman

    Your characterization of “mean trolls” is just right. The ability to hurt feelings — which logically ought to be impossible — is the secret possession of every nasty commenter. We no more expect to read vile insults (instead of reasoned discourse), than to bite into a dead mouse (while munching a donut). You know, a really useful NSA would filter out such comments, each one a miniature of psychological terrorism.

  • Not James Shore

    “At this initial stage, they’re mainly allowing only people who have already authored papers that are in PubMed or have received certain government grants to comment on other people’s papers.”

    With 23 million citations, and the average number of authors per article hovering around 5 or 6, I’d figure there were a lot of authors. I picked a random last name “Shore” and searched PubMed for Shore[au] for 3324 hits. Wanting to limit that down, I added the first initial J (as in Jersey Shore) and still get 389 instances. Full name shows in the MEDLINE view, so let’s see:
    The first 20 results are from
    Shore, Jay x3

    Shore, Jay H x9
    Shore, John W x3
    Shore, James H

    Shore, J H
    Shore, Jennifer L
    Shore, Jane

    That’s at least 6 different authors with the 3243 most popular name in the country (according to http://www.pbs.org/pov/thesweetestsound/popularityindex.php )

    By the way, James is the #80 most popular last name and james[au] gives 23,460 results.