Randomised controlled trial of OA

July 31, 2008 at 6:43 pm · Filed under Open Access ·Tagged , ,

A noteworthy pair of articles have been published by BMJ on July 31 (my thanks to Geoff Hynes for this information):

An editorial by Fiona Godlee, Open access to research, BMJ 2008(July 31);337:a1051.

A research article by Philip M Davis and 4 co-authors, Open access publishing, article downloads, and citations: randomised controlled trial, BMJ 2008(July 31);337:a568. Abstract:

Objective To measure the effect of free access to the scientific literature on article downloads and citations.

Design Randomised controlled trial.

Setting 11 journals published by the American Physiological Society.

Participants 1619 research articles and reviews.

Main outcome measures Article readership (measured as downloads of full text, PDFs, and abstracts) and number of unique visitors (internet protocol addresses). Citations to articles were gathered from the Institute for Scientific Information after one year.

Interventions Random assignment on online publication of articles published in 11 scientific journals to open access (treatment) or subscription access (control).

Results Articles assigned to open access were associated with 89% more full text downloads (95% confidence interval 76% to 103%), 42% more PDF downloads (32% to 52%), and 23% more unique visitors (16% to 30%), but 24% fewer abstract downloads (–29% to –19%) than subscription access articles in the first six months after publication. Open access articles were no more likely to be cited than subscription access articles in the first year after publication. Fifty nine per cent of open access articles (146 of 247) were cited nine to 12 months after publication compared with 63% (859 of 1372) of subscription access articles. Logistic and negative binomial regression analysis of article citation counts confirmed no citation advantage for open access articles.

Conclusions Open access publishing may reach more readers than subscription access publishing. No evidence was found of a citation advantage for open access articles in the first year after publication. The citation advantage from open access reported widely in the literature may be an artefact of other causes.

5 Comments »

  1. tillje said

    July 31, 2008 @ 10:45 pm

    Stevan Harnad has responded in an email message distributed on July 31. Excerpt:

    To show that the OA advantage is an artefact of self-selection bias (or any other factor), you first have to produce the OA advantage and then show that it is eliminated by eliminating self-selection bias (or any other artefact).

    This is not what Davis et al did. They simply showed that they could detect no OA advantage one year after publication in their sample. This is not surprising, since most other studies don’t detect an OA advantage one year after publication either. It is too early.

    Reply

  2. Heather Morrison said

    August 2, 2008 @ 3:06 am

    This study indicates an access advantage to open access (more downloads); it may be too early to draw any conclusions about the citation impact advantages. As Harnad has pointed out elsewhere, citation impact advantage studies have found that early differences in downloads such as this correlate with differential citation rates in about a year or two. This makes sense – if more people are reading this year, they may be in the early stages of a research project that will include that citation in a finished publication in about a year or so.

    Reply

  3. tillje said

    August 4, 2008 @ 3:38 pm

    The Rapid Responses to the article by Philip M Davis and co-authors include letters from Stevan Harnad, Gunther Eysenbach and Trish Groves (Deputy editor, BMJ). An excerpt from the latter:

    Stevan Harnad and Gunther Eysenbach both point out, helpfully, that Davis et al have reported here an interim analysis of their RCT of open access. We should have made this clearer in the paper, and no doubt the study’s authors will elaborate on this in a further response.

    An excerpt from Gunther Eysenbach’s letter, entitled “Word is still out: Publication was premature“:

    In summary, this would have been an important and much more credible paper if it would have been published in 2-3 years as opposed to a salami approach.

    Also worrisome is the fact that this premature publication makes it possible to sabotage the RCT, making interpretation of future results from this RCT more difficult. Imagine Davis’ et al would find – in their follow-up report – a significant difference between citation counts of OA articles and non-OA articles (or again no difference). Who can then rule out that these additional citations one way or the other aren’t intentionally produced by Open Access advocates (or critics), who now deliberately start citing preferentially those green-lock-marked articles from Davis’ dataset (or vice versa)?

    Added August 7, 2008:

    Philip M. Davis has posted an Authors’ Response to criticisms of the interim analysis. Excerpt:

    To summarize, we believe that our research provides strong evidence that open access increases the dissemination of scientific articles, as indicated by our download results. However, we find no evidence of an open access citation effect, even after incorporating six additional months of citation data. There are many societal benefits to making the scientific literature freely available beyond the research community; a citation advantage may not be one of them.

    Comment:

    This randomized trial has already provided additional strong evidence that open access increases the dissemination of scientific articles. The enhanced transfer of knowledge to those whose access currently is restricted because of price barriers is an important example of the advantages of OA.

    The absence of evidence for a citation advantage after a second interim analysis, done after incorporating six additional months of citation data, still does not provide strong evidence against an OA citation advantage. If, after 2 to 3 years, the trial continues to yield no significant evidence for a citation advantage, then alternative explanations for the OA citation advantage found by others would need to be considered more seriously.

    If the self selection postulate does gain support, what’s the problem? If authors selectively choose to make their best articles OA, or if highly cited authors prefer OA options, this means that higher quality articles are being self selected for wider dissemination. Why not regard this as yet another example of the advantages of OA?

    Added August 8, 2008: The above comment has been posted as a Rapid Response to the article in BMJ. See: Re: Authors’ Response, August 8, 2008.

    Reply

  4. tillje said

    August 7, 2008 @ 10:36 pm

    A blurb about this article was published on July 31. See: Free articles get read but don’t generate more citations, Cornell study finds, Susan Lang, Cornell Chronicle Online. It’s being noticed. See, for example, Open Access Doesn’t Mean More Citations, Jan Novak, CM Law Library Blog, August 06, 2008.

    Added August 8, 2008

    See also: Open Access Doesn’t Lead To More Citations, Says Study, Scientific Blogging News, July 31, 2008.

    Reply

  5. tillje said

    August 22, 2008 @ 5:19 pm

    See also: Author-choice Open Access Publishing, Philip Davis, Scholarly Kitchen blog, August 19, 2008. Excerpts:

    According to a study of 11 biological and medical journals that allow authors the choice of making their articles freely available from the publisher’s website, few show any evidence of a citation advantage. For those that do, the effect appears to be diminishing over time.

    …

    As the author of this study, I hope that readers do not come away with the feeling that I’m advocating against an author-choice program. There may be many benefits to making scientific results freely available; however, scientists should understand that open access may not buy them more citations.

    Reply

RSS feed for comments on this post · TrackBack URI

Leave a Comment