Assessing health services research journals

Public policies about health services vary greatly both between and within countries. “Health services research” (HSR) is a field of research that deals with various aspects of health systems. It’s a field that’s of interest not only to those who do the research, but also to those who make decisions about health policy, and to those who are affected by such decisions. For examples of articles about HSR, see the collection published by CMAJ.

How to assess existing HSR journals? Which HSR journals publish articles that are openly accessible, and hence, are readily available to all those with an interest in health policy and services?

First, a Canadian perspective. Canada’s unique policies for the delivery of health services merit research that has a focus on this particular health system. An example of a Canadian health services research journal is Healthcare Policy. This journal published its first issue in September of 2005. See: Healthcare Policy journal, CIHR Institute of Health Services and Policy Research (IHSPR). Excerpt:

Healthcare Policy is designed to be a key knowledge translation vehicle to stimulate communication and cross-fertilization between researchers and health care decision makers.

An excerpt from the home page of the journal. It is:

For health system managers, practitioners, politicians and their administrators, and educators and academics. Authors come from a broad range of disciplines including social sciences, humanities, ethics, law, management sciences, and knowledge translation.

The publisher of this journal is Longwoods Publishing. The SHERPA/RoMEO summary of publisher copyright policies & self-archiving for Longwoods Publishing indicates that a post-print (i.e. final draft post-refereeing) may be archived 12 months after date of online publication. Information about this 12-month embargo is included in the License to Publish (PDF, 190 KB).

This 12-month embargo can be avoided if authors select the Optional Open Access paid-OA option that’s available for Longwoods’ four journals. Fee for 2008: $2,500.

For an appraisal of the Optional Open Access policy, see: Longwoods Press rolls out Open Access policy, by Greyson, Social Justice Librarian, January 10, 2008. An excerpt from one of Greyson’s comments that follow the appraisal highlights the importance of independent Canadian journals:

I think it’s deeply important for us to continue to have independent Canadian journals, particularly in health policy where our concerns are often quite different from those in the US.

From a broader perspective, results obtained from the application to health-related issues of research methods derived from a range of disciplines, including the social sciences, are reported in a variety of journals. There are sites that permit a variety of journals in a particular topic area to be ranked according to specified measures of impact.

For example, the Longwoods journal Healthcare Quarterly is currently ranked #1201 in a SCImago ranking of journals in the Subject Area: “Medicine” and the Subject Category: “Medicine (miscellaneous)”. However, Healthcare Quarterly is a journal that has a focus on:

Best practices, policy and innovations in the administration of healthcare. For administrators, academics, insurers, suppliers and policy pundits.

So, it’s a journal intended for research on healthcare policy and services. It seems inappropriate to classify it as a “Medicine (miscellaneous)” journal.

The Longwoods journal Healthcare Policy wasn’t found in the SCImago database via a Journal Search, so it isn’t included in any rankings.

Another website that can be useful for the evaluation of journals is There’s a claim on its “why eigenfactor?” page that it “ranks journals much as Google ranks websites. One option that’s provided is a “cost-effectiveness search“. When this search was done for the “field: medicine” and “fine category: health policy and services”, the cost-effectiveness search results yielded a total of 28 journals. The five top-ranked journals were: Health Affairs, Medical Care, American Journal of Managed Care, Milbank Quarterly and Health Services Research.

Neither of the Longwoods journals Healthcare Policy nor Healthcare Quarterly are included in the database, so they don’t appear among the 28 journals ranked in the results this particular search.

Similarly, SCImago can be used to obtain rankings of journals in the “Subject Area: Social Sciences” and the “Subject Category: Health (social science)”. They can be ranked by H index, or ranked by SJR (SCImago Journal Rank). Healthcare Quarterly wasn’t found among the 81 journals ranked in the results of this search. Apparently, this is because Healthcare Quarterly has been put into the Subject Area category: “Medicine” and the Subject Category: “Medicine (miscellaneous)”, as noted above. If it had been included, it would have been ranked about 68th of 81 Health (social science) journals, when ranked by SJR.

As is also noted above, Healthcare Policy is currently not included in the SCImago database.

Four of the five journals identified above (as highly ranked in terms of cost-effectiveness) were also found among the top ten journals when ranked according to either H Index or SJR. These four journals and their publishers are listed below. Also included is information about their compliance with the access policies of a range of funding agencies (according to the SHERPA/RoMEO database).

1) Health Affairs (Project Hope): A search of the SHERPA/RoMEO database yielded no information about compliance of this journal with the access policies of funders, nor was I able to find any relevant information via the journal’s website.

2) Medical Care (Lippincott Williams & Wilkins): Results from the SHERPA/RoMEO database indicate that this journal is non-compliant with the access policies of the funders that are identified.

3) Milbank Quarterly (Blackwell): Results from the SHERPA/RoMEO database indicate that this journal is compliant with the access policies of all of the funders that are identified. However, the exact duration of any embargo period, post-publication and prior to provision of free accessibility, isn’t specified for this particular journal. A visit to the journal’s website also didn’t yield this information.

4) Health Services Research (Blackwell): Results from the SHERPA/RoMEO database indicate that this journal also is compliant with the access policies of all of the funders that are identified. However, according to the Copyright Assignment Form, the policy of HSR about postprints is:

Posting of the published Article on an electronic public server can only be done with written permission from Blackwell Publishing.

In contrast, the policy for Standard Blackwell Publishing Journals is that postprints may be posted:

… on free public servers in their subject area after the embargo period stipulated by the journal has been passed (note that the embargo period will vary by journal, e.g. 6 months or 12 months, and that some journals do not have any embargo on self-archiving at all …

Thus, the policy about postprints (author’s final version after peer review) for HSR appears to differ from that of other Blackwell journals.

Some, but not all, Blackwell journals offer a hybrid OA option, called Online Open. According to a list dated May 11, 2007 [xls file], Health Services Research is one of the journals that offers this option. The fee for 2007 is US$2,600. (According to the list dated May 2007, Milbank Quarterly does not offer this hybrid OA option).

There are, of course, other journals devoted to various aspects of research relevant to health policy and health services. Two examples of Open Access journals are BMC Health Services Research (published by BioMed Central; the article-processing fee for 2008 is US$1,870), and Health Research Policy and Systems (published by BMC in collaboration with the World Health Organization; the current fee is US$1,675).

A comparison of SJR data from SCIamgo for these two journals and for the journal Health Services Research indicates that the SJRs for all of these journals have increased after 2003, and that the SJRs for the two BMC open access journals in 2006 were similar to the SJR for Health Services Research 5-6 years earlier.

So, what lessons might be learned from these comparisons? Some points to consider:

a) The policies of publishers about Green OA (archiving of post-prints) for these journals range from non-compliant (Lippincott Williams & Wilkins) to complex (Blackwoods) and/or obscure (Project Hope, Blackwoods). The effort required to try to obtain unambiguous information about these Green OA policies is sometimes ridiculous, to an extent that is almost comical.

b) The preferred alternative to Green OA is Gold OA (paid open access to the full text of journal contents). The most prominent OA publisher in the HSR field is BioMed Central (BMC).

c) Some publishers (e.g. Blackwell, Longwoods) do provide Hybrid OA (paid access to individual articles), but the article-processing fees are expensive (e.g. in comparison with BMC’s Gold OA journals). Such programs may also have other drawbacks. See Peter Suber’sNine key questions to ask about any hybrid journal program“, SPARC Open Access Newsletter, issue #101, September 2, 2006.

d) The warning at the SHERPA/RoMEO site should be noted: “All information is correct to the best of our knowledge but should not be relied upon for legal advice“. When a publisher’s policies vary from journal to journal, the information may be correct for one journal, but not for another.

e) Sites designed to permit comparisons of journals, such as SCImago and, may neglect or misclassify some journals (especially, “niche journals“). It would be prudent not to use any single site of this type as the sole source of information for an assessment of journals. Journal comparisons need to take into account multiple variables, not all of which are likely to be incorporated into any particular source of information.

Added July 2, 2008: Cite as: Till J. Assessing health services research journals. Be Openly Accessible or Be Obscure blog. Self-Archived at WebCite® 2008-Jul-2 []



  1. tillje said

    Received from Anton Hart, Longwoods Publishing, and posted with his permission:

    The web and the OA paradigm provide challenges for all publishers. No matter what their affiliation or business model they need to pay the bills. The analysis and commentary provided by pundits can and does provide us with new information and insight and we pay attention. We take it into account and it helps us evolve as we aim for respectability and excellence of both editorial and distribution strategies for all the information we make available.

  2. tillje said

    Four “Green” journals that permit self-archiving of the postprint (final draft post-refereeing), and publish articles related to health policy or health services research, are listed below. Their 2005 Impact Factors and their 2006 SJR values are shown, together with relevant links to the SCImago and SHERPA/RoMEO databases:

    Health Economics (Wiley), IF: 1.919; SJR (2006): 0.210; SCImago; SHERPA/RoMEO.

    Health Policy (Elsevier), IF: 0.964; SJR (2006): 0.132; SCImago; SHERPA/RoMEO.

    Journal of Health Politics Policy and Law (Duke U), IF: 0.718; [Not in SCImago]; SHERPA/RoMEO.

    Social Science and Medicine (Elsevier), IF: 2.619; SJR (2006): 0.231; SCImago; SHERPA/RoMEO.

    These four journals are identified in a recent poster entitled Open Access to Health Services and Policy Research, prepared by Devon Greyson, Steven Morgan, Gillian Hanley and Desy Wahyuni, Centre for Health Services and Policy Research, University of British Columbia.

  3. tillje said

    The methodology described in the post More baseline data from PubMed (April 7) was used to search for articles that were identified in PubMed as having been based on research that received NIH support. The key words used in these PubMed searches were: Research Support, N.I.H., Extramural [pt] OR Research Support, N.I.H., Intramural [pt]. The searches were done on April 12, 2008, and were limited to searches for articles published, within the last two years, in each of the four journals listed below. Data for the percentages of articles for which links to the free full text (FFT) were provided by PubMed were:

    Health Affairs: 4/55=7% FFT
    Medical Care: 8/139=6% FFT
    Milbank Quarterly: 0/1=0% FFT
    Health Services Research: 0/55=0% FFT

    Thus, for the journal Health Affairs, 55 NIH-supported articles were identified as having been published within the past two years. Only 4 of these (7%) were identified as freely accessible.

    Will the new NIH policy about open access, being implemented as of April 7, 2008, lead to an increase in these percentages? It should.

  4. tillje said

    Journals That Submit Articles To PubMed Central (PMC):

    This is a list of the journals that submit articles directly to PubMed Central on behalf of their authors. If you publish in one of these journals, no further action is needed to comply with the submission requirement of NIH Public Access Policy.

    Of the health services/health policy journals assessed, the following appear on the list of journals that submit articles to PMC (list updated April 24, 2008):

    BMC Health Services Research;
    Health Research Policy and Systems.

  5. tillje said

    I’ve used the methodology that’s described in a post about assessing immunology journals to obtain data based on searches for the MeSH headings Health Services Research (HSR), OR Health Policy (HP). The results (FFT means that the free full text is accessible via PubMed):

    NIH-supported articles about HSR or HP:
    published in the last 2 years: 191/904=21% FFT;
    published in the last 1 years: 57/373=15% FFT.

    Total articles about HSR or HP:
    published in the last 2 years:2310/18220=13% FFT;
    published in the last 1 years: 784/7924=10% FFT.

    Articles that are OA (Bethesda definition)/articles for which the free full text (FFT) is available via PubMed Central:
    published in the last 2 years: OA/FFT=798/1578=51%;
    published in the last 1 years: OA/FFT=258/491=53%.

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: