Skip to main content
  • Expert Commentary
  • April 21, 2008

Building a Better Guideline: Overcoming Challenges and Establishing Transparency

Last year, the American College of Chest Physicians (ACCP) published its guideline development processes, including how it addresses conflicts-of-interest, funding, and other policies and procedures, with the goal of contributing to the national and international dialogue on evidence-based medicine and guideline development (1). The Health and Science Policy (HSP) Committee of the ACCP is charged with oversight of ACCP guidelines and the process for guideline development, which is forever a work-in-progress (1,2). Recent improvements have led to significant changes in new guidelines and subsequent editions of previously published ACCP guidelines.

Experience has taught us that the development of accurate and useful evidence-based clinical guidance requires a rigorous process. Recognizing that guideline development must always improve and adjust with the times, the ACCP has developed and refined its process to be responsive to feedback and changes in the healthcare environment. As a result, the ACCP has enhanced its methodology while publishing nine evidence-based guidelines in the past 7 years.

Earlier this year, the National Guideline Clearinghouse (NGC) released their summaries of the Diagnosis and Management of Lung Cancer: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (2nd Edition) (3). As the development process evolved over the years between the two editions of these lung cancer guidelines, the ACCP faced several challenges that needed to be addressed in a methodological and inclusive manner. Since publication of the first edition of these guidelines in 2003, there have been several overall improvements in the planning and development process leading to specific enhancements for the final products.

Changes in Planning and Development

First, the scope for the second edition of these lung cancer guidelines was increased. The 2003 edition was the first set of ACCP guidelines to be both evidence-based and comprehensive, incorporating prevention, screening, physiologic assessment, diagnosis, staging and treatment, special topics in lung cancer, follow-up and surveillance, palliative treatment, and end-of-life care. Based on readers' feedback, new chapters on pathology, bronchoalveolar carcinoma, and integrative oncology have now been included in the second edition, and many subtopics have been significantly expanded. New algorithms have been included for management of patients with varying sizes of solitary pulmonary nodules and for preoperative physiologic assessment of perioperative risk.

As these guidelines are so comprehensive and voluminous, resources must be prioritized and allocated accordingly. In the first edition, the Agency for Healthcare Research and Quality (AHRQ)-sanctioned Evidence-based Practice Center at Duke University was contracted to perform a formal systematic review in the areas of screening, diagnosis, and staging for lung cancer. In the second edition, emphasis was placed on the treatments for lung cancer by stage.

Feedback on the ACCP grading system led to a specially convened taskforce in 2005. The inclusion of relevant stakeholders in the deliberations and multiple iterations of this project culminated successfully in the new ACCP grading system (4). That same year, a similar procedure was followed when ACCP assembled a taskforce of experts in economic analyses in health care to compose a paper on incorporating consideration of resource allocation issues in select guideline recommendations (5). Resource allocation was and will continue to be a difficult but important area with which guidelines developers struggle, especially those who write guidelines that address international target audiences crossing multiple specialties.

The subject of conflicts of interest continues to be a challenge but the ACCP and the HSP Committee policies specific to guidelines and guideline panels have been revised several times in the past few years. An HSP subcommittee has been charged with examination of conflicts and determinations for action. If an individual is determined to have a conflict, real or perceived, actions to manage the conflict vary from removal of the individual from the topic of conflict and reassignment to a different topic, to disqualification from the panel altogether. Firewalls were created and are managed by strict adherence to a set of regulations that are periodically reviewed and revised as necessary (1). Based on the input of several committees, the ACCP devised a College-wide policy on identification and management of conflicts of interest and firewalls. We continue to participate in national discussions on the subject.

Comprehensive review of the manuscripts is integral to ACCP's accountability to the guideline users and, ultimately, their patients. HSP has constructed two algorithms defining the review process for smaller and larger ACCP guidelines. Common to both is the full-scale review by both content and methodological experts, including all members of the HSP Committee and the Executive Committee of the ACCP Board of Regents. Upon acceptance by the Editor in Chief of Chest, the guidelines are sent for further peer review through the established channels of the publisher. Some ACCP guidelines are reviewed by 20 to 30 individuals in total prior to the final publication. Each reviewer is held to the same standards as the guideline panelists relative to conflicts of interest. In addition to the content and methodological appraisal, the reviewers are requested to ascertain whether the discussions and recommendations are balanced and unbiased.

Changes at the Guideline Level

Following improvements in the overall planning and development process, several enhancements were made at the guideline level to the methodology, content, and format. The methodology has been improved by the new ACCP grading system (4), final conference protocol, and rigor of the evidence base in the foundation of the recommendations. First, as mentioned above, the new ACCP grading system was developed after careful efforts by a specially convened taskforce in 2005 and approval of the HSP Committee. The new grading system is easier to understand and use, both for the guideline panelists and for the readers. Second, the guideline panelists saw a change in the design of the final conference. For this update of the lung cancer guidelines, in addition to full panel review of the chapters, panelists were charged with identifying all recommendations that were deemed to be controversial or that addressed clinical topics with considerable variation in practice, so they could receive full review, discussion, and voting. Finally, although the increased volume of scientific data allowed for higher standards of evidence-based recommendations, the strict adherence to the evidence left several controversies unanswered but highlighted the need for additional research.

The publication format has also been improved. Most guideline users are not going to read these manuscripts from cover-to-cover, but they will search for the recommendations that are pertinent to the patient they are treating at the moment. Thus, all ACCP recommendations begin with the appropriate patient population(s). In addition, all recommendations now appear twice in each chapter, once following the review and discussion of pertinent literature that leads to the recommendations and again summarized in a new section following the abstract. Readers will also find that the executive summary, which now includes all clinical recommendations, facilitates locating recommendations appropriate for particular patient populations.

With our efforts to overcome these challenges, the ACCP realized that all guideline developers face similar challenges. To promote further debate, learn from others, and share what was learned, the HSP leadership made a decision to publish our policies and procedures (1). We believe that it is important to be transparent about our methods. It is hoped that other guideline developers will share their challenges and how they overcame them and that this will further the national and international discussions that AHRQ and the Guidelines International Network have attempted to promote and facilitate through this column and other venues.


Sandra Zelman Lewis, PhD
Assistant Vice President
Health & Science Policy and Quality Improvement
American College of Chest Physicians


The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Guideline Clearinghouse (NGC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor, ECRI Institute.

Potential Conflicts of Interest

Dr. Lewis declared no potential financial conflicts of interest with respect to this expert commentary.


  1. Baumann MH, Lewis SZ, Gutterman D. ACCP evidence-based guideline development: a successful and transparent approach addressing conflict of interest, funding, and patient-centered recommendations. Chest 2007; 132:1015-24 External Web Site Policy
  2. External Web Site Policy
  3. Diagnosis and management of lung cancer: American College of Chest Physicians evidence-based clinical practice guidelines (2nd Edition). Chest 2007; 132(suppl): 1S-422S External Web Site Policy
  4. Guyatt G, Gutterman D, Baumann MH, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an American College of Chest Physicians Task Force. Chest 2006; 129:174-81 External Web Site Policy
  5. Guyatt G, Baumann M, Pauker S, et al. Addressing resource allocation issues in recommendations from clinical practice guideline panels: suggestions from an American College of Chest Physicians Task Force. Chest 2006; 129:182-7 External Web Site Policy

About NGC Expert Commentaries

Commentaries are editorial insights on current issues of importance to the guildeline and/or measure fields.

Guideline Summaries

New This Week

View more and sign up for our Newsletter

Get Adobe Reader

Make a Comment

All submitted comments will be reviewed by NGC Staff and NGC's Editorial Board who will elect what to publish on this web site.

* indicates required field

Submit a Comment

Do you have any Disclosures and/or Conflicts of Interest to report? *

Optional Information

About Commenting

The National Guideline Clearinghouse ™ (NGC), sponsored by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services, welcomes responses to our posted expert commentaries on the NGC Web site.

All responses will be reviewed by NGC Staff and NGC's Editorial Board. Please keep your response brief and to the point. Responses must be presented in clear statements. We reserve the right to 1) publish your response, 2) copyedit your response, and 3) invite the author of the Expert Commentary to reply to a posted response. Excessively long or offensive and nonobjective responses will be disregarded.

Please complete the form if you wish to have your response made accessible to our users or if you wish for NGC to act on your response. NOTE: Providing such information however, does not ensure that your comment will be published or made available or that NGC will take any course of action. That will be determined by NGC staff in conjunction with our Editorial Board after review of the information you supply. Criteria for publication of responses to commentaries include:

  • Relevance of response to the subject of the commentary
  • Responses that provide a unique perspective to the commentary
  • Responses that offer an alternative view than the commentary
  • Responses that are clearly articulated
  • Responses that are neither excessively long or offensive and/or nonobjective
Form Approved    OMB No. 0935-0106    Exp. Date 11/30/2017

Public reporting burden for this collection of information is estimated to average 90 seconds per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0106) AHRQ, 5600 Fishers Lane, Rockville, MD 20857.

Note: Responses accepted for publication will be posted to the NGC Web site along with the submitter's name. Current position or occupation, organizational affiliation, and disclosed potential conflicts of interest are optional and will be posted when provided. Email and other address are for internal use only and will not be published with responses or made publicly available.