Skip to main content

Guideline Summaries FAQs

NGC's Revised Inclusion Criteria

What are the main changes to the inclusion criteria?

The two main changes are:

  • Documentation will need to be provided showing that the guideline is based upon a systematic review of the evidence.
  • Documentation must contain an assessment of the benefits and harms of the recommended care and alternative care options.

How do these changes affect the submission process? What documents* are required?

NGC requires more detailed information to verify that the guideline is based on a systematic review of the evidence and that there was an assessment of the benefits and harms of the recommended care and alternative care options. See Revised Inclusion Criteria 3 and 4 for a description. Documents required to meet inclusion include:

  • Original guideline document(s)
  • Document(s) detailing the specific methodology for the evidence underpinning the guideline(s) (e.g., systematic reviews, evidence syntheses, technology assessment reports)
  • Either of the above must also include an assessment of the benefits and harms of the recommended care and alternative care options.

Other supporting documents that describe the developer's process for guideline development may also be submitted.

*Documents may be submitted as Word, PDF, HTML, or by providing the URL(s) where access can be obtained.

What documentation is required to ensure the guideline was developed, reviewed, or revised within the past 5 years?

All new/updated guidelines submitted to NGC must be based on a recent systematic review. Currency will be established by review of the following: a description of the search strategy that includes a listing of database(s) searched, a summary of search terms used, and the specific time period covered by the literature search including the beginning date (month/year) and end date (month/year), and the date(s) when the literature search was performed. In addition, the guideline's bibliography will be assessed for consistency with the date ranges of the literature searches.

What if the documentation is not published or in a report?

NGC staff will follow up with guideline developers on an individual basis for any missing documentation. In situations where the documentation may not sufficiently meet the inclusion requirements, the submission will be reviewed by NGC senior staff, members of the Editorial Board, and by AHRQ for a final determination.

Will NGC have guidelines that meet two different sets of criteria (i.e., 1997 criteria versus 2013 [revised] criteria)? How long will guidelines accepted under the old criteria remain in NGC?

Guidelines submitted and accepted for inclusion under the 1997 criteria will remain on the NGC site until they are revised by the developer, withdrawn by the developer, or withdrawn by NGC because they are no longer considered current (i.e., when they are ≥5 years old). As a result, there could be guidelines on NGC that were accepted under the 1997 criteria through 2019. However, it should be noted that guidelines included under the 1997 criteria could also meet the standards of the 2013 (revised) criteria.

How will I know which guidelines in NGC meet the 2013 (revised) criteria?

You can use the "Meets 2013 Inclusion Criteria" filter on the search results page to see which guidelines meet the revised criteria. Guidelines that meet the 2013 (revised) inclusion criteria also have a note in the "Guideline Status" field.

How do the 2013 (revised) criteria help me evaluate the guidelines in NGC?

NGC's 2013 (revised) inclusion criteria reflect the 2011 Institute of Medicine (IOM) definition of a clinical practice guideline. The definition emphasizes two important aspects of a guideline that should be represented in good evidence-based guidelines: being based on a systematic review and assessing the benefits and harms of recommended care and alternative care options. These revised inclusion criteria ensure that guidelines in NGC will meet this new minimum standard.

Why were NGC's inclusion criteria revised?

The Inclusion Criteria were revised to keep them current with advances in the field. In March 2011, the Institute of Medicine (IOM) published Clinical Practice Guidelines We Can Trust External Web Site Policy* which redefined clinical practice guidelines (CPGs) as "statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options." The 2013 (revised) inclusion criteria bring NGC into alignment with the new definition of guidelines.

The revised inclusion criteria represent several years of effort that included expert knowledge, the input of guideline developers and implementers, and assessment of over 100 guidelines represented in NGC.

For additional tools and resources regarding the new criteria, see the expert commentary Introducing the National Guideline Clearinghouse Revised Inclusion Criteria and Webinar: The National Guideline Clearinghouse's Revised Criteria for Clinical Practice Guideline Inclusion External Web Site Policy.

*Institute of Medicine. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editor(s). Clinical practice guidelines we can trust. Washington (DC): National Academies Press; 2011. 2p.

Why aren't all of the clinical practice guidelines produced by the Agency for Healthcare Research and Quality (AHRQ) in NGC?

Guidelines produced by AHRQ are no longer included in NGC because these documents are more than five years old. AHRQ also considers these guidelines, either in whole or in part, out of date. For more information, see the AHRQ Web site External Web Site Policy. Users can also see the Guideline Summaries Archive.

Does NGC require the use of GRADE?

No, the use of GRADE is not required for inclusion in NGC. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) External Web Site Policy approach is a widely used mechanism for grading the strength of clinical practice guideline recommendations and the quality (or certainty) of the evidence underpinning them. Providing such ratings is cited as a standard for trustworthy guidelines by the Institute of Medicine (now the National Academy of Medicine)*. NGC does not, however, require that guidelines represented in NGC have been developed using GRADE, a modification thereof, nor any similar system.

*Institute of Medicine. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editor(s). Clinical practice guidelines we can trust. Washington (DC): National Academies Press; 2011. 2p.

NEATS Assessments

Why is NGC assessing guidelines?

The 2011 Institute of Medicine (IOM) report on trustworthy guidelines recommended that AHRQ have NGC indicate the extent to which clinical practice guidelines (CPGs) adhere to the standards for trustworthiness*.

*Institute of Medicine. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editor(s). Clinical practice guidelines we can trust. Washington (DC): National Academies Press; 2011. 2p. Also available: http://nationalacademies.org/hmd/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx External Web Site Policy.

How will NGC determine the extent of adherence to the IOM standards?

Based on stakeholder feedback, the AHRQ/NGC team developed the National Guideline Clearinghouse Extent Adherence to Trustworthy Standards (NEATS) Instrument and pilot tested it to ensure reliability. NGC staff trained to use the NEATS instrument will assess the extent to which CPGs adhere to elements of these IOM standards:

  • 1.1 (Establishing transparency)
  • 2.2 (Management of conflict of interest)
  • 3.1 and 3.2 (Guideline development group composition)
  • 4.1 (Clinical practice guideline–systematic review intersection)
  • 5.1 (Establishing evidence foundations for and rating strength of recommendations)
  • 6.1 (Articulation of recommendations)
  • 7.1 (External review)
  • 8.1 (Updating)

When will NGC implement the NEATS guideline assessments?

Determining and posting assessments of the extent to which guidelines accepted for inclusion in NGC adhere to the IOM standards may begin sometime in 2017.

Will NGC assess all the guidelines represented on the site?

No, only guidelines accepted for inclusion after implementation of NEATS will be assessed.

Can guideline developers give input on NGC's assessment of their guidelines?

Yes, guideline developers can review and respond to the NEATS assessments of their guidelines prior to publication on the NGC Web site.

Can a guideline developer choose not to have their guideline assessed?

No, following implementation, any guideline accepted for NGC inclusion will be assessed.

Can users see which guidelines have assessments?

Yes, search results will display a NEATS Assessment icon next to assessed NGC summaries to help users identify them. In addition, a specialized search filter will identify all NGC summaries in the database that have assessments.

What do the NEATS assessments mean?

The NEATS assessments provide insight into whether a guideline was developed in a transparent and rigorous fashion, as deemed by the IOM standards. Please note, NGC does not develop, produce, approve, or endorse the guidelines represented on this site. Refer to the Disclaimer page for more information.

Will a lower score on any of the NEATS items mean that the guideline will be withdrawn from NGC?

No, once a guideline is accepted into NGC, it will remain represented on the web site until it no longer meets NGC's inclusion criteria.

Is there a single overall assessment for each guideline assessed?

No, at this time, there is no overall assessment. Each item stands alone. Determining how best to weight the individual items in order to make an overall assessment requires more thought.

Guideline Summaries

Why don't all of the guideline summaries include references supporting the recommendations?

NGC uses a systematic approach to decide when it is appropriate to include author, year citations in the Major Recommendations field and full citations in the References Supporting the Recommendations field. Two criteria drive the decision-making process: (1) the recommendations are explicitly stated, and (2) those explicit statements are referenced. The first criterion means that the recommendations in the guideline document are easily identified, stand-alone, action-oriented statements. Explicit recommendation statements are not interspersed among rationale in the guideline document. Rationale for the recommendations may be provided in sections before or after the actual recommendations but not within the same body of content as the recommendations. The second criterion means that the reference(s) supporting the explicit recommendation is (are) provided at the end of the statement. References cited in the rationale for the recommendation are not captured.

Why are some fields, like the References Supporting the Recommendations field, displayed in some summaries but not in others?

NGC will not display a field when the original guideline lacks relevant content.

What are UMLS concepts, and how are they used in NGC?

The U.S. National Library of Medicine (NLM) External Web Site Policy has developed the Unified Medical Language System (UMLS) External Web Site Policy to facilitate the development of computer systems that behave as if they "understand" the meaning of the language of biomedicine and health. The UMLS consists of three knowledge sources, the Metathesaurus, the Semantic Network, and the Specialist Lexicon, as well as computer tools to help developers integrate the information from these sources.

The Metathesaurus is a collection of more than 100 controlled vocabularies, thesauri, and code sets linked together via high-level semantic concepts. NGC's master's level indexers apply terms from selected UMLS vocabularies to each guideline summary, expert commentary, and guideline synthesis to facilitate searching and browsing and to create relationships between similar documents. These vocabularies are:

  • Healthcare Common Procedure Coding System (HCPCS)
  • International Classification of Diseases – Clinical Modification (ICD-9-CM)
  • Medical Subject Headings (MeSH)
  • Physician Data Query (PDQ)
  • Standard Product Nomenclature (SPN)
  • Systemized Nomenclature of Medicine (Clinical Terms) (SNOMED CT)
  • UMLS Metathesaurus (MTH)
  • Universal Medical Device Nomenclature System (UMDNS)

What are Multiple Chronic Conditions (MCCs)?

Chronic conditions are conditions that last a year or more and require ongoing medical attention and/or limit activities of daily living. Multiple chronic conditions (MCCs) are concurrent chronic conditions. In other words, multiple chronic conditions are two or more chronic conditions that affect a person at the same time. For example, either a person with arthritis and hypertension or a person with heart disease and depression, both have multiple chronic conditions. (Source: http://www.hhs.gov/ash/about-ash/multiple-chronic-conditions/index.html External Web Site Policy).

The MCC tag in a guideline summary identifies that the guideline provides actionable guidance addressing the treatment of two or more concurrent chronic conditions.

Are guideline summaries available in a downloadable format?

Each guideline summary provides a printable view via the Print link and several download types: PDF (download the free Adobe Reader External Web Site Policy), Word (compatible with Word 1997 and higher), HTML, and XML. A citation for individual guideline summaries can also be downloaded into EndNote by clicking on the Citation Manager link.

How do I obtain the original full-text guideline upon which the NGC guideline summary is based?

For guidelines for which NGC has received the necessary permissions from the guideline developer, links are provided from the NGC summary to the full-text document at the developer's Web site. Users can find this link by referring to the "Source" tab at the top of each summary or by going to the Guideline Availability field that appears near the end of each summary. NGC is unable to fulfill orders for printed copies of the original full-text document.

How do I cite the information found at the NGC Web site (e.g., a guideline summary, guideline synthesis, or expert commentary)?

The suggested citation format for a Guideline Summary:

  • National Guideline Clearinghouse (NGC). Guideline summary: [insert title of summary]. In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited YYYY Mon DD]. Available: http://www.guideline.gov.

    Example: National Guideline Clearinghouse (NGC). Guideline summary: Screening for breast cancer. In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited 2010 Jul 21]. Available: http://www.guideline.gov.

The suggested citation format for a Guideline Synthesis:

  • National Guideline Clearinghouse (NGC). Guideline synthesis: [insert title of synthesis]. In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited YYYY Mon DD]. Available: http://www.guideline.gov.

    Example: National Guideline Clearinghouse (NGC). Guideline synthesis: Screening for breast cancer. In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited 2010 May 7]. Available: http://www.guideline.gov.

Note: If you are citing the original guideline upon which the summary or synthesis is based, please refer to the Source field included in each guideline summary or the sources listed under the Guidelines Being Compared section found at the top of each guideline synthesis.

The suggested citation format for an Expert Commentary:

  • Author [insert Commentary author]. Commentary title. In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited YYYY Mon DD]. Available: http://www.guideline.gov.

    Example: Wachter RM. Is the Measurement Mandate Diverting the Patient Safety Revolution? In: National Guideline Clearinghouse (NGC) [Web site]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [cited 2009 Nov 8]. Available: http://www.guideline.gov.

Publication Activity

How can I find out about new guideline summaries published to the NGC Web site?

See the New this Week page.

You can register for the NGC update e-mail service to receive notices of weekly additions and updates posted to the Web site. Go to the Subscribe to Weekly E-mail page to register for this service.

How can I find out whether a particular guideline will be added to NGC in the future?

Users can go to the Guideline Summaries In Progress page to see a list of guidelines in the work queue. Guidelines on this page include new and updated content that meet Inclusion Criteria and have the necessary copyright permissions.

How can I obtain an earlier version of a guideline currently included in NGC?

NGC summaries reflect the most recent version of a guideline. NGC does not provide access to summaries of earlier versions of guidelines that have been updated or withdrawn by the developer. Contact the guideline developer directly for information regarding earlier versions of a particular guideline.

What happens when a guideline summary is removed from the NGC Web site?

See the Guideline Summaries Archive for more information. This feature provides a complete list of summaries that have been withdrawn from the NGC Web site. Information regarding a current NGC summary, where available, is provided. The listings are organized alphabetically by guideline developer.

Why are many guideline summaries removed from NGC at the end of the year, and how do I know whether they are going to be updated?

As part of the NGC Annual Verification, many guidelines are withdrawn from the Web site at the end of every year because they no longer meet our Inclusion Criteria with respect to date. NGC's inclusion criteria specifically require that guidelines represented in our database have been developed, reviewed, or revised within the last five years. All guidelines that no longer meet this criterion are removed from the Web site at the end of each calendar year. See the Guideline Summaries Archive for a list of withdrawn or updated summaries.

To learn whether a guideline update is in progress, contact the guideline developer directly.


Page Last Reviewed or Updated: May 26, 2017