Skip to main content
  • Expert Commentary
  • July 05, 2010

Focus on Accessibility Results in Improved Function and Content for NGC and NQMC

This month (July 2010), the National Guideline Clearinghouse™ (NGC) and National Quality Measures Clearinghouse™ (NQMC) will unveil their redesigned Web sites. Updating and improving the NGC and NQMC Web sites involved several years of planning by the Agency for Healthcare Research and Quality (AHRQ); its contractor, ECRI Institute; and ECRI's technology provider, Silverchair. The intent of this commentary is two-fold: first to discuss the importance of Web content accessibility and its effect on the redesign process and second to inform NGC and NQMC users about the many improvements at the redesigned Web sites. We believe that the newly redesigned sites provide both a high level of accessibility and standards-based best practices for health care information delivery.

Standards-based Accessibility

Chief among AHRQ's priorities for the NGC/NQMC relaunch was the need for clinical practice guidelines and quality measures to remain highly accessible and deployable on a standards-based technology platform that supported and promoted their meaningful use across many types of health care settings. Accessibility has always been critical to NGC's and NQMC's mission to disseminate and promote the use of evidence-based clinical practice guidelines and quality measures. But since the launch of NGC and NQMC by AHRQ in 1999 and 2002 respectively, accessibility requirements and expectations have evolved (and continue to do so) with the rapidly growing adoption of digital technology and use of the Internet for disseminating health care information.

Inherent in the concept of accessibility is access, as in making information available to and usable by people of all abilities. But true accessibility also suggests information that is useful and usable—where the knowledge within documents can be found with precision and where new knowledge can be discovered. In this era of information explosion, computers are being called on to become increasingly intelligent agents to help humans parse through the vast amount of information on the Web. But in order for this information to become accessible to computers, it must be identified semantically—that is, according to its meaning—through indexing with standard or "controlled" vocabularies that allow computers to identify the knowledge within those documents and to break the documents down into component pieces of data that can be found with precision and connected across health care settings. In order for these connections to be meaningful and useful, agreement upon and use of consistent standards is essential. The adoption of such standards, particularly controlled vocabularies, has been noted as one of the grand challenges of health care in pursuing the policy goals of the American Recovery and Reinvestment Act (ARRA). (1) According to Halamka, the enablers of standards are "content, vocabulary, and ways for getting data from place to place." (1)

Our goal to provide true accessibility by offering guideline and measure content that is both useful and usable influenced the redesign team's use of standards, such as controlled vocabulary and indexing classification systems, to improve both function and content at the NGC and NQMC Web sites.

What's New, What's Different at the Web Sites?


Users of the new and improved sites will notice significant upgrades to the accessibility of information, which have been accomplished through the following:

  • Improved site architecture designed according to Web usability standards
  • Improved home page and site navigation that allow users to more easily understand and access the sites' content (also, per Web usability standards)
  • Improved presentation of summaries for guidelines and quality measures, for ease of navigation both within the summary and to related content at the Clearinghouses
  • Improved browse and search functionality that better leverages the classification of guidelines and quality measures using key vocabulary standards

Home Page/Site Redesign

The redesigned Web sites display a modernized "look and feel," with home page emphasis on how visitors can use the sites to "find, learn, compare, and ask." Global navigation promotes the ability of users to conjure up a "mental map" of the site content, including not only the core content of guideline and measure summaries but also expert commentaries, guideline syntheses, and tools for performing side-by-side comparisons of guidelines or measures. The site redesign was guided by rigorous usability testing by users wanting complete and accurate answers to questions quickly. In addition, to challenge pre-test assumptions, testers were asked nontraditional questions such as, "How would you navigate through the summary?"; "What do you think you will get if you click on the Guideline/Measure Classification tab?"; and "If you were looking for Expert Commentaries, where would you click?" The responses helped ECRI Institute and Silverchair make adjustments to better reflect the specific needs of NGC/NQMC users.

New — Indexing Classification System

The most significant addition to the display of guideline and quality measure summaries is the exposure of the indexing classification system applied by the ECRI Institute team to each summary. Although ECRI Institute has indexed the content on NGC and NQMC since their inception, this effort occurred "behind the scenes." With the relaunch, this indexing will be displayed and used to connect related content to provide users with paths to highly relevant information. This new feature drew enthusiastic approval from users who reviewed beta releases of the Web sites. These physicians, nurses, researchers, health policy experts, and experts in patient safety and quality improvement recognize the role controlled vocabulary standards play in allowing interoperability among electronic health records, decision support tools, data repositories, and the resources of the National Library of Medicine, to name a few.

Controlled vocabularies are the normalizing, connecting force in health care information. The National Library of Medicine developed the Unified Medical Language System (UMLS) to manage a robust collection of vocabularies, including MeSH, SNOMED CT, ICD-9-CM, UMDNS, and others, with which ECRI Institute's experts index each summary, expert commentary, and guideline synthesis. The new content delivery platform and its search engine use these classifications of the content to facilitate searching and browsing within the sites and to create relationships between similar documents—and potential relationships to information in other databases and applications. See the table below for the terminology classification systems used in NGC and NQMC.

Search Results

Users will note significant improvements in the accuracy of search results due to the introduction of a semantic search engine, which uses the layers of indexing meta-data (the data about the data) as signifiers of meaning, as opposed to a full-text search engine that relies on the way a document is written (i.e., in a humanly inconsistent way) to find a match with a user's search query. Thus, semantic matches score more highly in our search algorithm because they are infused with valuable knowledge applied in the indexing process. In addition, Silverchair incorporated its proprietary thesaurus of 289,000 equivalents (synonyms, abbreviations, jargon, British spellings, etc.) into the search process to overcome the common problem of searching for health care information: people use different terms to search for exactly the same thing (and content creators use different terms to write about the same thing). Once the sites are relaunched, users can be confident that searches for chronic obstructive pulmonary disease and COPD will find the same summaries, as will searches for EHR, electronic health record, and EMR, and electronic medical record, for example.

Other notable improvements include "breaking down" broad search terms and the ability to search using ICD-9 or SNOMED codes. For example, entering a search term like diabetes will prompt users to specify which type of diabetes they are looking for: Type 1? Type 2? Both? In addition, users who know code sets like ICD-9 or SNOMED or UMDNS can search using numeric codes and receive the same results they would if searching for a concept by name.

At the conclusion of a demonstration of the redesigned Web sites to the NGC/NQMC Core Editorial Board, one member commented that he could not think of any other features these sites could need. Although this was gratifying for our team to hear after the tremendous effort made for the relaunches, we believe that improving the Clearinghouses will be a continuing endeavor, as accessibility requirements continue to evolve in partnership with technology developments and user needs and expectations. AHRQ, ECRI Institute, and Silverchair are committed to this ongoing effort and ask that you, the users of NGC and NQMC, communicate with us about the new features and continue to challenge us to improve these powerful health care information resources.

Table: Main Terminology Classification Systems Used in NGC and NQMC

Terminology Classification Systems Expansion Terminology Developer
MeSH Medical Subject Headings National Library of Medicine
SNOMED CT Systematized Nomenclature of Medicine—Clinical Terms College of American Pathologists and the International Health Terminology Standards Development Organisation (IHTSDO)
ICD-9-CM International Classification of Diseases, Ninth Revision, Clinical Modification World Health Organization
UMDNS Universal Medical Device Nomenclature System ECRI Institute
MTH Metathesaurus UMLS
PDQ Physician Data Query database National Cancer Institute


Elizabeth Willingham, BA
Chief Operating Officer
Silverchair, Charlottesville, VA

Lisa T. Haskell, MS, OTR/L
Project Manager
National Guideline Clearinghouse
ECRI Institute, Plymouth Meeting, PA

Vivian H. Coates, MBA
Project Director
National Guideline Clearinghouse
National Quality Measures Clearinghouse
ECRI Institute, Plymouth Meeting, PA

Project Officer
National Guideline Clearinghouse
National Quality Measures Clearinghouse
Agency for Healthcare Research and Quality (AHRQ), Rockville, MD

Potential Conflicts of Interest

Elizabeth Willingham is the Co-founder and Chief Operating Officer of Silverchair. Lisa T. Haskell and Vivian H. Coates are employees of ECRI Institute.


  1. Halamka, John. Joseph Leiter National Library of Medicine/Medical Library Association Lecture. May 26, 2010. Available at: External Web Site Policy. Accessed 6/14/10.

Make a Comment

All submitted comments will be reviewed by NGC Staff and NGC's Editorial Board who will elect what to publish on this web site.

* indicates required field

Submit a Comment

Do you have any Disclosures and/or Conflicts of Interest to report? *

Optional Information

About Commenting

The National Guideline Clearinghouse ™ (NGC), sponsored by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services, welcomes responses to our posted expert commentaries on the NGC Web site.

All responses will be reviewed by NGC Staff and NGC's Editorial Board. Please keep your response brief and to the point. Responses must be presented in clear statements. We reserve the right to 1) publish your response, 2) copyedit your response, and 3) invite the author of the Expert Commentary to reply to a posted response. Excessively long or offensive and nonobjective responses will be disregarded.

Please complete the form if you wish to have your response made accessible to our users or if you wish for NGC to act on your response. NOTE: Providing such information however, does not ensure that your comment will be published or made available or that NGC will take any course of action. That will be determined by NGC staff in conjunction with our Editorial Board after review of the information you supply. Criteria for publication of responses to commentaries include:

  • Relevance of response to the subject of the commentary
  • Responses that provide a unique perspective to the commentary
  • Responses that offer an alternative view than the commentary
  • Responses that are clearly articulated
  • Responses that are neither excessively long or offensive and/or nonobjective
Form Approved    OMB No. 0935-0106    Exp. Date 11/30/2017

Public reporting burden for this collection of information is estimated to average 90 seconds per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0106) AHRQ, 5600 Fishers Lane, Rockville, MD 20857.

Note: Responses accepted for publication will be posted to the NGC Web site along with the submitter's name. Current position or occupation, organizational affiliation, and disclosed potential conflicts of interest are optional and will be posted when provided. Email and other address are for internal use only and will not be published with responses or made publicly available.