By: Denis Bélanger, B.Sc.Phm., ACPR
Dr. Ned Calonge raises some very good points in his Expert Commentary on the GRADE approach. In addition to the groups he identified that use GRADE, the Canadian Agency for Drugs and Technologies in Health (CADTH) also applies GRADE for generating recommendations. To date, recommendations on insulin analogues and self-monitoring of blood glucose (SMBG) have been developed by our expert review committee, known as COMPUS Expert Review Committee (CERC). CERC develops recommendations and advice with the aim of promoting optimal drug therapy and fostering a sustainable health care system for Canadians. CERC considers the practical needs of policy makers, health care providers, and consumers who will be implementing and using the recommendations.
While applying GRADE, CADTH researchers and CERC members have encountered both advantages and challenges that we would like to share.
From our perspective, the early engagement of the expert committee in identifying critical outcomes for generating a recommendation is a significant advantage. As CERC is responsible for making a recommendation, it is useful for members to be involved in determining what evidence will be required for the committee to fulfill its mandate, and the list of outcomes helps CADTH researchers compile all relevant information from available evidence. With this step in place, the research team avoids guessing about which outcomes should be reported or excluded from the analyses.
Traditional means of assessing quality of evidence focus on individual studies. In contrast, the GRADE process appraises the overall quality of evidence available for each outcome of interest. Quality of evidence can vary across outcomes reported in the same study, since some outcomes may be more prone to bias than others. For example, ascertainment bias due to lack of blinding is more likely for a subjective outcome, such as quality of life, than for an objective outcome, such as plasma glucose concentration. The GRADE approach provides a clear picture of the confidence that the expert committee has in the results for each outcome.
For CADTH, the clarification of how evidence is presented to CERC and how CERC arrives at a specific recommendation is an additional benefit of the GRADE process. The fact that CERC must discuss and visibly disclose the values and preferences around its recommendation helps our audiences better understand the recommendation. This explanation also helps the reader compare his or her values against those of the committee. A reader who places more importance on a value that is aligned with CERC is more likely to adopt the recommendation. For example, a recommendation encouraging the use of a specific agent because of improved safety profile (e.g., significantly less weight gain than the comparators) would be more appealing to consumers who place more value on avoiding the specific side effect. Likewise, a reader with values and preferences that differ from CERC can better judge whether he or she wishes to adopt the recommendation.
While the process for developing GRADE evidence profiles is very well documented by the GRADE Working Group, the process a committee uses to move from evidence to recommendations is not as well defined. The challenge faced by groups and organizations who want to adopt GRADE is how to operationalize GRADE at the Panel/Committee level. To address this challenge, CADTH describes the process that CERC used to generate recommendations in our published optimal therapy recommendation reports on insulin analogues (1) and self-monitoring of blood glucose (2). With each project, we identify opportunities for improved transparency and efficiencies for developing and presenting the evidence, as well as developing, documenting and presenting recommendations.
In our experience, the GRADE process has provided CADTH with a sound and consistent way of approaching the development of recommendations. Regular contact between GRADE Working Group members, CERC members, and CADTH researchers so that they may exchange ideas and share challenges, opportunities, and lessons learned proves invaluable to the successful implementation of GRADE in our work.
Denis Bélanger, B.Sc.Phm., ACPR
Canadian Agency for Drugs and Technologies in Health (CADTH)
The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Guideline Clearinghouse™ (NGC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor, ECRI Institute.
Potential Conflicts of Interest
Denis Bélanger is the Director, Topics and Research for the Canadian Agency for Drugs and Technologies in Health (CADTH). In addition, he lists the Canadian Association for Population Therapeutics as a professional affiliation. He states no financial or personal conflicts of interest.
- Canadian Agency for Drugs and Technologies in Health. Optimal therapy recommendations for the prescribing and use of insulin analogues. Optimal Therapy Report - COMPUS 2009;2(7). Available: http://www.cadth.ca/media/pdf/compus_IA_OT_rec_report.pdf (PDF Help) (accessed 2010 Mar 12).
- Canadian Agency for Drugs and Technologies in Health. Optimal therapy recommendations for the prescribing and use of blood glucose test strips. Optimal Therapy Report - COMPUS 2009;3(6). Available: http://www.cadth.ca/media/pdf/compus_BGTS_OT_Rec_e.pdf (PDF Help) (accessed 2010 Mar 12).
PDF Help: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader® .
Submit Response on Expert Commentary
The National Guideline Clearinghouse™ (NGC), sponsored by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services, welcomes responses to our posted expert commentaries on the NGC Web site.
All responses will be reviewed by NGC Staff and NGC's Editorial Board. Please keep your response brief and to the point. Responses must be presented in clear statements. We reserve the right to 1) publish your response, 2) copyedit your response, and 3) invite the author of the Expert Commentary to reply to a posted response. Excessively long or offensive and nonobjective responses will be disregarded.
Please complete the form below if you wish to have your response made accessible to our users or if you wish for NGC to act on your response. NOTE: Providing such information however, does not ensure that your comment will be published or made available or that NGC will take any course of action. That will be determined by NGC staff in conjunction with our Editorial Board after review of the information you supply. Criteria for publication of responses to commentaries include:
- relevance of response to the subject of the commentary
- responses that provide a unique perspective to the commentary
- responses that offer an alternative view than the commentary
- responses that are clearly articulated
- responses that are neither excessively long or offensive and/or nonobjective
Public reporting burden for this collection of information is estimated to average 60 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0106) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.
Note: Responses accepted for publication will be posted to the NGC Web site along with the submitter's name. Current position or occupation, organizational affiliation, and disclosed potential conflicts of interest are optional and will be posted when provided. Email and other address are for internal use only and will not be published with responses or made publicly available.