Submitted by
Dawna Royall, MSc, RD, Evidence Analyst Contractor, PEN® Team Canada
making the grade
With the start of the school year, our thoughts turn to grading, and in particular, grading evidence. We've been hearing a lot about GRADE - the Grading of Recommendations Assessment, Development and Evaluation. In this overview, we'll look at the evidence grading approach that PEN® uses and compare it to the GRADE approach.
Grading evidence-based research allows the user to quickly draw conclusions about the quality of the evidence and use this information to guide their decisions. GRADE distinguishes itself from other grading systems by providing a systematic approach to grading the quality of evidence and the strength of recommendations (
http://www.gradeworkinggroup.org). Many international organizations have started to use this approach, including the Canadian Task Force on Preventive Health Care, the National Institute for Clinical Excellence (NICE), and the World Health Organization.
PEN's Grading System
Key Practice Points in PEN
® are graded as one of four levels (A, B, C, or D) by assessing the quality of evidence as reported in all of the evidence statements (
http://www.pennutrition.com/EvidenceGrade.aspx). This grading approach takes into consideration the following factors:
- study design for answering the practice question (i.e. different types of studies are applicable when considering treatment or prevention)
- quality of the studies
- consistency across studies
- clinical impact or potential benefit of applying the recommendation to a population
- generalizability to other populations
- applicability to practice settings
Thus, an 'A' grade of evidence tells us that given the best quality evidence available, the conclusion is supported by good evidence. In contrast, a 'D' grade of evidence suggests that a conclusion is not possible or extremely limited due to a lack of evidence or extremely poor quality evidence.
The GRADE system
Using the GRADE system, the quality of evidence is similarly categorized into one of four levels:
- High = further research is very unlikely to change our confidence in the estimate of effect
- Moderate = further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate
- Low = further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate
- Very low = any estimate of effect is very uncertain.
Evidence from randomized controlled trials are initially rated as "high quality", but can be decreased for reasons such as study limitations, inconsistency of results, indirectness of evidence, imprecision and/or reporting bias. In contrast, observational studies start with a "low quality" rating, but can be graded upwards if the magnitude of the effect is large or if there is a dose-response relation.
Panelists using the GRADE approach, then consider the strength of a recommendation on the basis of further criteria, including:
- Balance between desirable and undesirable effects
- Variability or uncertainty in values and preferences
- Costs (resource utilization)
A strong recommendation is based on high quality evidence that an intervention’s desirable effects are clearly greater than its undesirable effects (strong recommendation for an intervention), or are clearly not (strong recommendation against an intervention). This implies that most individuals will be best served by the recommendation.
A weak recommendation is made when there is uncertainty about the trade-offs (because of low quality evidence or because the desirable and undesirable effects are closely balanced). This implies that while some people may want the recommended course of action, many would not.
PEN versus GRADE - you decide
Although PEN® grading does not force a strong or weak recommendation for or against a strategy, the overall conclusion in a key practice point considers factors such as applicability and generalizability. In both systems, clinicians need to consider the evidence and recommendations using a client-centred approach. The GRADE system does avoid confusion created by different organizations using different systems to grade evidence and provide recommendations. For example, the same evidence and recommendation could be graded "Grade B, Level 2", "A" or "strong evidence, strongly recommended" depending on what system is being used. PEN® needs a simple and reliable grading system because of the large number and diversity of writers; however it is uncertain whether introducing the GRADE system will improve our decision-making based on the quality of evidence. The PEN® team keeps up-to-date on new evidence grading systems and changes that may be occurring with existing ones in an effort to continue to keep our grading system credible.