In this article, we describe how to include considerations about resource utilization when making recommendations according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach.
Study Design and Settings
We focus on challenges with rating the confidence in effect estimates (quality of evidence) and incorporating resource use into evidence profiles and Summary of Findings (SoF) tables.
Results
GRADE recommends that important differences in resource use between alternative management strategies should be included along with other important outcomes in the evidence profile and SoF table. Key steps in considering resources in making recommendations with GRADE are the identification of items of resource use that may differ between alternative management strategies and that are potentially important to decision makers, finding evidence for the differences in resource use, making judgments regarding confidence in effect estimates using the same criteria used for health outcomes, and valuing the resource use in terms of costs for the specific setting for which recommendations are being made.
Conclusions
With our framework, decision makers will have access to concise summaries of recommendations, including ratings of the quality of economic evidence, and better understand the implications for clinical decision making.
Introduction
What is new?
Key points
•
Grading of Recommendations Assessment, Development, and Evaluation (GRADE) offers a transparent and structured process to include resource use in the development of health care recommendations.
•
Important differences in resource use should be included along with other important outcomes in evidence profiles and Summary of Findings tables.
•
Key steps in considering resource use are the identification of resource use that is potentially important to decision makers, rating the confidence in effect estimates for important effects on resource use, and valuation of resource use in terms of costs for the specific setting for which ecommendations are being made.
In previous articles of this series, we described the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to formulating a structured clinical question and rating the confidence in effect estimates (quality of evidence) for clinical outcomes. In this article, we highlight economic outcomes of alternative management strategies or interventions and describe how to include evidence on the impacts of interventions on resource use and costs in the GRADE approach. We focus on challenges with rating the confidence in effect estimates and its reporting in evidence profiles and Summary of Findings (SoF) tables.
Section snippets
Resource use and economic evaluation
Health care resources include inputs used at any point in a defined treatment management pathway (Box 1). Non-health care resources include all those inputs provided by other service sectors at any point in the treatment pathway, such as social welfare services (e.g., home adaptation, formal social care, housing) or crime and justice services. Patient and informal caregiver resources include all those inputs provided by patients, their families, or caregivers [1].
What resource use measures to
The GRADE approach
The GRADE recommends that important differences in resource use should be included along with other important outcomes in evidence profiles and SoF tables. Key steps in considering resources in making recommendations with GRADE are as follows:
1.
Identify items of resource use that may differ between alternative management strategies and that are potentially important to patients and decision makers;
2.
Find evidence for the differences in resource use between the options being compared;
3.
Rate the
Identifying potentially important resource use
The first step in identifying important resource use is to clearly state the viewpoint (perspective) from which recommendations are being made. One option is to adopt a societal perspective, that is, a broad viewpoint that includes all important health care, non-health care, and patient and informal caregiver resources, regardless of who pays for them (e.g., third-party payers, patients, families) [5]. This has the advantage of ensuring that who pays does not determine whether an item of
Making judgments regarding confidence in estimates of effect for resource use
There are more than 20 published checklists and instruments for assessing the quality of health economic analyses [17]. However, none are specifically constructed to assess the quality of a body of evidence as defined by GRADE—that is, the confidence in estimates of effect [18].
The GRADE recommends that the confidence in effect estimates for each important or critical economic outcome should be appraised explicitly using the same criteria as for health outcomes. Judgemnts about the confidence
Attaching monetary values to resource use
When a recommendation is made in a specific context, attaching appropriate monetary values to quantities of resource use can aid consistent and appropriate valuation of these outcomes by decision makers. In principle, the values should reflect opportunity costs.
So far as possible, monetary valuation of resource use should be made by applying up-to-date and locally relevant unit costs (i.e., applicable to the context of the guideline) to the measured quantity (i.e., number of units) of each item
Resource use and costs in SoF tables
Table 2 represents a SoF table for the comparison of buprenorphine and methadone for opioid maintenance treatment summarizing the effect estimates and the confidence in those estimates, including resource use and costs. The availability of the evidence profile makes all of the evidence considered for inclusion in the SoF table available to those who want it. In our example, there was little or no difference in health outcomes between buprenorphine and methadone, and buprenorphine cost more. For
Finding economic evidence
Evidence for resource use may be found in a range of research-based sources, including clinical trials, observational studies, technology appraisals, and economic evaluations. It may be published concurrently with, or separately from, reports of clinical studies. Methods for locating previously published and unpublished economic evaluations are summarized elsewhere [36]. Evidence for resource use in a specific setting may be also retrieved from national or local databases, such as drug use from
Conclusions
We described the GRADE approach to rating the quality of economic evidence and how the standard GRADE profile can capture both clinical evidence and data on the resource impact of interventions. Guidelines and recommendations have the potential to help decision makers, clinicians, and patients to improve the quality of care, ensuring the best use of limited resources. Although some guideline developers do not consider resource use and cost explicitly, resource use and costs are just other
Acknowledgments
This article is dedicated to the memory of Alessandro Liberati, a friend and colleague, who will be greatly missed.
The authors would like to thank Silvia Minozzi and Nick Clark for their help in developing the opioid maintenance treatment example and Mirella Longo for useful comments.
Methods for the economic evaluation of health care programmes
(2005)
M. Sculpher et al.
Generalisability in economic evaluation studies in healthcare: a review and case studies
Health Technol Assess
(2004)
M.J. Buxton et al.
Modelling in economic evaluation: an unavoidable fact of life
Health Econ
(1997)
G.H. Guyatt et al.
Grading recommendations: incorporating considerations of resources use
BMJ
(2008)
Palivizumab, a humanized respiratory syncitial virus monoclonal antibody, reduces hospitalization from respiratory syncytial virus infection in high risk infants
Pediatrics
(1998)
M.F. Drummond et al.
Guidelines for authors and peer reviewers of economic submissions to the BMJ
BMJ
(1996)
I. Shemilt et al.
A web-based tool for adjusting costs to a specific target currency and price year
This is the 23rd in a series of articles describing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the certainty of evidence and strength of recommendations for systematic reviews, health technology assessments, and clinical guideline development.
We outline how resource utilization and cost-effectiveness analyses are integrated into health-related recommendations, using the GRADE Evidence to Decision (EtD) frameworks.
Through iterative discussions and refinement, in-person, and online meetings, and through e-mail communication, we developed draft guidance to incorporate economic evidence in the formulation of health-related recommendations. We developed scenarios to operationalize the guidance. We presented a summary of the results to members of the GRADE Economic Evaluation Project Group.
We describe how to estimate the cost of preventing (or achieving) an event to inform assessments of cost-effectiveness of alternative treatments, when there are no published economic evaluations. Evidence profiles and Summary of Findings tables based on systematic reviews of cost-effectiveness analyses can be created to provide top-level summaries of results and quality of multiple published economic evaluations. We also describe how this information could be integrated in GRADE’s EtD frameworks to inform health-related recommendations. Three scenarios representing various levels of available cost-effectiveness evidence were used to illustrate the integration process.
This GRADE guidance provides practical information for presenting cost-effectiveness data and its integration in the development of health-related recommendations, using the EtD frameworks.
Sugar-sweetened beverages (SSBs) providing excess energy increase adiposity. The effect of other food sources of sugars at different energy control levels is unclear.
To determine the effect of food sources of fructose-containing sugars by energy control on adiposity.
In this systematic review and meta-analysis, MEDLINE, Embase, and Cochrane Library were searched through April 2022 for controlled trials ≥2 wk. We prespecified 4 trial designs by energy control: substitution (energy-matched replacement of sugars), addition (energy from sugars added), subtraction (energy from sugars subtracted), and ad libitum (energy from sugars freely replaced). Independent authors extracted data. The primary outcome was body weight. Secondary outcomes included other adiposity measures. Grading of Recommendations Assessment, Development, and Evaluation (GRADE) was used to assess the certainty of evidence.
We included 169 trials (255 trial comparisons, n = 10,357) assessing 14 food sources at 4 energy control levels over a median 12 wk. Total fructose-containing sugars increased body weight (MD: 0.28 kg; 95% CI: 0.06, 0.50 kg; PMD = 0.011) in addition trials and decreased body weight (MD: −0.96 kg; 95% CI: −1.78, −0.14 kg; PMD = 0.022) in subtraction trials with no effect in substitution or ad libitum trials. There was interaction/influence by food sources on body weight: substitution trials [fruits decreased; added nutritive sweeteners and mixed sources (with SSBs) increased]; addition trials [dried fruits, honey, fruits (≤10%E), and 100% fruit juice (≤10%E) decreased; SSBs, fruit drink, and mixed sources (with SSBs) increased]; subtraction trials [removal of mixed sources (with SSBs) decreased]; and ad libitum trials [mixed sources (with/without SSBs) increased]. GRADE scores were generally moderate. Results were similar across secondary outcomes.
Energy control and food sources mediate the effect of fructose-containing sugars on adiposity. The evidence provides a good indication that excess energy from sugars (particularly SSBs at high doses ≥20%E or 100 g/d) increase adiposity, whereas their removal decrease adiposity. Most other food sources had no effect, with some showing decreases (particularly fruits at lower doses ≤10%E or 50 g/d). This trial was registered at clinicaltrials.gov as NCT02558920 (https://clinicaltrials.gov/ct2/show/NCT02558920).
The GRADE system has been developed by the GRADE Working Group. The named authors drafted and revised this article. A complete list of contributors to this series can be found on the JCE Web site at www.jclinepi.com.