Book Navigation

Making the AGREE tool more user-friendly: the feasibility of a user guide based on Boolean operators

Overview
Title:
Making the AGREE tool more user-friendly: the feasibility of a user guide based on Boolean operators
Authors:
Scott NA, Moga C, Harstall C
Journal:
J Eval Clin Pract
Publication date:
2009
Volume:
15
Issue:
6
First page:
1061
Last page:
1073
Link to details:
http://www.ingentaconnect.com/content/bsc/jecp/2009/00000015/00000006/art00029
Publication type:
Journal
Free text

Rationale, aims and objectives. The Appraisal of Guidelines Research and Evaluation (AGREE) instrument is a generic tool for assessing guideline quality. This feasibility study aimed to reduce the ambiguity and subjectivity associated with AGREE item scoring, and to augment the tool's capacity to differentiate between good- and poor-quality guidelines. Methods. A literature review was conducted to ascertain what AGREE instrument adjustments had been reported to date. The AGREE User Guide was then modified by:

  • constructing a detailed set of instructions, or dictionary, using Boolean operators, and
  • overlaying seven criteria to categorize guideline quality.

The feasibility of the Boolean-based dictionary was tested by three appraisers using three randomly selected guidelines on low back pain management. The dictionary was then revised and re-tested. Results. Of the 52 published studies identified, 14% had modified the instrument by adding or deleting items and 35% had adopted strategies, such as using a consensus approach, to overcome inconsistencies and ensure identical item scoring among appraisers. For the feasibility test, Pearson correlation coefficients ranged from 0.27 to 0.81. Revision and re-testing of the dictionary increased the level of agreement (range 0.41 to 0.94). Application of the revised dictionary not only decreased the variability of the domain scores, but also reduced the tool's reliability among inexperienced appraisers. Conclusion. Appraisers found the Boolean-based AGREE User Guide easier to use than the original, which improved their confidence in the tool. Good reliability was achieved in the feasibility test, but the reliability and validity of some of the changes will require further evaluation.

Preview

Making the AGREE tool more user-friendly: the feasibility of a user guide based on Boolean operators Scott NA, Moga C, Harstall C. J Eval Clin Pract 2009; 15(6):1061-1073.

Page last updated: Oct 14, 2010
Copyright © 2002-2016 Guidelines International Network.
All rights reserved

The Guidelines International Network is formally constituted as a Scottish Guarantee Company under Company Number SC243691 and recognised as a Scottish Charity under Scottish Charity Number SC034047 with its Registered Office at J. & H. Mitchell W.S., 51 Atholl Road, Pitlochry, Perthshire PH16 5BU, Scotland.

Back to top