Research Measurement Standardization SIG

Research Measurement Standardization Special Interest Group

Advisory Group Chair:
Professor Jeremy Bray (Chair of SIG), Professor and Department Head of Economics, The University of North Carolina at Greensboro. Email:

The activity of the group to date has primarily been focussed on the creation of a core outcome set for alcohol brief interventions in all settings. This project, the Outcome Measures in Brief Intervention Trials: Alcohol (ORBITAL) has been funded by Alcohol Research UK, and the ORBITAL principal investigator is Professor Dorothy Newbury Birch. In Mid-Late 2018, we hope to publish the Core Outcome Set in full, and will be developing new strands of research to extend the work. If you are interested in taking part in the extension, please contact the group Chair


ORBITAL Project Principal Investigator:

Professor Dorothy Newbury-Birch. Professor of Alcohol and Public Health Research, Teesside University. Email: 

ORBITAL project team members:

  • Dr Gillian W Shorter, ORBITAL Project Manager and Consensus Meeting Chair, Teesside University. Email for correspondence:
  • Dr Emma L Giles, Senior Research Lecturer in Public Health, Teesside University. Email:
  • Dr Amy O’Donnell. NIHR School for Primary Care Research Fellow, Newcastle University. Email:
  • Dr Aisha Holloway, CNO Clinical Academic Re-engagement Research Fellow, University of Edinburgh. Email:
  • Professor Nick Heather, Emeritus Professor of Alcohol & Other Drug Studies, Northumbria University. Email:
  • Dr Anne Berman, Associate Professor of Clinical Psychology, Karolinska Institutet. Email:
  • Dr Carolina Barbosa, Health Economist, RTI International. Email:

Funder: This project is funded by Alcohol Research UK

Overall ORBIT project aim

This project aims to develop a Core Outcome Set for Alcohol Brief Interventions to improve the measurement of alcohol-related change. Core outcome sets in other healthcare areas have reduced selective and inconsistent reporting in trials, improved the quality of guidance, and increased the number of studies which can be combined in systematic reviews to determine best practice. The protocol has been published in Trials and is freely available  It has four phases:

Phase one: Systematic review to identify outcome measures and domains used in trials evaluating alcohol screening and brief interventions

Aim: To create a list of domains and instruments used to evaluate outcomes in ABI research.

Method: We conducted a systematic review of ABI trials to identify domains and instruments in ABI evaluations across the range of settings. We searched key electronic databases (e.g. MEDLINE, CINAHL, EMBASE, PsycINFO, the Cochrane Library, Scopus, and Web of Knowledge) for ABI trials, consult with expert groups (INEBRIA members, ISAJE, SSA, KBS, Addaction and other service user groups) to identify the readily available published, peer reviewed trials of alcohol brief interventions. Data extracted included study details, domains, instruments, and psychometric properties of potential outcome measures (COSMIN). The information will be used to form narrative reviews of the state of the heterogeneity in domains, instruments, and reporting.

UPDATE Jan 2018: The review was registered on PROSPERO CRD42016047185 The review of outcomes is being updated in preparation for submission to a journal (up to December 2017). Early findings of the review were communicated at the INEBRIA 2017 meeting in New York, and the review has been informed by comments at both the INEBRIA 2016 and 2017 meetings. The COSMIN checklist will be used to inform the how to measure the outcomes in the Core Outcome Set, with the ranking information given in an explanatory paper to supplement the main Core Outcome Set.

Phase two: Building consensus on domains to form a Core Outcome Set (COS)

Aim: To seek views on which domains are to form part of the COS for ABI research.

Methods: This comprised a two round email Delphi study to evidence expert opinion and help identify a core set of priority measures for inclusion in the statement. Delphi ratings were sought from international ABI experts, trialists, trial design, service users, healthcare professionals, journal editors, and policy makers. To reach these groups we contacted all INEBRIA members, asked for participants from international stakeholder groups and distribution lists, and used personal referral (particularly to recruit those with lived experience). In round one, participants rated each measure using a scale of 1-9, with 1-3 labelled ‘not important for inclusion’, 4-6 labelled ‘important but not critical’ and 7-9 labelled ‘critical for inclusion’. Participants had the opportunity to add additional outcomes and to comment on why they have ranked outcomes as they have. All outcomes were carried through to the second round with first round scores displayed for each outcome. Ethics approval was sought and granted from Teesside University to conduct the Delphi analysis.

UPDATE Jan 2018: The Delphi study concluded in August 2017, with 150 participants enrolled. The scores informed the Consensus meeting held at the INEBRIA conference in 2018, and a manuscript is in preparation of the findings.

Phase three: Consensus meeting to decide on Core Outcome Set (COS)
Aim: To discuss the findings of the Delphi analysis and decide on the final COS domains.

Methods: The results of the Delphi analysis was compiled and presented to a consensus panel prior to the start of the INEBRIA 2017 conference. Participants at the consensus meeting debated the reasons why outcomes should not be included in the Core Outcome Set. The consensus meeting was Chaired by Dr Gillian Shorter who did not vote on the outcomes.

UPDATE Jan 2018: A ten item core outcome set was debated and agreed on (from 15 outcomes ranked in the Delphi exercise). Work is underway to rate the psychometric properties of measurements of these ten domains to create a user friendly core outcome set that covers what needs to be measured, and makes recommendations on how these might be measured based on the current state of the literature. The manuscript is expected to be ready by end March 2018.

Phase four: Reporting, dissemination, and next steps

Aim: To disseminate the ABI statement which includes a user-friendly COS of domains and instruments with corresponding reporting guidance.

Methods: The ORBITAL statement will include the following:

  • Domains of the COS: What should be measured as a minimum standard for ABI research?
  • Instruments for each of the Domains of the COS: How the domain should be measured as a minimum standard for ABI research. Alternative instruments will also be listed to assist researchers, but the Core Instruments will clearly be identified
  • Reporting guidance: also included in the statement would be guidance on how specific measurements should be reported ( e.g., alcohol consumption in standard drinks or units) and consensus would be sought in respect of a range of reporting conventions, including reporting of parameters and effect sizes even if no significant effects are found.

Update Jan 2018: The team are busy creating this ORBITAL statement to share with the ABI community at the earliest opportunity. At present we are ranking the different items based on their Psychometric properties. We are also interested in discussing next steps for the measurement and standardisation in the field of alcohol brief interventions. If you would like to be involved or have ideas and time to contribute, contact the SIG Chair Professor Jeremy Bray If you have questions about the Core Outcome Set, please contact Professor Dorothy Newbury-Birch or Dr Gillian Shorter