Selecting and validating outcome measures for the Domestic Violence and Abuse Core Outcome Set (DVA-COS)

Selecting and validating outcome measures for the Domestic Violence and Abuse Core Outcome Set (DVA-COS)

Highlights

  • There are numerous interventions for families experiencing domestic abuse. However, it is unclear which programmes are most effective because evaluations vary widely in both what and how they measure.
  • The Children and Families Against Domestic Abuse (CAFADA) Wellbeing and Safety Scale was selected as the preferred measurement tool to assess two core outcomes: family relationships and feelings of safety.
  • No consensus was reached for a measurement tool capturing the core outcome of freedom to go about daily life.
  • The Short Warwick-Edinburgh Mental Wellbeing Scale (SWEMWBS) and Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) were found to be valid and acceptable measures of wellbeing in domestic abuse experienced child and adult populations, respectively.

Executive summary

Download

Work Package 1

Download

Work Package 2

Download

Summary

There are numerous interventions for families experiencing domestic abuse. However, because evaluations vary in both what and how they measure, it is unclear which programmes are most effective . To mitigate this issue, previous work was conducted to develop a ‘core outcome set’ (DVA-COS) —a small number of outcomes that researchers, service providers, and survivors of domestic abuse agree are the most important to measure: 1) child emotional health and wellbeing; 2) feelings of safety; 3) caregiver emotional health and wellbeing; 4) family relationships; 5) freedom to go about daily life. The DVA-COS aims to help service commissioners and survivors understand the possible benefits of one programme over another.

We commissioned the University of Sussex to carry out a project building on this previous work. This project aims to provide recommendations as to how the five outcomes in the DVA-COS should be measured. Specifically it seeks to:

  • Identify, appraise and select outcome measurement instruments (OMIs) to assess three of the five core outcomes in the DVA-COS: family relationships, feelings of safety, and freedom to go about daily life
  • Validate the SWEMWBS and the WEMWEBS for use with children (aged 11 to 18 years) and adults who have experienced domestic abuse respectively.

Aims

The overall aim of the project was to develop agreement on how to measure domestic abuse outcomes consistently. Even where there is consensus on the outcome construct that should be measured, for example “emotional wellbeing”, it is difficult to compare the effectiveness of different programmes when different measurement tools are used. This inconsistency leads to wasted research and evaluation efforts.

This project had two specific aims:

  • Identify, appraise and select outcome measurement instruments (OMIs) to assess three of the five core outcomes of the DVA-COS: family relationships, feelings of safety, and freedom to go about daily life
  • Validate the Short-Warwick Edinburgh Mental Wellbeing scale (SWEMWBS) and the Warwick Edinburgh Mental Wellbeing scale (WEMWBS) for use with children (aged 11 to 18 years) and adults who have experienced domestic abuse respectively.

Method

The first aim was addressed using an iterative, mixed methods design. This included stakeholder consultation, systematic literature searches, evaluation of tools for scientific strength, usability and acceptability, and consensus methods.

The second aim was addressed using a combination of qualitative and quantitative methods to explore the validity and acceptability of the SWEMWBS and WEMWBS for children and adults with experience of domestic abuse.

Key Findings

  • The Children and Families Against Domestic Abuse (CAFADA) Wellbeing and Safety Scale was selected as the preferred measurement tool to assess two core outcomes: family relationships and feelings of safety.
  • No agreement was reached for an OMI capturing freedom to go about daily life.
  • The work to validate the SWEMWBS and WEMWBS demonstrated the validity and acceptability of the tools for use with in domestic abuse experienced child and adult populations, respectively.

Recommendations

  • A provisional recommendation is made for use of CAFADA Wellbeing and Safety Scale to measure child and adult reports of family relationships and feelings of safety. It is strongly recommended that before widespread use this OMI is subject to further adaptation and evaluation by (or in collaboration with) the tool developers.
  • We recommend that the SWEMWBS (for children aged 11+) and WEMWBS (for adults) should be used to measure wellbeing in the context of evaluation studies (of any quantitative design) seeking to assess the impact of child or family focussed domestic abuse interventions.
    • We suggest minor adaptations to the (S)WEMWBS for use in the domestic abuse context to enhance the acceptability of the measure to children and adults.
  • Further work on measure identification, development and evaluation is required to support use of the DVA-COS. Specifically to:
    • Develop and evaluate a measure of freedom to go about daily life for children and young people and adults.
    • Identify an alternative OMI or adapt the SWEMWBS for use to measure the wellbeing of children under the age of 11.
  • We recommend the development of guidelines for practitioners and researchers about how to use the tools in a ‘care first’ way and how to guard against the tools being used for screening or triaging, or rationing care, as well as guidance for commissioners on how to interpret and use evidence, generated by the completion of OMIs, for the basis of decision making:
    • This guidance needs to reflect the balance between the benefits of data driven decision making and the risk of unduly narrowing the breadth of services or thwarting innovation in the sector.
    • The OMIs’ implementation (including the use of guidance) should be closely monitored and evaluated, to inform any associated refinements and to develop an in-depth understanding of the process and outcomes associated with embedding routine measurement in practice.

Implications for Policy

Previous work has identified a common set of outcomes that researchers, service providers, and survivors of domestic abuse agree are the most important to measure. This project went one step further and identified ways to measure these outcomes that are valid and acceptable to children and adults affected by domestic abuse. This can help policy makers, service commissioners and survivors understand the possible benefits of one programme over another. Further thought is needed as to how the outcome set can be integrated into existing systems and processes to support the use and collection of data for service evaluation and improvement.

Implications for future research

This work makes important strides towards the realisation of a DVA-COS, which will help unify outcome measurement across research and practice contexts. Continued work is required to further develop and evaluate selected measures, as well as to develop a new tool to measure freedom to go about daily life. Further work is needed to describe and evaluate the implementation of the DVA-COS, and to track the benefits and harms associated with outcome harmonisation in this field of research and practice.

Linked Project

You can view the project linked to this publication here: 

Developing outcome measures: Domestic abuse core outcome set
SHARE

Related Publications

See all publications

Domestic Abuse

The REACH Plan: A five-year plan to find out what works to prevent domestic abuse & support child victims

Learn More
  • Report

Domestic Abuse

Strengthening knowledge and awareness in family services of domestic abuse (SKAFADA)

Learn More

Cost ratings:

Rated 1: Set up and delivery is low cost, equivalent to an estimated unit cost of less than £100.

Rated 2: Set up and delivery is medium-low cost, equivalent to an estimated unit cost of £100–£499.

Rated 3: Set up and delivery is medium cost, equivalent to an estimated unit cost of £500–£999.

Rated 4: Set up and delivery is medium-high cost, equivalent to an estimated unit cost of £1,000–£2,000.

Rating 5: Set up and delivery is high cost. Equivalent to an estimated unit cost of more than £2,000.

Set up and delivery cost is not applicable, not available, or has not been calculated.

Click here for more information.

Child Outcomes:

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.

Supporting children’s mental health and wellbeing: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Preventing child maltreatment: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Enhancing school achievement & employment: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Preventing crime, violence and antisocial behaviour: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Preventing substance abuse: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Preventing risky sexual behaviour & teen pregnancy: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Preventing obesity and promoting healthy physical development: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient.

Evidence ratings:

Rated 2: Has preliminary evidence of improving a child outcome from a quantitative impact study, but there is not yet evidence of causal impact.

Rated 2+: Meets the level 2 rating and the best available evidence is based on a study which is more rigorous than a level 2 standard but does not meet the level 3 standard.

Rated 3: Has evidence of a short-term positive impact from at least one rigorous study.

Rated 3+: Meets the level 3 rating and has evidence from other studies with a comparison group at level 2 or higher.

Rated 4: Has evidence of a long-term positive impact through at least two rigorous studies.

Rated 4+: Meets the level 4 rating and has at least a third study contributing to the Level 4 rating, with at least one of the studies conducted independently of the intervention provider.

Rating has a *: The evidence base includes mixed findings i.e., studies suggesting positive impact alongside studies, which on balance, indicate no effect or negative impact.

Click here for more information.