Introduction
This motivated health services to try and detect such cases at an earlier stage by monitoring blood oxygen levels in people diagnosed with COVID-19 at home using pulse oximetry. This could reassure people who did not need to go to hospital, whilst more quickly identifying individuals with dangerously low blood oxygen saturations (
,
This was followed by a national implementation during the Winter of 2020/21. The service was known as COVID Oximetry @home ([email protected]) and by the end of January 2021 it was operating in all clinical commissioning areas of England.
Some sites started by only enrolling individuals aged 65 or over, or who were deemed extremely clinically vulnerable. Others extended enrolment to a wider age group, and often these criteria changed over time.
,
[Preprint] Beaney T, Clarke J, Alboksmaty A, et al. Population level impact of a pulse oximetry remote monitoring programme on mortality and healthcare utilisation in the people with covid-19 in England: a national analysis using a stepped wedge design. medRxiv 2021. https://doi.org/10.1101/2021.11.29.21266847.
,
Our quantitative approach used combinations of unlinked, aggregated population-level data and hospital administrative data. In doing so we were able to undertake a rapid analysis that not only complemented the other evaluations but provided valuable insight in the future evaluation of similar programmes implemented at scale.
Methods
Study design
between the evolving coverage of the programme within each area and outcome. We analysed four outcomes: mortality from COVID-19, hospital admissions for people with confirmed or suspected COVID-19, in-hospital mortality for these admissions and their lengths of stay. For the in-hospital outcomes, we used an observational design relating in-hospital mortality and lengths of stay at an individual patient level to the degree of coverage of the [email protected] programme within the area at the time of admission.
Setting and participants
Data and variables
If someone had more than one positive test within the previous seven days, then only one was counted. These data were aggregated by week, age band and CCG. The selected age bands were 65 to 79 and 80 plus. Numbers of people onboarded to [email protected] were sourced from a bespoke national data collection for the programme and aggregated by the team at Imperial College London undertaking one of the other two simultaneous evaluations. Due to small numbers, aggregation was performed by fortnight, rather than week, and by the same age bands and by CCG. To comply with data protection rules, these data were also rounded to the nearest five individuals, or, for smaller values, labelled as between one and seven.
Data on hospital admissions and outcomes were obtained from Hospital Episode Statistics (HES). Although most of the non-hospital data was available weekly, we aggregated to fortnightly data in order to match the aggregation of the onboarding data. We restricted our statistical analysis to the period between 2 November 2020 and 21 February 2021 when numbers of cases and outcomes were at their peak. Also, outside that period there were too many low numbers at our chosen level of granularity.
We estimated coverage in two ways. One was to calculate it for each CCG regardless of whether a service was operating at the time, and this was used in our analysis. However, to understand what coverage was achievable once a service was implemented, we also estimated coverage within individual CCGs over periods when we knew a service was operating there. For this we only included fortnights over which a service was operating within the CCG for the entirety.
Comparisons between included and excluded CCGs
We compared population characteristics and COVID-19 incidence rates between the CCGs we included, because their data was believed to be complete, and the remaining CCGs to test how representative the included CCGs were. The mean values and proportions associated with each CCG were treated as the separate observations. Normality was assessed by viewing Q-Q plots of the variables and comparisons were carried out using Student t-test, or Mann-Whitney U-tests where data were skewed. We also investigated their geographical spread.
Analysis of mortality
Other options for lagging the time between diagnosis, onboarding and mortality were tested in sensitivity analysis and reported in the supplementary material.
Analysis of hospital admissions
Hospital admissions over the study period were extracted from Hospital Episode Statistics (HES). We considered any admission where COVID-19 or suspected COVID-19 appeared as a diagnosis in the first episode of care, whether as a primary or secondary diagnosis (ICD-10 codes U07.1 and U07.2). If a patient was readmitted with one of these diagnoses within a 28-day period, we only considered the first admission. To match the onboarding data, numbers were aggregated by age band and fortnight.
We undertook a similar procedure for hospital admissions as for mortality, although with different weights, since the time between diagnosis and admission tended to be shorter.
Again, for our sensitivity analysis, we tested different options for lagging the time between diagnosis, onboarding and outcomes. We also tested the option of only including admissions where COVID-19 or suspected COVID-19 was the primary diagnosis.
Analysis of in-hospital outcomes
to analyse the impact on lengths of stay of the weighted coverage for the relevant CCG, again with individual patient characteristics as confounders. Stays longer than 60 days were trimmed to 60 days to mitigate the influence of very long stays. Because we used negative binomial models, ratios in outcomes led to our deriving the impact on length of stay as a percentage change rather than a number of days.
Using rounded data
Patient and public involvement
Members of the study team met to discuss the study with service users and public members of the NIHR BRACE Health and Care Panel and patient representatives from NIHR RSET. Although mostly used for the qualitative evaluations in the wider study, meetings were held during the data analysis phase to share learning and cross-check our interpretations of findings.
Data governance and ethics
The receipt of aggregated data from Public Health England was governed by a data sharing agreement. Receipt of aggregated onboarding data from Imperial College was governed by their separate data sharing agreement with NHS Digital. The access and use of HES was governed by an existing data sharing agreement with NHS Digital covering NIHR RSET analysis (DARS-NIC-194,629-S4F9X). Since we were using combinations of aggregated data and datasets for which we already had approval to use, no ethics committee approval was needed for this analysis. No patient consent was required for this study.
Role of the funding source
The funders had no role in study design, data collection, analysis and interpretation and the decision to publish the manuscript. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR, NHSE&I, NHS Digital or the Department of Health and Social Care. CSJ and TG had access to individual HES records and the raw data on new cases and deaths under the terms of the data sharing agreements with NHS Digital and Public Health England, respectively. All authors had access to aggregated data from these sources as well as all other data used in the study. All authors decided to submit the manuscript for publication.
Discussion
[Preprint] Beaney T, Clarke J, Alboksmaty A, et al. Population level impact of a pulse oximetry remote monitoring programme on mortality and healthcare utilisation in the people with covid-19 in England: a national analysis using a stepped wedge design. medRxiv 2021. https://doi.org/10.1101/2021.11.29.21266847.
, could be carried out more rapidly and, if COVID-19 continues to stretch national health services, it could be more readily repeated as new data become available, provided the right information is routinely collected at source.
We were able to handle small number suppression and the rounding of aggregate data by multiple random sampling throughout the range of possible values and it was encouraging to discover that this uncertainly did not have a large impact on results.
However, obtaining linked individual-level data is a complex and potentially long process that may not always be feasible when there is a need to provide rapid feedback to a developing programme and where resources are stretched. Unfortunately, however, our ability to provide rapid feedback was compromised by delays in obtaining onboarding data which proved an understandable challenge for local services in the midst of a pandemic.
We anticipated that finding a suitable comparator group during the national implementation of a programme was likely to be problematic, and we therefore avoided this problem by treating the relationship between coverage and outcome as a dose-response. However, the power to detect any impact in such an analysis depends on the level of coverage which, in practice, was lower than we hoped.
which could have had a confounding effect on our analysis.
[Preprint] Beaney T, Clarke J, Alboksmaty A, et al. Population level impact of a pulse oximetry remote monitoring programme on mortality and healthcare utilisation in the people with covid-19 in England: a national analysis using a stepped wedge design. medRxiv 2021. https://doi.org/10.1101/2021.11.29.21266847.
However, the study did find reductions in mortality and increases in hospital attendance (yet with lower use of critical care) amongst people enroled onto the programme after attending the Emergency Department (ED).
[Preprint] Beaney T, Clarke J, Alboksmaty A, et al. Evaluating the impact of a pulse oximetry remote monitoring programme on mortality and healthcare utilisation in patients with covid-19 assessed in Accident and Emergency departments in England: a retrospective matched cohort study. medRxiv 2021. https://doi.org/10.1101/2021.11.25.21266848.
A study of 4384 high risk patients receiving home monitoring of vital signs, including pulse oximetry, in one region of Galicia, Spain, found lower admissions, lengths of hospital stays and in-hospital mortality when compared with other local regions.
A retrospective cohort study from South Africa evaluated the use of pulse oximetry for people diagnosed with COVID-19 to read their own blood oxygen levels without remote monitoring by local health services. As with the Spanish study, the implementation was limited to people deemed to be of high clinical risk.
They found a significant improvement in mortality but no impact on admissions to hospital: the reduction in mortality being explained by earlier admission. A recent study of [email protected] carried out at one site demonstrated reductions in 30-day mortality and lengths of stay amongst people admitted to hospital. This, however, is currently a pre-print prior to peer review and lacks some details about the comparability of the control group. In another study implemented in the UK during the first wave, patients with suspected COVID-19 attending ED were discharged home with an oximeter. They observed a reattendance rate of 4.7% compared to 22.7% amongst a retrospective control group.
However, this was a younger cohort (median age 41 years) and the absolute numbers of reattendance were small (nine in all). Other studies have reported on the safety of similar programmes, but have lacked comparators.
,
,
,
,
At the start of this study we anticipated the services would have higher coverage and complete data would be available from more CCGs. Although the use of aggregated population-level data can enable more rapid evaluation of a new service, these two elements had an influence on the power of the analysis to detect an impact. The resulting shortfall in expected data reflects the challenges of trying to centrally manage a bespoke data collection while services are already stretched. However, sufficient quantities of data are vital to determining whether a service is effective, so it is important to understand how this can be improved, for example, by concentrating data collection in a few sites and using routinely collected data wherever possible.
Furthermore, low coverage raises questions about capacity of both staff and resources in the midst of high infection rates and how it is possible to secure the best value from such a service under the circumstances. The fact that at least one CCG managed to achieve reasonably good coverage indicates the possibility for learning from others.
This study provides an evaluation of the national implementation of remote home monitoring of pulse oximetry for people diagnosed with COVID-19 across the English NHS. Although we detected no significant impact on outcomes, there are potential explanations for this finding that are unrelated to the effectiveness of the programme. Taking due account of populations that may respond less well to oximetry, there is no evidence that future implementation of similar programmes would be unsafe. However, the challenges of providing sufficient data so that effectiveness can be adequately measured need to be overcome.
Acknowledgements
We would also like to thank our Clinical Advisory Group for providing insights and feedback throughout the project (Dr Karen Kirkham (whose previous role was the Integrated Care System Clinical Lead, NHSE/I Senior Medical Advisor Primary Care Transformation, Senior Medical Advisor to the Primary Care Provider Transformation team), Dr Matt Inada-Kim (Clinical Lead Deterioration & National Specialist Advisor Sepsis, National Clinical Lead – Deterioration & Specialist Advisor Deterioration, NHS England & Improvement) and Dr Allison Streetly (Senior Public Health Advisor, Deputy National Lead, Healthcare Public Health, Medical Directorate NHS England & Improvement).
Authors’ contributions
CSJ, TG, SM, SMT and EM contributed to study design and methodology. CSJ and TG participated in data curation, analysis and validation. CSJ and TG have verified the underlying data. NJF led the overall mixed methods evaluation of which this study relates to one workstream. All authors provided input from their own workstreams either with raw data or to help interpret findings. CSJ led on the preparation and writing of the manuscript. All authors reviewed and provided feedback on the manuscript and approved the final version.
Data sharing statement
Individual patient-level data and data supplied under specific data sharing agreements cannot be made available by the study team. Sources for data that are already publicly available are supplied either in the text or the references. Aggregate survey data collected by the study team will be presented when findings from the relevant workstreams to which they correspond have been published.
link
