IMPRESS: Improving outcomes for ovarian cancer Patients by Refining Evidenced-based tools that drive Surgical Standardisation

Project Title IMPRESS: IMproving outcomes for ovarian cancer Patients by Refining Evidenced based tools that drive Surgical Standardisation

Project Lead & Pilot location Professor Richard Edmondson - University of Manchester

Inequality Criteria: Age, minority groups and location

Objective Criteria: Improve survival rates & access to care as well as improve patient experience

IMPRESS: IMproving outcomes for ovarian cancer Patients by Refining Evidenced based tools that drive Surgical Standardisation

Full QI Project Summary (10- 15 minute read)

Outline

Focus of the Project:

Ovarian cancer is a devastating disease that affects thousands of women in the UK. Unfortunately, the reality for these women is harsh and often filled with uncertainty. One of the most alarming aspects of ovarian cancer treatment is the significant variation in outcomes depending on where a woman lives. Shockingly, the chance of a woman being alive just one year after diagnosis can range from 63% to 76% based on her geographical location. This disparity in treatment outcomes is primarily driven by the differences in how the cancer is treated. It is clear that there is a pressing need to address this issue and ensure that all women diagnosed with ovarian cancer receive the optimal care they deserve, regardless of their location. Recognising the potential of machine learning in healthcare, the IMPRESS project has been initiated to explore its application in improving the treatment and outcomes for women with ovarian cancer.

Aims:

Building upon preclinical work, the project aims to demonstrate that machine learning algorithms can effectively identify patients who may not be receiving optimal care;
• Develop tools for personalised outcome predictions for women with advanced ovarian cancer.
• Enable shared decision-making between women and their clinicians for optimised treatment decisions.
• Expand the database of patient cases to include 7 UK centres and gather detailed information on over 200 variables per patient.
• Improve understanding of decision-making in MDT for ovarian cancer treatment.
• Consult with patients to design a tool that facilitates shared decision-making, powered by personalised prediction of treatment outcomes this will empower women to advocate for the best treatment options for their specific needs.

Issues & Background Information:

The Ovarian Cancer Audit Feasibility Project (OCAFP) has confirmed significant variations in practice between Cancer Alliances in England, leading to potential inequality in treatment. Alarmingly, a considerable number of women with advanced ovarian cancer in the UK may be receiving suboptimal care due to their geographic location. Notably, older women are particularly at risk of not being offered treatment in some centres.

Currently, the decision about what treatment is received is made by an MDT. Whilst highest survival rates are associated with treatments that include both surgery and chemotherapy many patients are treated with other combinations including surgery alone, chemotherapy alone or no treatment at all. Worryingly the rates of these treatment pathways vary considerably around the UK, for instance, the rate of "no treatment" ranges from 15.8% to 27.6% between Cancer Alliances.

Significant Facts:

Disparities in treatment pathways are influenced by factors such as:
• patient age
• fitness
• cancer stage/extent
• surgical expertise
• system resources e.g. theatre time
• the prevailing ethos within different MDTs

The absence of personalised evidence hampers teams from questioning alternative treatments. To address this, we propose a personalised predictive system, empowering patients and their clinical teams to freely explore treatment options. However, challenges include developing accurate predictive models, patient-friendly representation, and integrating the tools into clinical practice.

Solution - project design

Introduction:
The project design aims to leverage a data-driven approach to develop a comprehensive surgical data collection system. The goal is to determine the optimal number of patients for surgery, enhancing survival rates while minimising complications for unsuitable patients. Additionally, the project seeks to expand upon existing algorithms that utilise data to provide decision aids, enabling shared care between clinicians and patients through personalised outcome predictions. A full-time project manager was recruited at the University of Manchester and started in post in April 2022, to coordinate ethical approvals and to monitor project progress.

Identification of Sites:
We carefully selected diverse sites, encompassing both large and small centres with varying surgical resection rates. These participating centres represent a comprehensive range of practices across the UK. Among them, one centre surpasses the 99th percentile for resection rates, while two centres fall below the 1st percentile. Additionally, three centres represent mainstream practice. Experienced clinical leads from each site collaborated as co-applicants, and the existing staff undertook obtaining and uploading of the required data. All sites were allocated funding to facilitate data collection.

Ethical Approval:
Obtaining ethical approval proved more time-consuming than anticipated. Initially classified as a service evaluation, it became clear that additional approval was necessary for data analysis and publication. Extensive ethical applications were submitted to the Health Research Authority (HRA) and the University of Manchester. Despite delays and uncertainties, ethical approval was granted without changes. Data collection and queries were completed by six initial sites.

Data and Measures:

Data Risk:
Collecting accurate and high-quality data was identified as a major risk in the project, given the complexities associated with obtaining data from NHS systems. Within the NHS, data can be stored in various formats and locations, such as electronic patient records, operating theatre management systems, and laboratory results systems. To mitigate this risk, a tightly defined data dictionary, a predefined case report form, and ongoing discussions with data managers were implemented to ensure accurate data collection and uniformity between centres.

Data Collection:
Data from participating sites was transferred securely to the University of Manchester using the approved REDCap system. This allowed data collectors and clinical leads at each site to enter information securely using a predefined case report form. Anonymisation of data eliminated the need for additional confidentiality advisory group approval. At the time of writing, data has been received from 6 out of 7 sites, with a total of 1139 patients included in the dataset. Detailed information about the collected data fields can be found in the project's protocol.

Number of Patients in Data:
The original funding application estimated data collection for 1500 patients. However, as of now, data has been received for 1139 patients from the 6 participating sites.

Resource Requirement Analysis:

As part of the data collection process, sites were requested to provide resource requirements information to understand the time required for data collection at each site. Manchester's data collection was excluded from this analysis as it was completed prior to the funding application.

The analysis aimed to determine the amount of staff time needed if data collection expands to additional sites in the future. While it was assumed that the time needed to collect each record would decrease with familiarity, there was no clear evidence to support this assumption. Additionally, no correlation was found between the number of patients and the time required for data collection. The number of systems required to obtain the data varied across the five centres that provided resource requirements information.

Current practice evaluation:
Data is being used to understand current practice across the six participating centres. Initial validation studies confirm data quality and reveal variations in treatment patterns and survival rates between centres.

Validating Algorithms:
Ongoing work is focused on externally validating the previously described predictive algorithms.

Additional Studies:

Consultation with patients - ADDEPT:
As well as creating a predictive algorithm that provides information on the best treatment to clinicians, we also would like the predictive tool to involve and empower patients to ensure they are receiving the treatment that is best for them, rather than allowing the clinician to steer treatment towards the sites preferred method.

In order to understand what information patients would like to see in a personalised predictive tool, we conducted a Delphi questionnaire involving 25 patients and 20 clinicians to gather input on a personalised predictive tool. Patients emphasised the importance of the following:

• survival prediction
• surgical morbidity
• long-term quality of life

These insights will inform the design of our predictive algorithm. Results are forthcoming for publication.

MDT Observational Study:
In order to better understand our current practice of decision making for patients, and thus how this could be improved we carried out a study of clinical decision making in MDTs. A manuscript of the study is accepted for publication in European Journal Surgical Oncology.
• Trained observers attended MDTs virtually at five participating cancer centres
• Used the GO-MDT-MODe tool to measure input and quality of discussions
• Significant differences were found between centres in meeting structure and format
• Varied input from different staff members to each case reviewed by the MDT
• Notable variation in surgical input, reflecting diverse surgical practices in England
• Highlighted the need for an independent tool to predict patient outcomes and enhance decision- making.

Results:

Given the project's main goals were to develop a consortium to generate a data resource, the key endpoints were as follows:

• Data collection: Determine the proportion of available data collected within the project's timeframe and resource allocation.
• Data quality: Assess the data quality by evaluating the proportion of missing data and the length of follow-up data.

Other outcome metrics specific to individual side projects will be defined in results.

Data Collection Results:

Data were collected from 1139 patients across six centres, with one centre yet to report. This represents the largest dataset of detailed clinical information in the UK. The average time taken for data collection per patient was 27 minutes. These findings inform resource requirements for future projects and enable comprehensive analysis of patient characteristics and treatment outcomes.

Outcome Results:

Analysis of the collected data revealed variations in treatment patterns across the centres. The "ideal" treatment, combining surgery and chemotherapy, ranged from 29% to 76% among the centres. Survival probabilities also varied, highlighting differences in treatment effectiveness and patient outcomes.

MDT Data Results:

During 17 MDT meetings, 870 cases were observed, including 145 advanced ovarian cancer cases. Analysis of discussions for advanced cases showed variations in quality, indicating differing rates of low-quality discussion among the centres.

Delphi Data Results:

Two rounds of questionnaires were conducted with patients (n=22) and clinicians (n=21). The analysis confirmed the importance of predicting survival and perioperative mortality to patients. These insights guide the development of personalised predictive tools and patient-centred decision-making processes.

Lessons learned – challenges & changes:

Lessons Learned:
The project findings revealed significant variation in one-year survival rates (ranging from 63% to 86%) and the proportion of patients receiving ideal treatment (ranging from 29% to 76%) across different centres.

Additionally, some centres exhibited a high proportion of "low quality" MDT discussions, reaching as high as 45%. It was evident from patient feedback that they desired more information to aid in their decision-making process regarding their care, including estimates of overall survival and surgical mortality.

The project demonstrated that high-quality data collection from multiple sites is possible. However, it was also recognised that such an endeavour requires significant resources to ensure accurate and comprehensive data collection.

Limitations and Changes for Future Project:

By addressing these limitations and implementing changes, such as improved time management, enhanced data collection, and continued collaboration, future projects can optimise resources and provide valuable insights to benefit patients.

• Time limitations: Delays in staff recruitment and regulatory approvals were encountered, necessitating consideration of the time required for document preparation, submission to the Health Research Authority (HRA), and receiving confirmation from sites' Research and Development (R&D) departments. Clear understanding of required approval levels is crucial from the project's inception.
  
• Data completeness challenge: Incomplete data sets were obtained from most centres, particularly due to patients receiving treatment at different centres for various stages of their care. Standardisation and consistency in data collection across patients and centres should be emphasised to minimise variations in completed data fields.

Limitations – what we would do differently now:

Improvements and Ongoing Benefits:

• Exploiting the data resource: The successful formation of a unique data resource resulted from collaboration among seven UK centres, which should continue beyond the project's completion to deliver sustained benefits for patients. This entails utilising the data for further analysis, research, and enhancing patient care and outcomes.
  
• Side projects and dissemination: Completed "side" projects, including the ADEPPT study and the MDT study, have generated significant data on decision-making processes and future requirements. Actively presenting and publishing these findings contribute to expanding knowledge in the field.

Conclusion:

Overall, the project has been very successful with the formation of a unique data resource which is currently being exploited to be deliver benefits for patients. This has come about through the formation of an important collaboration between seven centres in the UK which will continue to deliver benefits even after this project has been completed.

The “side” projects including the ADEPPT study and the MDT study have both completed and are being presented and published at present. These have generated really important data regarding current decision making and what is required in the future.

Future Funding:

The progress achieved during the IMPRESS project has paved the way for the consortium to develop a future grant application aimed at further advancing the predictive tool and assessing its usability in clinical practice.

All current sites involved in the IMPRESS project will continue their participation in the future work, while new connections have been established to strengthen the application and project design.

Notable additions include experts in shared decision making, Darren Flynn, a health economist, Joanne Gray, and an expert in machine learning, Glen Martin.

Additionally, the Institute of Cancer Research Clinical Trials unit has contributed to project design and statistical powering, and the governance team at the University of Manchester has provided guidance on the necessary regulations for conducting a trial involving a machine learning algorithm as a medical device within the NHS.

The grant application is being developed for submission in 2023, with the aim of fully utilising the collected data to develop a personalised shared decision-making tool and conducting a clinical trial to evaluate its acceptability among patients and clinicians. The trial will also assess if the tool significantly contributes to ensuring patients receive the most suitable treatment, regardless of the treatment centre. To achieve these goals, the future funding project will be divided into the following work packages:

WP1 – Investigating Shared Decision Making (SDM) & Personalised Decision Support and Shared Decision Making (PDS-SDM): Demonstrate the feasibility and acceptability of shared decision making and a personalised decision support tool for managing advanced ovarian cancer, addressing both patient and clinician perspectives.

WP2 – Adding Decision Support Tool to Shared Decision-Making: Conduct a phase II clinical trial to evaluate the efficacy of integrating a personalised decision support tool into the pathway for patients with advanced ovarian cancer. Stepped-wedge cluster randomisation will be used, comparing shared decision-making (SDM) to personalised decision support and shared decision making (PDS-SDM).

WP3 – Expanding and Refining the Database: Expand and refine the existing database from the IMPRESS project to support the personalised decision support tool.

WP4 – Intellectual Property & Commercialisation: Facilitate the registration of the personalised decision support tool, incorporating a machine learning algorithm, as a medical device with the Medicines and Healthcare products Regulatory Agency (MHRA).

WP5 – Analysing Cost Implications and Training Needs: Analyse the cost implications of practice changes resulting from the personalised decision support tool. Assess the need for additional training for surgeons due to an increase in potentially complex cases.

These work packages aim to further advance the personalised decision support tool, its implementation in clinical practice, and the overall benefits for patients with advanced ovarian cancer.

Project Outputs

We are considering using a separate box on the case study webpage rather than place within the main body text. Then we will link items listed below to the resource library.

The project produced several outputs, including:

Publications

• MDT practice determines treatment pathway for patients with advanced ovarian cancer: a multi-centre observational study. T Khassan, E Smitten, N Wood, C Fotopoulou, J Morrison, M MacDonald, K Baxter, RJ Edmondson. Accepted for publication Eur J Surg Oncol
• Defining optimal rates of surgery in advanced ovarian cancer. K Baxter, B Russell, A Hawarden, M Gee, N Khalil, D Jones, VN Sivalingam, RJ Edmondson. Accepted for publication Int J Gyn Cancer

Presentations

• A Delphi Questionnaire to Engage Patients and Clinicians in the Design of a Personalised Predictive Decision Tool K Baxter, J Booth, RJ Edmondson. Presented at the Blair Bell Meeting London Feb 2023
• Treatment regimens for ovarian cancer determine one-year survival: a study of 6 UK cancer centres. K Baxter, N Wood, M Macdonald, J Morrison, J Yap, C Fotolopoulou, J Booth RJ Edmondson