Trial Review

Registering a new trial?

To achieve prospective registration, we recommend submitting your trial for registration at the same time as ethics submission.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12623000986673p
Ethics application status
Submitted, not yet approved
Date submitted
11/11/2022
Date registered
11/09/2023
Date last updated
11/09/2023
Date data sharing statement initially provided
11/09/2023
Type of registration
Prospectively registered

Titles & IDs
Public title
An artificial intelligence model to predict the ideal time for ruptured brain aneurysm treatment to prevent re-rupture of the aneurysm prior to its treatment.
Scientific title
Optimal Ruptured Aneurysm CLosure via Endovascular-FIRST TREAtmeant Timing (ORACLE-FIRST TREAT): a patient-specific machine learning clinical prediction model for pre-treatment re-bleeding prevention following saccular aneurysmal subarachnoid haemorrhage
Secondary ID [1] 308392 0
Nil known
Universal Trial Number (UTN)
Trial acronym
ORACLE-FIRST TREAT
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Aneurysmal subarachnoid haemorrhage
 328190 0
Condition category
Condition code
Stroke 325242 325242 0 0
Haemorrhagic

Intervention/exposure
Study type
Observational
Patient registry
False
Target follow-up duration
Target follow-up type
Description of intervention(s) / exposure
This study will use retrospective machine learning analysis of prospectively collected, de-identified data from the Sir Charles Gairdner Hospital saccular aneurysmal subarachnoid haemorrhage database from the 1st of January 2009 to the 31st of December 2022 inclusive. The database has been formulated from the amalgamation of two independently compiled, prospective institutional databases of Sir Charles Gairdner Hospital in Nedlands, Perth, Western Australia. This included the Department of Neurosurgery database composed of hospital admissions, clinical management software entries and Business Intelligence Unit data used for funding allocation together with the Department of Neuroradiology Transcranial Doppler (TCD) ultrasonography database composed of all non-palliated patients admitted to SCGH with subarachnoid haemorrhage of any aetiology. All non-duplicated patients entered into either of these two independently compiled databases within the study period with aneurysmal subarachnoid haemorrhage due to a singular culprit saccular aneurysm identified at the time of admission using either CT-angiography (CT-A), Magnetic Resonance Angiography (MRA) and/or Digital Subtraction Catheter Angiography (DSA) were included. Patients were excluded if they had SAH from a non-saccular aneurysmal source or recurrent SAH from a previously treated saccular aneurysm. The age and sex of the patient at the time of admission will be retrospectively collected. We will retrospectively collect data regarding the following evidence based predictors of pre-treatment re-bleeding: the World Federation of Neurosurgical Societies clinical grade at presentation, the Fisher radiological grade at presentation, the maximal dome size of the ruptured aneurysm in millimetres, the parent vessel location of the ruptured aneurysm, the presence of hydrocephalus at the time of diagnosis, the presence of an intracerebral haematoma at diagnosis, the presence of a subdural haematoma at the time of diagnosis and the presence of intraventricular hemorrhage at diagnosis. We will also retrospectively determine for each patient whether they were managed with an intention for acute aneurysm treatment, an intention for delayed treatment pending neurological improvement and the time from imaging diagnosis of subarachnoid haemorrhage to the time of aneurysm re-bleeding or treatment. No additional prospective information will be collected from patients and all demographic, predictor and outcome information above outlined will be determined through retrospective assessment of each patient's existing medical records, blood test results and radiological imaging including request forms. During the entire 14-year study period, our institutional management protocol for aSAH from a rupture saccular aneurysm was treatment within 24 hours (ultra-early) using an 'endovascular-first' approach in accordance with the results of the 2002 International Subarachnoid Aneurysm Trial (ISAT), the 2002 publication 'Ultra-early surgery for aneurysmal subarachnoid hemorrhage: outcomes for a consecutive series of 391 patients not selected by grade or age' published in the Journal of Neurosurgery as well as both the 2012 and 2023 American Heart Association/American Stroke Association Guideline for the Management of Patients With Aneurysmal Subarachnoid Hemorrhage.
Intervention code [1] 324845 0
Diagnosis / Prognosis
Comparator / control treatment
No control group - this is a retrospective analysis of already collected de-identified data
Control group
Uncontrolled

Outcomes
Primary outcome [1] 335437 0
Pre-treatment rebleeding following single saccular aneurysmal subarachnoid hemorrhage. An episode of pre-treatment rebleeding is defined as a sudden deterioration in the consciousness state or a sudden increase in headache with an elevation of systolic blood pressure together with an increase in acute blood seen on a computerised tomography (CT) brain scan. This will be ascertained by scrutiny of participant discharge summaries as well as electronic radiology imaging including reports and request forms.
Timepoint [1] 335437 0
Using data collected up to and including 31st December 2022
Secondary outcome [1] 424510 0
The time-at-risk for each participant will be calculated in hours from the time of aneurysmal subarachnoid haemorrhage diagnosis by computerised tomography (CT) brain scan or cerebrospinal fluid (CSF) analysis to the time-of-outcome; regarded as the time of aneurysm closure, pre-treatment re-bleeding, death or the time of discharge from hospital against medical advice if treatment was refused. The time of endovascular aneurysm closure will be the time of the final digital subtraction catheter cerebral angiography (DSA) run performed showing embolization of the aneurysm. The time of microsurgical aneurysm closure will be estimated as 1-hour before the time of discharge from the post-anaesthetic care unit which is noted for all patients undergoing surgery at our institution's electronic Theatre Management System software suite. The time of pre-treatment re-bleeding will be regarded as the time of the CT brain scan confirming an increase in the volume of subarachnoid, intraventricular and/or intra-parenchymal blood following the initial aSAH but before aneurysm closure. The time of death will be that which is listed for the participant in the iSoft clinical manager software. The time of discharge will be that which is similarly listed for the participant in the iSoft clinical manager software.
Timepoint [1] 424510 0
Using data collected up to and including 31st December 2022

Eligibility
Key inclusion criteria
The Sir Charles Gairdner Hospital (SCGH) saccular aneurysmal subarachnoid haemorrhage database contains prospectively collected data for patients 18 years or over admitted to SCGH in Nedlands, Western Australia (WA) between 1st January 2009 and 31st December 2022 inclusive with aneurysmal SAH confirmed using either non-contrast computerised tomography (CT) brain scanning and/or cerebrospinal fluid spectroscopic analysis due to the rupture of a singular culprit saccular cerebral aneurysm as identified on CT-angiography, magnetic resonance angiography and/or digital subtraction catheter angiography.
Minimum age
18 Years
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
No
Key exclusion criteria
Patients were excluded from the study population if they experienced cryptogenic subarachnoid haemorrhage (SAH), peri-mesencephalic SAH, SAH due to arterial dissection, fusiform, mycotic or blister type aneurysm, recurrent SAH from a previously treated saccular aneurysm or SAH due to a ruptured arteriovenous malformation.

Study design
Purpose
Natural history
Duration
Longitudinal
Selection
Defined population
Timing
Retrospective
Statistical methods / analysis
During the study period 1st January 2009 to 31st December 2022 inclusive, 1060 episodes of single culprit saccular aneurysmal subarachnoid haemorrhage (aSAH) suffered by 1058 distinct patients were recorded. Of these we excluded 51 episodes of saccular aSAH from the study population. Twenty-one episodes were excluded as there was no CT or CSF evidence of SAH with aneurysm treatment proceeding due to large aneurysm size, multilobulated appearance, a suggestive clinical syndrome, symptomatic neural compression and/or symptomatic brain oedema. Nine episodes were excluded due to the absence of a causative saccular aneurysm on both the initial computerised tomography angiogram scan and digital subtraction catheter angiogram; precluding the use of parameters at admission for re-bleed prediction. Fifteen episodes were excluded as SAH was due to re-rupture of a previously treated aneurysm which would introduce further predictor parameters outside the scope of this study such as the adequacy of initial aneurysm treatment. Four episodes were excluded as they were initially diagnosed and managed in South-East Asia introducing treatment delays not reflective of routine management patterns. A further two episodes were excluded as both were paediatric aSAH cases. This yielded a final study population of 1009 individual episodes of single culprit saccular aSAH suffered by 1007 distinct patients.

We used the pmsampsize package of Riley et al. for sample size calculation for clinical prediction modelling implemented in the R statistical computing environment. We used as an input a Cox-Snell R squared statistic of 0.126 derived from the ARISE-extended model of van Lieshout et al. as it is the most similar recently described model for pre-treatment rebleed prediction. This was also computed using the pmsampsize package and the ARISE-extended model population rebleed prevalence of 0.1296, an approximate predictor parameter number of 20 and their reported optimism corrected C-statistic of 0.79. We then used pmsampsize inputting the Cox-Snell R squared of 0.126, the overall re-bleed rate for our study population of 0.0037 person-hours, the mean follow-up time for our cohort of 22 hours and a timepoint of 24 hours. For our total study population, pmsampsize estimations yielded a maximal allowable predictor parameter number of 15 with an ideal event to predictor parameter ratio of 5.40 for a regression-based survival model such as that predicting time to rebleeding, treatment or death and 5.52 for a regression-based binary outcome model such as that predicting rebleeding vs. no rebleeding. As such, we settled on five binary predictor parameters, three categorical predictor parameters and a single continuous predictor parameter for all machine learning (ML) models to be tested.

Descriptive statistics
The rate of pre-treatment re-bleeding over 14-years will be calculated by dividing the number of cases by the total study population. The overall cumulative time at risk in person-hours will be reported by summation of each participant’s time to outcome. The proportion of study participants palliated as opposed to treated, treated according to each one of three initial management intentions and treated with microsurgical as opposed to endovascular modalities will be presented year-by-year for each of the 14 years. Mean, standard deviation, median and interquartile ranges (IQR) for the time-to-outcome will be presented in hours for both the overall study population and for only the subset of participants who received aneurysmal treatment or re-bled. Descriptive statistics will be presented for demographic factors as well as all clinical and radiological predictor parameters as medians with IQR if continuous and absolute count numbers with percentages if categorical. Differences in these variables will be calculated using Chi square tests for normally distributed data or Mann-Whitney tests for non-parametric data.
Machine learning analysis
We will then train several different ML algorithms using the entire study population as a training data-set. The two objectives of ML modelling will be:
1) Re-bleeding prediction as a binary classification problem
2) Time-to-treatment recommendations as a survival analysis

Due to the significant class imbalance present in the training dataset and the nature of the dataset being a binary classification problem, a similar study protocol to what has been presented in the public domain and has been peer reviewed but as yet unpublished titled 'Supervised machine learning and hematology parameters for blood culture classification' a Masters by Research thesis submitted to The University of Western Australia by Benjamin McFadden. Based on this previous work, we will explore the following methods both alone and in combination as voting ensembles for ML modelling:
- Decision trees
- Random forests
- XGBoost
- Logistic regression
- Support vector machines
- K-nearest neighbours

Similar approaches will then again be used to perform the survival analysis for ML objective (2).

Internal validation
Due to the relatively small size of the dataset, we will use stratified 10-fold cross validation to
evaluate the final model(s). This will provide the opportunity for the evaluation of additional data in the future.

Model performance evaluation
The performance of each considered model(s) will be assessed for discrimination, calibration and utility on decision analysis. Discrimination will be determined using area under the receiver operator characteristic curve analysis. Calibration will be presented as a graphical flexible calibration curve comparing predicted probability to observed proportion as well as numerical intercept and slope values. Decision analysis will be presented as a graphical decision curves of net benefit across threshold probabilities with each considered model(s) presented on a single graph in addition to the policies of treating all participants immediately and treating all participants routinely.

Model deployment
The developed machine learning model will be integrated into a clinical calculator that will be made available as an open-source tool. The tool will be developed using the python programming language and the streamlit library will allow us to build a browser-based interface quickly.

Recruitment
Recruitment status
Not yet recruiting
Date of first participant enrolment
Anticipated
2/10/2023
Actual
Date of last participant enrolment
Anticipated
1/11/2023
Actual
Date of last data collection
Anticipated
1/11/2023
Actual
Sample size
Target
1009
Accrual to date
Final
Recruitment in Australia
Recruitment state(s)
WA
Recruitment hospital [1] 23547 0
Sir Charles Gairdner Hospital - Nedlands
Recruitment postcode(s) [1] 38969 0
6009 - Nedlands

Funding & Sponsors
Funding source category [1] 314342 0
Hospital
Name [1] 314342 0
Sir Charles Gairdner Hospital
Country [1] 314342 0
Australia
Funding source category [2] 314343 0
Government body
Name [2] 314343 0
Australian Government Department of Education Research Training Program
Country [2] 314343 0
Australia
Funding source category [3] 314344 0
University
Name [3] 314344 0
The University of Western Australia
Country [3] 314344 0
Australia
Primary sponsor type
Hospital
Name
Department of Neurosurgery
Address
Department of Neurosurgery
G Block, Level 1
Sir Charles Gairdner Hospital
Hospital Avenue, Nedlands, 6009
Perth, Western Australia
Country
Australia
Secondary sponsor category [1] 316292 0
University
Name [1] 316292 0
The University of Western Australia
Address [1] 316292 0
35 Stirling Highway
Crawley, 6009
Perth, Western Australia
Country [1] 316292 0
Australia

Ethics approval
Ethics application status
Submitted, not yet approved
Ethics committee name [1] 313701 0
Sir Charles Gairdner Osborne Park Hospital Group Care Group Human Research Ethics Committee
Ethics committee address [1] 313701 0
Ethics committee country [1] 313701 0
Australia
Date submitted for ethics approval [1] 313701 0
28/08/2023
Approval date [1] 313701 0
Ethics approval number [1] 313701 0

Summary
Brief summary
Trial website
Trial related presentations / publications
Public notes

Contacts
Principal investigator
Name 122954 0
Dr Arosha Dissanayake
Address 122954 0
Department of NeurosurgeryLevel 1, G BlockSir Charles Gairdner HospitalHospital Avenue, Nedlands, 6009Perth, Western Australia
Country 122954 0
Australia
Phone 122954 0
+61 8 64577206
Fax 122954 0
Email 122954 0
[email protected]
Contact person for public queries
Name 122955 0
Arosha Dissanayake
Address 122955 0
Department of NeurosurgeryLevel 1, G BlockSir Charles Gairdner HospitalHospital Avenue, Nedlands, 6009Perth, Western Australia
Country 122955 0
Australia
Phone 122955 0
+61 8 64577206
Fax 122955 0
Email 122955 0
[email protected]
Contact person for scientific queries
Name 122956 0
Arosha Dissanayake
Address 122956 0
Department of NeurosurgeryLevel 1, G BlockSir Charles Gairdner HospitalHospital Avenue, Nedlands, 6009Perth, Western Australia
Country 122956 0
Australia
Phone 122956 0
+61 8 64577206
Fax 122956 0
Email 122956 0
[email protected]

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
Yes
What data in particular will be shared?
All de-identified individual participant data used in the formulation and internal validation of the model will be available indefinitely (i.e. with no end date)
When will data be available (start and end dates)?
From 1st of Jan 2024 onwards indefinitely (i.e. with no end date)
Available to whom?
To approved researchers in the field in response to a genuine request
Available for what types of analyses?
For re-analysis fo result verification purposes and for meta-analytic purposes
How or where can data be obtained?
Contacting the principal investigator via email ([email protected])


What supporting documents are/will be available?

No Supporting Document Provided



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
No additional documents have been identified.