Using artificial intelligence to predict post-operative outcomes in congenital heart surgeries: a systematic review

Abstract

Introduction

Congenital heart disease (CHD) represents the most common group of congenital anomalies, constitutes a significant contributor to the burden of non-communicable diseases, highlighting the critical need for improved risk assessment tools. Artificial intelligence (AI) holds promise in enhancing outcome predictions for congenital cardiac surgery. This study aims to systematically review the utilization of AI in predicting post-operative outcomes in this population.

Methods

Following PRISMA guidelines, a comprehensive search of Pubmed, Scopus, and Web of Science databases was conducted. Two independent reviewers screened articles based on predefined criteria. Included studies focused on AI models predicting various post-operative outcomes in congenital heart surgery.

Results

The review included 35 articles, primarily published within the last four years, indicating growing interest in AI applications. Models predominantly targeted mortality and survival (n = 16), prolonged length of hospital or ICU stay (n = 7), postoperative complications (n = 6), prolonged mechanical ventilatory support time (n = 4), with additional focus on specific outcomes such as peri-ventricular leucomalacia (n = 2) and malnutrition (n = 1). Performance metrics, such as area under the curve (AUC), ranged from 0.52 to 0.997. Notably, these AI models consistently outperformed traditional risk stratification categories. For instance, in assessing the risk of morbidity and mortality, the AI models demonstrated superior performance compared to conventional methods.

Conclusion

AI-driven prediction models show significant promise in improving outcome predictions for congenital heart surgery. They surpass traditional risk prediction tools not only in immediate postoperative risks but also in long-term outcomes such as 1-year survival and malnutrition. Further studies with robust external validation are necessary to assess the practical applicability of these models in clinical settings.

The protocol of this review was prospectively registered on PROSPERO (CRD42024550942).

Introduction

Congenital heart disease (CHD) is the most prevalent type of major congenital anomalies, constituting nearly one-third of such disorders [1]. Not long ago, only a small fraction of patients with moderate and severe CHDs reached adulthood. In the 1950s, however, the introduction of cardiopulmonary bypass significantly advanced the application of surgery as a treatment for congenital heart disease, resulting in marked improvement in long-term outcomes [2, 3]. Yet even with advancements in surgery, CHD plays a substantial role in the overall burden of non-communicable diseases (NCDs) [4], with postoperative complications in 46–74% of all operated patients [5, 6]. Prediction of these outcomes can be critical in operative and post-operative decision-making.

To predict postoperative outcomes in CHD patients, such as morbidity and mortality, several risk stratification categories have been introduced, including Risk Adjustment in Congenital Heart Surgery-1 (RACHS-1) [7] and Society of Thoracic Surgery-European Association for Cardiothoracic Surgery Congenital Heart Surgery Mortality Categories (STS-EACTS). However, it has been demonstrated that only the risk stratifications that depend predominantly on expert opinion and consensus, underperform in comparison to those that are more evidence-based [8]. Therefore, there is a need for new prediction models based on big data. [9]. Linear logistic regression models have shown promise in the past [10], yet the advent of artificial intelligence (AI) and the development of more complex AI models has shown great applicability [11].

In recent years, AI has emerged as a novel promising approach to data science [12]. With regard to AI prediction models, the most distinctive factor is that these models learn from examples rather than being programmed by rules as in traditional predicting models [13]. This makes these AI models non-linear complex tools well-suited for identifying and illustrating patterns that are either unknown to or too complex for traditional biostatistics [14, 15]. Moreover, AI models can distinguish variables most impactful in a particular process, allow for a greater number of predictive variables to be incorporated into a model, and learn important features by training on original datasets [16]. As the diverse array of AI predictive modeling techniques and the utility of each may seem confounding, Table 1 showcases models that are most frequently utilized by reports in the literature in order to provide better insight [17,18,19,20,21]. Numerous AI models have been designed to predict outcomes following various types of surgeries, including cardiac surgeries [22].

Table 1 A brief description of the most frequently used AI modeling techniques and advantages and disadvantages

The aim of the current study was to systematically review the relevant literature and identify all studies that have utilized AI to predict post-operative outcomes in patients undergoing surgery for congenital heart defects.

Methods

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [23]. The protocol of this review was prospectively registered on PROSPERO (CRD42024550942).

Literature search strategy

A comprehensive search was conducted on May 4th, 2024 in 3 electronic databases, Pubmed, Web of Science, and Scopus by one author (IM). No language or publication limitations were applied. The search results were then passed on to S.R. and M.H. for further evaluation.

The search terms used included combinations of 3 separate search parameters, combined using Boolean operators:

Parameter a: ("artificial intelligence", "deep learning", and "machine learning").

Parameter b: ("congenital heart defect(s)", "tetralogy of Fallot", "atrial septal defect", "ventricular septal defect", "patent ductus arteriosus", and "patent foramen ovale").

Parameter c: ("surgery", "operation", "heart", and "cardiac").

Screening and eligibility criteria

Screening was carried out in 2 phases, with an initial title-abstract screening followed by a full-text evaluation, by two independent authors (S.R., M.H.), with discrepancies being resolved through discussion with a third reviewer (I.M.). Papers employing an AI model to predict a post-operative outcome after congenital heart surgery were included based on the PICOS (population, intervention, comparator, outcome, and study design (PICOS) criteria defined below. Artificial intelligence predictive models were identified according to the definitions provided by Jovel et al. [24].

Population: Patients undergoing congenital heart surgery.

Intervention: AI models and algorithms.

Comparator: Current algorithms used to predict outcomes, if available.

Outcome: Postoperative outcomes.

Study design: AI development and/or validation studies.

Papers that were conducted on animals were excluded. Study designs including reviews, meta-analyses, case reports, preclinical studies, editorials, book chapters, and conference abstracts were also excluded. In addition, due to ambiguity in defining what exactly constitutes a machine learning model, some consider LR a traditional statistical model, while others refer to it as a machine learning one [17,18,19,20,21]. For the purposes of this review, we considered LR to be an AI model and thus included predictive models employing it.

Outcomes

The intention of the reviewers was to review all post-operative outcomes examined in the literature. After the completion of full-text screening, we ended up reviewing the following outcomes: prediction of risk in congenital heart surgery based on RACHS (risk adjustment for congenital heart surgery) classification, critical events, morbidities, and complications such as prolonged length of hospital stay and prolonged mechanical ventilatory support time, in-hospital and 30-day mortality, periventricular leukomalacia (PVL), one-year transplant-free survival after Norwood procedure, pulmonary arterial hypertension, echocardiography variables related to the right atrium, postoperative pulmonary vein obstruction, interstage mortality between stage I and II of the Norwood procedure while at home, and malnutrition.

Data extraction and risk of bias assessment

A data extraction template was prepared beforehand which consisted of the following data: first author, year of publication, country, source of data, type of source, type of study, population size, population age, gender, whether the patients are adult or pediatric, type of defect, algorithm/model, NO. of variables, outcomes, mode of validation, calibration, missing data strategy, measures of performance of each outcome, the best algorithm, and main findings. The template was independently filled with the required qualitative and quantitative data by two authors (SH.R., M.H.), with a third author (I.M.) re-evaluating the extracted data and correcting any disagreements.

The risk of bias in the selected articles was assessed using the PROBAST tool [25] by two independent reviewers (SH.R., M.H.) with a third reviewer (I.M.) resolving any variations through discussion. This tool consists of 4 domains for bias detection, Participants, Predictors, Outcome, and Analysis, with a total of 20 criteria. Each domain can be graded with a low risk, a high risk, or an unclear risk of bias. Studies with a low risk of bias in all four domains were classified as low-risk while having a high risk of bias in even one domain would result in judging the study as high-risk. If one or more domains had an unclear risk of bias with the rest being low-risk, the risk of bias attributed to the study would then be declared unclear.

World map was designed using https://www.mapchart.net/world.html.

Results

Study selection

A total of 335 articles were identified in the literature search. There were 90 duplicates, the removal of which yielded 245 references. A further 188 references were excluded in the initial phase of screening based on the title and the abstract, and 57 articles were sought for full-text retrieval. During the second phase of screening 22 studies were excluded due to the targeted population not being that of patients undergoing congenital heart surgeries (n = 8), the intervention not being AI-based (n = 4), or the assessed endpoint not being a postoperative outcome (n = 10), resulting in the inclusion of 35 articles in the database search. (Fig. 1).

Fig. 1

PRISMA flowchart detailing the search and screening process, made using the Shinyapp by Haddaway et al. [26]

Study characteristics

The characteristics of the included population in each study are outlined in Table 2. The included studies were published from 2013 to 2023 and mostly from United States (Fig. 2). The source of data in the majority of studies were the local electronic medical records (EMR). The included studies trained and tested their models in between 56 and 221,335 individuals, mostly pediatric patients with no particular focus on the type of congenital heart disease. Among the studies with a focus on specific types of defects, single ventricle physiology and tetralogy of Fallot (TOF) were the most frequently investigated (Fig. 2).

Table 2 Characteristics of the populations included in the studies

Fig. 2

Graphical abstract: a geographic distribution of the studies; b freaquancy of the machine learning algorthims for each congenital heart disease; c application and accuracy of machine learning algorithms in congenital heart surgery. Abbreviations: Boosting (B), Congential heart surgery (CHS), Single ventricle physiology (SVP), Aortic reconstruction (AR), Total anomalous pulmonary venous connection (TAPVC), Light gradient boosting (LGB), Extreme gradient boosting (EGB), CatBoost (CB), Transplant free survival (TFS), Mechanical ventilator support (MVS), Length of hospital stay (LoHS), Length of intensive care stay (LoICU), Acute kidney injury (AKI), artificial neural network (ANN), Neural network (NN), Deep neural network (DNN), Gradient boosting (GB), Multilayer Perceptron (MLP), Naive Bayes (NB), Bag Decision Trees (BDT), Deep venous thrombosis (DVT), Low cardiac output syndrome (LCOS), Support vector machine (SVM), Convolutional neural network (CNN), Random forest (RF), Decision tree (DT), K-nearest neighbor (KNN), Logistic regression (LR)

The details regarding the AI prediction models constructed in each study are displayed in Table 3. The most commonly utilized algorithm was logistic regression (LR, n = 16) followed by boosting (B, n = 14), random forests (RF, n = 9), support vector machine (SVM, n = 8), decision trees and artificial neural networks (DT and ANN, n = 7 each), k-nearest neighbor and naïve bayes (KNN and NB, n = 4 each) (Fig. 2). In addition, the most frequently assessed outcome was short-term mortality with 11 studies either reporting in-hospital or 30-day mortality. All of the included studies were retrospective in design. Moreover, most of the studies (n = 15) utilized K-fold cross-validation while 14 used training and testing with or without bootstrapping as their mode of validation. Only 6 studies externally validated their models.

Table 3 The characteristics of AI prediction models

AI Models and Measures of Performance

Overall AUCs above 0.7 were achieved in at least one model in all studies which reported their respective AUC, with the exception of one study by Sunthankar et al. [52]. Short-term mortality was the most investigated outcome, and different AI models across four different articles were used to measure it (Fig. 3). The best performance among the models predicting this outcome was achieved by logistic regression (LR) with an AUC of 92.6% as reported by Zürn et al. [54], while the worst-performing model was also an LR model with an AUC of 72%, as reported by Cocomello et al. [36].

Fig. 3

AUCs of different models investigating short term mortality. Algorithm abbreviations: random forest (RF), Multilayer perception (MLP), Logistic regression (LR), Bagged decision trees (BDT), Extra trees (ET), Ada boost classification (ABC), Stochastic gradient boosting (SGB), Extreme gradient boosting (XGBoost), Optimal classification trees (OCT), Gradient boosting (GB), Adaptive boosting (ADA)

Risk of bias assessment

Overall, ten, fourteen, and eleven studies had high, low, and unclear risk of bias, respectively. The most common domain of bias among the studies was domain 4 (analysis), while the domain imposing the least amount of bias was domain 3 (outcome). Overall, 60% of the included studies had a high or unclear risk of bias, highlighting the need for more detailed quality control in this field (Fig. 4).

Fig. 4

Quality assessment of the studies using PROBAST. Domains: D1: Participants/ D2: Predictors/ D3: Outcome/ D4: analysis

Discussion

This systematic review explores the application of AI (deep learning and machine learning) in the prediction of outcomes in congenital cardiac surgery. Overall 35 articles were included, most of which were published in the past four years, showing an increased interest in the use of AI. A meta-analysis to assess and compare the performance of the models was not possible due to missing data, yet we can still observe the robust predictive performance exhibited by different artificial intelligence models, regardless of the outcomes they were designed to predict.

The models included focused on the prediction of mortality and survival (n = 16) [27, 29, 31, 33, 37, 40, 46,47,48, 52,53,54,55, 60], prolonged length of hospital or ICU stay (n = 7) [29, 40, 43, 46, 47, 57, 58], postoperative complications (n = 6) [34, 35, 41, 42, 44, 59] prolonged mechanical ventilatory support time (n = 4) [40, 47, 57, 59], with additional focus on specific outcomes such as periventricular leucomalacia [32, 39], acute kidney injury [56], malnutrition [50]. The AUCs of the models ranged between 0.52 to 0.997, with most models achieving an AUC above 0.7, highlighting the predictive potential of artificial intelligence in congenital heart surgery. Details for each assessed outcome are laid out below:

Discrimination evaluated by area under the curve was above 0.8 in almost all of the models evaluating mortality, with the exception of one study by Sunthankar et al. [52] which predicted mortality between stages I and II of single ventricle surgery. Other mortality-predicting models focused on survival during the hospital stay and up to 1-year post-operation. The sensitivity of the models ranged from 0.1 in the LR model by Jalali et al. [46] to 0.92 in the random forest model by Chang Junior et al. [37]. The specificity of the predictions ranged from 0.542 in the gradient-boosted tree model by Sunthankar et al. [52] to 0.985 from the multilayer perception model by Chang Junior et al. [37]. The number of predictors used in each prediction model varied greatly yet some were of high predictive value throughout the different models; the type of procedures [27, 31, 40, 47, 52], age [27, 31, 33, 40, 48, 52], weight at the time of surgery [27, 31, 33, 40, 47, 53], arterial oxygen saturation [37, 48], and days since the previous hospitalization [40, 47] were all among these, with the type of procedure contributing more than 45% of predictive value in two instances [40, 47].

The type of operative procedure itself was a significant predictor in some models, with the Norwood procedure being the most predictive of mortality in some studies [27, 31, 47, 52]. This is in accordance with relevant literature stating the higher mortality rate of this procedure in comparison to most other procedures [9, 62]. Crowe et al.’s analysis clearly demonstrated that the Norwood procedure has the highest 30-day mortality rate among congenital heart surgeries, with a statistically significant difference compared to the second-highest rate, observed in interrupted aortic arch repair [27]. Sunthankar et al. developed a model aimed at predicting mortality after the Norwood procedure in single ventricle physiology and found the Norwood/Sano conduit as a significant predictor of mortality, yet deemed the hybrid Norwood method of less importance [52]. Another study also aimed to predict mortality after the Norwood procedure; however, it did not use the type of Norwood procedure as a predictor, yet still managed to obtain an accuracy of 88% for the prediction of mortality [33]. It should also be noted the extreme gradient boosting machine model developed by Du et al. _which included a large variety of procedures_ did not attribute a significant predictive value to the type of procedure [48]. Additionally, STS-EACTS scores _ estimates of the mortality risk associated with each type of congenital cardiac procedure_were also found to be highly predictive of mortality by some studies [53, 54]. Whether the type of procedure can be used to correctly predict mortality after congenital heart surgery requires more extensive investigations, yet it seems to be promising in the few instances where it has predictive value.

The application of age in the predictive models was rather heterogeneous, with some finding gestational age [52] and age at first extubation [52] as a significant predictor, while others found the age at the time of surgery to be an important predictor [27, 31, 40, 48, 53]. One study by Chang Junior et al., however, found a maximum of 2.7% predictive importance for age at the time of the procedure [37]. This factor similar to the previous one is also in accordance with the literature as a large study found increasing levels of age lower the likelihood of mortality [63]. The utilization of this predictor also shows promise.

Weight at the time of surgery was highly predictive in some instances [27, 31, 40, 47, 53], while in a model by Chang Junior et al. [37] showed rather diminished predictive value. This model however found height and BMI to be of high importance in their algorithms, suggesting that weight on its own may not be a reliable predictor and lacks generalizability. The value of weight as a predictor, much like the previous factors, is also evident in the literature [64].

The diagnostic group was also identified to be an important predictor of mortality by some studies [27, 31, 37]. Among the anomalies, univentricular heart status [27, 31] and by extension, hypoplastic left heart syndrome (HLHS) were found to be important predictors of mortality. Crowe et al. [27] classified patients with HLHS and pulmonary atresia with an intact ventricular septum as high risk groups for mortality, which was also corroborated by Rogers et al. [31]. Both studies also found that univentricular heart status is a separate indicator of 30-day mortality. In addition, Chang Jr. et. al. not only found that diagnostic groups are a significant predictor, they identified HLHS status as a separate significat predictor as well [37]. This finding is in alignment with the previous literature, which has often found univentricular heart anomalies to be the most fatal among congenital heart anomalis [65, 66].

Patient height at surgery [37], previous ICU admission [37], gender [33, 53], usage of cardiopulmonary bypass [27, 31], Down syndrome status [27], presense of congenital or acquired comorbidities [31], severity of illness [31], digoxin use at discharge [52], atrial shunt [48], and cardiopulmonary bypass time [52] were also notable predictors among the models for mortality prediction, yet these findings were only exclusive to one model or two models and were not found of considerable value in the other included models.

All in all, we observed an exceptional performance among the AI models predicting mortality in patients undergoing congenital heart surgeries; however, more studies need to be conducted in this area in order to perform quantitative synthesis and objectively measure their performance.

The area under the curve for studies predicting cardiac complications was more heterogeneous than mortality, with a range of 0.71 to 0.95. We observed that models tasked with predicting a broader spectrum of complications rather than a specific cardiac complication had a lower AUC. Furthermore, the sensitivity of predictions ranged from 0.34 in an LR model to 0.791 in an extreme gradient boosting model, both by Zeng et al. [44] Specificity of the models predicting complications was not reported. As for the predictors used in cardiac complications, birth weight [49], weight at surgery [44], gestational age [49], age at surgery [34, 49], cardio-pulmonary bypass time [41, 44], ECG variables [42], heart rate [35, 42], echocardiographic variables [34, 41, 49] and post-surgery oxygen pressure [41, 44, 49] were all important predictors. Echocardiographic variables, such as left ventricular ejection fraction [34], right ventricular global strain [41], and right pulmonary artery Z score [49], have been shown to correctly foretell cardiac complications [67] and due to the ubiquitous and simple nature of echocardiography, these variables can be of immense value in predicting cardiac complications.

Another intriguing aspect with regard to cardiac morbidity and mortality is the notable superiority of AI to traditional prediction models. Zeng et al. demonstrated that the model they produced using extreme gradient boosting achieved a higher AUC compared to univariable logistic regression models of Aristotle’s Basic Complexity (ABC) score, STS-EACTS mortality and morbidity score, and RACHS-1 [44], indicating the potential superiority of an AI model to the tools currently available to clinicians. In addition, a systematic review assessing the ability of these risk stratification systems to predict morbidity and mortality found that the AUC of ABC ranges from 0.59 to 0.743, the AUC of RACHS-1 ranges from 0.68 to 0.782, and the AUC of STS-EACTS score from 0.732 to 0.8 [68]. Given the fact that AI models consistently demonstrate AUCs higher than the numbers stated, adopting them would likely assist surgeons in better risk stratification and decision-making. This is notable as risk stratification tools play a crucial part in reducing the morbidity and mortality of patients undergoing congenital heart surgery [68]. More investigations are needed to compare and validate AI prediction models against traditional ones. Furthermore, to implement these models clinically, traditional machine learning models should be optimized with as few features as possible in order to require lower computational power [69], and models should be trained and externally validated on larger sets of data to maximize the predictive ability before commercialization.

The area under the curve for length of stay predictive models was even more heterogeneous than the previous outcomes, ranging between 0.54 to 0.95. Sensitivity also ranged from 0.32 in an LR model to 0.91 in a deep neural network, both developed by Jalali et al. [46]. The specificity of the models predicting length of stay was not reported. Variables associated with increased length of stay were unfortunately not provided in detail by the included models, yet Bertsimas et al. [40] found that the type of procedure, days since previous admission, weight, and age were all significant predictors of prolonged length of hospital stay. Moreover, Chang Jr. et al. found that mechanical ventilation time, weight, previous ICU admission, vasoactive-inotropic score, and height were significant predictors of prolonged length of IC stay [58].

This pattern of AUC appears to show artificial intelligence predictive models are more geared toward predicting singular binary outcomes, yet newer machine learning algorithms perform better than logistic regression models in this regard. This is evident in the few instances they were compared in the included studies [40, 44, 46, 52].

While most of the included articles did not focus on a specific CHD, outcomes in patients with single ventricle physiology (n = 8) and TOF (n = 4) were the most investigated. The management of patients with single-ventricle physiology is an intricate process, as not only do single-ventricle defects vary greatly in anatomical subtypes, but surgical palliation is undertaken in multiple stages [70]. As such, precise prediction models for post-operative outcomes in these patients are invaluable. AI models can predict postoperative complications [35, 42] and 1-year mortality [33, 46] with great accuracy. These results could not be replicated when predicting interstage mortality_i.e. mortality occurring between the first and second stages of the Norwood procedure_ as the AUC of neither the mortality-predicting models of Sunthankar et al. nor Smith et al.’s survival-predicting model surpassed 70% [52, 60]. Among the studies in patients with TOF, the reports were more divergent with respect to the measured outcome, yet complications, mortality, and indices of heart function have been successfully predicted by AI models [29, 34, 41, 49].

Our systematic review has several limitations. First, by the nature of congenital heart defects, the patient populations used to develop the models are quite heterogeneous. As a result, unless the models are specialized in one specific type of defect, very large sample sizes are needed to develop reliable AI models, a condition that was not met by some of the included studies. In addition, mainly due to the heterogeneous and incomplete reporting of the analyses conducted to develop the AI models, the quality of most of the included studies was either low or unclear. We suggest that future studies report the steps of model development thoroughly to avoid this issue. External validation _ i.e. evaluating the developed model on a separate original data set different than the one used to train and test the model_ was not used in most of the included studies in our review. Doing so is important to assess the realistic applicability of predictive models [71] and without this process, we cannot say how practical these models can be in the healthcare system, therefore more studies are needed to validate our findings. Another important point that was ignored by most of the included reports was first ensuring that the model was calibrated and second, reporting quantitative measures of calibration of the model in addition to plots. In addition, the high rate of heterogeneity among methods, outcomes, and participants did not allow us to perform a meta-analysis on any of the outcomes.

Conclusion

The implementation of artificial intelligence in congenital heart surgery has resulted in the creation of many predictive models that have the potential to change the landscape of clinical outcome prediction. When assessing the risk of postoperative morbidity and mortality, these AI-generated predictive models demonstrate superior performance compared to traditional risk prediction models. Their applicability, however, is not limited to the immediate risk of complications after surgery and encompasses long-term outcomes such as 1-year mortality and malnutrition. This systematic review provides a comprehensive report on such models and can serve as a bedrock to conduct future studies with more insight into the field.