ECG-based heart arrhythmia classification using feature engineering and a hybrid stacked machine learning

Abstract

A heart arrhythmia refers to a set of conditions characterized by irregular heart- beats, with an increasing mortality rate in recent years. Regular monitoring is essential for effective management, as early detection and timely treatment greatly improve survival outcomes. The electrocardiogram (ECG) remains the standard method for detecting arrhythmias, traditionally analyzed by cardiolo- gists and clinical experts. However, the incorporation of automated technology and computer-assisted systems offers substantial support in the accurate diagno- sis of heart arrhythmias. This research focused on developing a hybrid model with stack classifiers, which are state-of-the-art ensemble machine-learning techniques to accurately classify heart arrhythmias from ECG signals, eliminating the need for extensive human intervention. Other conventional machine-learning, bagging, and boosting ensemble algorithms were also explored along with the proposed stack classifiers. The classifiers were trained with a different number of features (50, 65, 80, 95) selected by feature engineering techniques (PCA, Chi-Square, RFE) from a dataset as the most important ones. As an outcome, the stack clas- sifier with XGBoost as the meta-classifier, trained with 65 important features determined by the Principal Component Analysis (PCA) technique, achieved the best performance among all the models. The proposed classifier achieved a perfor- mance of 99.58% accuracy, 99.57% precision, 99.58% recall, and 99.57% f1-score and can be promising for arrhythmia diagnosis.

Background

A heartbeat is a periodic relaxation and contraction of the heart muscle that drives blood through the circulatory system [1, 2]. In a healthy heart, impulses follow a regu- lar and coordinated pattern, often referred to as a sinus rhythm [3]. Heart arrhythmia is a common cardiac condition that describes any abnormal heart rhythm. It occurs when the electrical impulses that regulate the heartbeat go awry, causing the heart to beat quickly, slowly, or irregularly. Arrhythmia can happen independently or with other cardiovascular conditions [4]. Although some arrhythmias are not dangerous, some have the potential to cause abrupt cardiac arrest, heart failure, stroke, and other cardiovascular diseases (CVDs). An estimated 17.7 million people died because of CVDs in 2017, which accounts for 31% of all deaths [5]. An essential tool in diagnos- ing arrhythmia is the electrocardiogram (ECG) besides other biointegrated wearable and implantable optoelectronic devices [6,7,8,9]. ECG is a crucial medical equipment that captures the heart’s excitability, transmission, and recovery [10]. The result of an ECG is a signal representation corresponding to the heart’s electrical activity. Physi- cians inspect the pattern of the signals to identify any arrhythmias. With the advent of artificial intelligence and machine learning [11], researchers have been trying their best to incorporate machine learning in classifying arrhythmia in ECG signals.

ML techniques have been used in multidisciplinary fields for prediction purposes that include health informatics [12, 13], disaster forecasting [14], agriculture [15], monitoring systems [16], and so on. Similarly, several ML [17] and Deep learning (DL) [18, 19] techniques have been applied to classify heart arrhythmia. However, there is always room for improvement. Initially, ML algorithms were used to carry out such classification tasks. Melgani et al. [20] demonstrated the SVM algorithm’s capacity to generalize the classification of ECG beats. They used Particle Swarm Opti- mization (PSO) to boost the SVM classifier’s performance in terms of generalization (accuracy = 89.72%). Kumar et al. [21] described a beat-to-beat interval-based ECG classification approach for arrhythmic beats. The beat-to-beat intervals were extracted from the ECG signals and converted into Discrete Cosine Transform (DCT) as part of the methodology. Then, the transformed beats were classified using the Random For- est algorithm (accuracy = 92.16%). Park et al. [22] created a system that uses features like P wave and QRS complex for detecting heartbeats and the k-nearest neighbor (KNN) algorithm for classifying them (97.22% sensitivity and 97.4% specificity for heartbeat detection, 97.1% sensitivity and 96.9% specificity for classification). Ardeti et al. [23] utilized an improved filtering method to identify the extreme outliers of the signal for ranking features. A heterogeneous classification model based on an Opti- mized Random Forest (ORF) was also presented to increase the true positive of the ECG data. The majority voting technique was used to classify each type of heartbeat (accuracy = 96.17%).

Eventually, deep learning techniques have evolved, and studies now focus more on these newer techniques. Ubeyli [24] integrated recurrent neural networks (RNN) and eigenvector techniques to extract features and classify ECG beats based on the extracted features. Guler and Ubeyli utilized feedforward neural Networks (FFNN) [25] to classify ECG beats (accuracy = 96.94%). Li et al. [26]. suggested a general model based on ResNet to achieve the automated classification of regular rhythm. The 12- lead ECG signal was cut into a two-dimensional plane and rendered like a grayscale image. The intrinsic features of the two-dimensional ECG were extracted using DSE- ResNet. Furthermore, the DSE-ResNet’s hyper-parameters were optimized using an orthogonal experiment approach, and classification performance was increased using a multi-model voting strategy (test f1-score = 81.7%). For automatic arrhythmia clas- sification, Ramkumar et al. [27] proposed a combination of autoencoder (AE) and Bi-LSTM. An encoder in the AE-biLSTM approach extracts higher-level features. The decoder output reconstructs ECG signals using bi-LSTM, and heartbeats are finally categorized (accuracy = 97.15%). Madan et al. [28] suggested a deep learning technique that combined 2D Convolutional Neural Network (2D-CNN) and Long Short Term Memory (LSTM) to automate the detection and classification process. 2D Scalogram pictures were created from 1D ECG data for noise reduction and feature extraction. After obtaining experimental data, the proposed model was designed, which got 98.7% accuracy.

The stacking ensemble method, or the stack classifier, is a noteworthy state-of-the- art process that integrates the predictions of more than one base model to arrive at the final prediction. It is an ensemble technique that intends to acquire the capabilities of different models and improve the final performance [29]. In a stack classifier, the meta- learner model is used to aggregate the output of various base models that have been trained on the same dataset. The meta-learner learns to provide the final predictions using the underlying models’ predictions as input [30]. This architecture sets stack classifiers apart from single models or traditional ensemble methods like bagging and boosting. Though the stack classifier is found to perform better than other individual techniques in predictive tasks, this state-of-the-art approach is still uncommon in classifying heart arrhythmia. Again, only a few studies explored the optimal number of features required to properly classify heart arrhythmia applying numerous feature engineering techniques.

Therefore, the objectives of this research include exploring the performances of conventional ML and ensemble techniques for classifying heart arrhythmia from ECG signals and proposing a stacking classifier that employs an optimal number of features for classifying heart arrhythmia more effectively. Therefore, the key contributions of this research are as follows:

• A stack classifier that is trained and tested with five conventional ML models (Sup- port Vector Machine (SVM), K-Nearest Neighbours (KNN), Logistic Regression (LR), Decision Tree (DT), Multi-Layer Perceptron (MLP)) as weak learners and one model as meta learner to classify the dataset. The meta learner has been selected from the five different kinds of bagging or boosting classifiers, namely Random Forest (RF), Adaboost (AB), Gradient Boosting (GB), eXtreme Gradient Boost- ing (XGB), and Categorical Boosting (CB). Each developed stack classifier is then evaluated to determine the best-performing classifier.

• For determining the number of optimal features, three feature engineering tech- niques, namely the Chi-Square Test, Principal Component Analysis, and Recursive Feature Elimination, have been applied. Each of the techniques selects different sets of features from the dataset by their method of feature prioritization. These features were then used by the ML methods to classify heart arrhythmia.

• To validate the performance of the proposed stack classifiers, each of the conven-tional, bagging, and boosting ML models were trained separately using the same dataset. A comparison was carried out using different performance parameters.

The remainder of this paper is divided as follows: Sect. 2 describes the methodology in detail. Section 3 presents the research results. Section 4 presents the discussions and concludes the paper.

Methodology

We divide our whole methodology into 5 phases, namely data collection, data pre- processing, feature engineering, model development, and performance analysis. The methodological overview of this systematic approach is depicted in Fig. 1. The whole workflow is described in the following subsections:

Fig. 1

The methodological overview

Data collection

At first, necessary data was collected. The dataset used was the MIT-BIH arrhythmia database [31] taken from PhysioNet [32]. The MIT-BIH dataset contained 2-channel ambulatory ECG recordings of 48 half-hour snippets utilized from 47 patients in Beth Israel Hospital. The participants included 25 men ranging from 32 to 89 years and 2 women ranging from 23 to 89 years. The participants had a mixed population of 60% inpatients and 40% outpatients. The recordings were digitized over a ten mV range at 360 samples per second per channel. More than two cardiologists independently annotated the records. Finally, around 110,000 annotations were obtained, each having a heartbeat.

The dataset was processed by Kachuee et al. [33]. They converted each of the annotations into a matrix form. Each row of the matrix represents one heartbeat and has 188 columns. The first 187 columns indicate the amplitude of the heartbeat at different time instances. The final column represents the class of the heartbeat. This dataset was used in this study to train the models. There were a total of 5 classes in the dataset.

The train and test data were already split in the dataset. There were 87,554 heart- beats in the train data, 72,471 of which were classified as “Normal” heartbeats (Fig. 2a). The remaining heartbeats belonged to one of the four classes of arrhythmia. 2,223 heartbeats were “Supraventricular heartbeats” [34] (Fig. 2b), 5,788 heart- beats were “Ventricular heartbeats” [35] (Fig. 2 (c)), 641 were “Fusion heartbeats” (Fig. 2d) and the rest, 6,431 heartbeats, did not fall into any of the other four classes, and so were considered as “Mixed heartbeats” (Fig. 2e). The test dataset had 21,892 heartbeats, with 18,118 normal, 556 supraventricular, 1,448 ventricular, 162 fusion, and 1,608 mixed heartbeats. Sample images of the five classes of heartbeats available in the dataset are shown in Fig. 2.

Fig. 2

Classes of Heartbeat: (a) Normal (b) Supraventricular (c) Ventricular (d) Fusion (e) Mixed

Data synthesis

This phase addressed several data-related challenges to enhance the training process. To begin with, the issue of class imbalance within the dataset was tackled. After this, gaussian noise was added to the dataset to make the instances more robust to noises. Then, the number of important features was determined followed by feature engineering applying 3 different methods. An algorithm comprising the whole data preprocessing phase is given in Algorithm 1.

Algorithm 1 Algorithm for data preprocessing

Class balancing

Considering that the “normal” class boasted the highest number of instances, totaling 72,471, the Synthetic Minority Oversampling Technique (SMOTE) method [36] was implemented. The SMOTE method generates some synthetic instances of the classes with a lower number of samples to reduce the class imbalance. The formula for the SMOTE method is shown in Eq. 1.

$$\times_{sample}=\times+\eta\left(\times_{random}-\times\right)$$

(1)

Here, x_sample refers to the generated samples of minority classes x. Whereas, x_random refers to a value chosen randomly from the nearest neighbors of x with 0 ≤ x ≤ 1. This technique augments the instances in all classes to a consistent count of 72,500, resulting in a massive training dataset of 290,000 instances. The algorithm followed for oversampling with SMOTE is given in Algorithm 2.

Algorithm 2 Algorithm for oversampling

ECG signals are susceptible to various types of noise, including interference from external electrical devices and signal degradation due to electrode distance [37]. Gaus- sian noise was introduced to the dataset to bolster the model’s resilience against noise and enhance its ability to generalize effectively to unseen data. The equation to add Gaussian noise to the data is shown in Eq. 2.

$$\times_{noisy}=\times_{original}+N\left(0,\;0.5\right)$$

(2)

x_noisy refers to the generated noisy samples from the original samples x_original with the addition of the random variable N(0, 0.5). The random variable N was sampled using the Gaussian distribution of mean 0 and standard deviation 0.5. The impact of this noise addition on signal characteristics is visualized in Fig. 3.

Fig. 3

ECG signal before and after adding Gaussian noise

Likewise, for the test data, SMOTE harmonized the instance counts across all classes to achieve a uniform count of 20,000 instances, totaling 100,000 instances. The same Gaussian noise was added to this dataset with a mean distribution of 0 and a standard deviation of 0.5, ensuring consistency in noise robustness across the training and testing phases.

Feature engineering

Feature engineering is the process of selecting the features with the most impor- tant attributes and eliminating less important features from a dataset to increase the predictive performance of machine learning methods [38]. It is a method of finding out the best subset of features necessary to train a prediction model with superior performance. The dataset used in this research has a total of 187 features for each data instance. Training ML models with this enormous number of features is time- consuming, difficult, and a likely chance of curse of dimensionality [39]. Again, not all the features are necessary to develop a proper model. Therefore, it is necessary to find the right number of features that, when used to train ML models, bring out the best performance in the model with fewer complexities.

At first, the Ordinary Least Square (OLS) regression method [40] was used to determine the number of dataset features that held significant importance. The ran- dom forest classifier determines the feature importance based on the pureness of its leaf nodes. The purity of the leaf nodes is 100% if all the nodes point to one class. Otherwise, it is impure. The feature that shows more purity, has more importance. It is noticeable from Fig. 4 that the cumulative variances in data become stable at approximately 80 features. After that, the variance change is negligible. This infers that approximately 80 along the total 187 features hold more importance in determin- ing a class of heartbeat. For a more rigorous approach, we decide on a fixed number of different feature size closer to 80 (50, 65, 80, and 95) to train the models. Now, the 3 feature engineering techniques were implemented on the dataset. For each tech- nique, the most significant 50, 65, 80, and 95 features were selected, and the dataset with the selected features was then used to train each of the models separately. The 3 techniques are briefly described as follows:

Fig. 4

Diagrammatic view of the number of features significant to the dataset

Chi-Square test

The Chi-Square Test (CST) is one of the most useful feature engineering techniques in the field of ML [41]. It carries out a statistical evaluation where deviation is calculated from the predicted distribution when the feature event is independent of the class value and feature priority is determined by observing the relationships between them [42]. The formula of CST is shown in Eq. 3. In the equation, the observed values are the total real observations that fit a particular feature i, and the expected values are the total observations that are expected to occur. The prioritized features are selected based on the best scores of χ². For selecting the k best features, the python SelectKBest function was applied with k = n, where n is the total number of features.

$$\times^2={\textstyle\sum_{i=1}^n}\frac{\left(Observed\;Value_i-Expected\;Value_i\right)^2}{Expected\;Value_i}$$

(3)

Principal component analysis (PCA)

Principal Component Analysis (PCA) is a dimension reduction tool that prioritizes features by observing the correlation between characteristics to determine the most important features or components [43]. PCA maps the original n-dimensional con- structs into a k-dimensional construct where k < n [44]. These k features are new principal attributes that reduce the curse of dimensionality.

Recursive feature elimination (RFE)

Recursive Feature Elimination (RFE) is a wrapper technique used for removing fea- tures from training data by ranking them in the order of importance and eliminating the low-ranked features [45]. This is a recursive method that applies various ML models and determines feature importance at every iteration by removing the least important ones.

The pseudocode of feature engineering is shown in Algorithm 3.

Algorithm 3 Pseudocode for feature engineering

Development of models

In this phase, the models were developed with a Python tool called Scikit-learn [46] in the Kaggle platform. The development of models was carried out in 4 phases or techniques as seen in Fig. 1. The descriptions of the 4 techniques are given as follows:

Conventional ML model development

Technique 1 applies 5 conventional ML algorithms that have been extensively used in health informatics. The algorithms are SVM, KNN, Logistic Regression, Decision Tree, and MLP. These algorithms follow some fundamental structures that are used to carry out predictive tasks.

Bagging classifier development

Technique 2 develops a bagging classifier for prediction. A bagging classifier is an ensemble technique that integrates more than one base model on a random subset of the dataset with equal weights provided to each model and decides on a final result based on the individual predictions [47]. The bagging classifier used in this research is the random forest classifier. The Random Forest aggregates the results of several decision trees and reaches the final decision.

Boosting classifier development

Technique 3 develops a boosting classifier for prediction. A boosting classifier is an ensemble technique that combines a group of weak learners into a strong learner by reducing the error of the weak learners [48]. In this research, 4 boosting classifiers were developed namely Adaboost (AB), Gradient Boosting (GB), eXtreme Gradient Boosting (XGB), and Categorical Boosting (CB).

Proposed stack classifiers development

Ensemble learning uses many classifiers to obtain better forecasting accuracy than a single classifier; where the method known as stacking ensemble learning combines multiple weak classifiers using a meta-classifier. In this method, each of the classifiers in the first level receives the data samples as input. If the dataset has a dimension of r x c, then each classifier in the first level receives data of r x c dimensions. Then, each classifier provides its predictions. These predictions of the first level, along with the true values, are used as features in the classifier in the second level. If there are n classifiers in the first level, then the classifier in the second level will receive a dataset of r x (n + 1). Lastly, the prediction of the final classifier is considered as the final result [49]. An illustration of the stack classifier mechanism is given in Fig. 5.

Fig. 5

Mechanism of stack classifier and the architecture of the proposed classifier

The proposed method is a multi-layered stack architecture where the dataset is preprocessed and then sent to base learners at level 0. In level 0, the 5 conventional ML algorithms have been kept which were used in technique 1. They are SVM, KNN, Logistic Regression, Decision Tree, and MLP. Each base model learns from the dataset independently applying its prediction method. Each base model predicts outputs which are denoted by P1, P2, P3, P4, and P5 in Fig. 5. After this, the level 1 model receives the output of these base models as their features and gives the final output. 5 different algorithms were tried as the level 1 model while keeping the same base models at level 0. This resulted in 5 different types of the proposed classifier. The models are the bagging and boosting models used in techniques 2 and 3, respectively. A conceptual view of the proposed stack ensemble classifier has been given in Fig. 5.

Algorithm 4 Development of proposed stack classifier

Results

The performances of the ML models trained with different sets of features were mea- sured in terms of accuracy, precision, recall, and f1-score. The results of this rigorous evaluation are shown in Tables 1, 2, 3 and 4, where Table 1 shows the accuracy of the developed models, Table 2 shows the precision of the developed models, Table 3 shows the recall of the developed models, and Table 4 shows the f1-score of the developed models. The best performance among the models with the optimal future set has been highlighted in all the tables.

Table 1 Comparison of accuracy of ML models utilizing different numbers of features

Table 2 Comparison of precision of ML models utilizing different numbers of features

Table 3 Comparison of recall of ML models utilizing different numbers of features

Table 4 Comparison of f1-score of ML models utilizing different numbers of features

It is noticeable from Tables 1, 2, 3 and 4 that the performances of the proposed stack classi- fiers outperform other conventional techniques. Among the proposed stack classifiers, the stack classifier with the XGBoost algorithm as the meta-classifier achieved the best performance among all the other models in all 4 performance parameters with the dataset of 65 features selected by the PCA feature engineering technique. It achieved a remarkable accuracy, precision, recall, and f1-score of 99.58%, 99.578%, 99.58%, and 99.579%, respectively. Thus, it is proved that, among 187 features, 65 is the optimal number of features required to train the ML models. It is also evident that, the pro- posed stack classifier with XGBoost as the meta-classifier performs the best with the given dataset.

The f1-score of each model in predicting every class where the models were trained with 65 features extracted by the 3 different feature engineering techniques are shown in Table 5. The reason for showing the f1-score is because the f1-score is the harmonic mean of precision and recall, the other two evaluation metrics. On the other hand, accuracy alone is not a reliable evaluation metric since accuracy can be misleading sometimes [50]. The f1-score is shown for models trained with the 65 most important features since the best-performing model was obtained when the models were trained with the 65 most significant features. It is noticeable from Table 5 that most of the classes have better f1-score when the features were extracted by the PCA technique. The best f1-score per class per model is provided in bold font. From this, we can conclude with the given dataset, the PCA technique is the best in extracting the 65 most useful features for training the models.

Table 5 F1-score of each model per class feature-engineered with 3 methods and 65 features

Discussion

Three types of ensemble techniques with several classifiers were explored, trained, and tested along with conventional ML algorithms to classify heart arrhythmia from ECG signals in this study. Most previous works utilized only the conventional algorithms and very few studies focused on the stack classifier, a state-of-the-art technology. Therefore, the proposed technique based on a stack classifier where conventional, bag- ging, and boosting models were all integrated to achieve a better prediction is a novel contribution in this domain.

Again, most of the previous studies did not utilize feature engineering techniques to reduce the number of features and determine the optimal number of features. There- fore, in this research, 3 different feature engineering techniques (Chi-square, PCA, and RFE) were applied to determine the optimal number of features necessary to train ML models with a satisfactory performance avoiding any complexities like curse of dimensionality, training time, memory requirements and so on.

Finally, the performances of each of the developed models were evaluated based on accuracy, precision, recall, and f1-score to validate the efficacy and effectiveness of the developed models. It was found that the stack classifier that was developed using XGBoost as the meta-classifier and trained with the dataset consisting of 65 features selected by the PCA method outperformed not only all other models but also the previous works carried out with the same dataset. A performance comparison with the previous works is given in Table 6.

Table 6 Comparative analysis with existing ML approaches

Novelty of the study

The proposed research presents a novel and advanced approach to heart arrhythmia diagnosis by developing a sophisticated stack classifier system that leverages cutting- edge ensemble machine-learning techniques. The model is designed with XGBoost as the meta-classifier, a robust and highly effective algorithm known for its strong perfor- mance in classification tasks. This approach is further enhanced by the incorporation of advanced feature engineering techniques, including Principal Component Analysis (PCA), to extract and refine critical features from electrocardiogram (ECG) signals. One of the primary innovations of this research is the automation of the heart arrhythmia diagnosis process, significantly reducing the need for human intervention. Traditionally, the analysis of ECG data has relied heavily on cardiologists and clinical experts, which can be time-consuming and prone to human error. By utilizing this automated system, the research addresses these limitations, offering a faster, more accurate, and reliable alternative for detecting arrhythmias.

The ensemble machine-learning techniques employed in this study, particularly the use of XGBoost, offer substantial improvements over conventional machine-learning methods. XGBoost’s ability to handle large datasets, its superior speed, and its high predictive power make it an ideal choice for this application. Moreover, the integration of PCA allows for the selection of 65 optimal features from the ECG data, ensuring that the classifier is trained with the most relevant information, thus enhancing its performance.

The results of this study are highly promising, with the stack classifier achieving exceptional performance metrics: 99.58% accuracy, 99.57% precision, 99.58% recall, and 99.57% F1-score. These results not only demonstrate the effectiveness of the pro- posed model but also its potential to revolutionize the field of medical diagnostics, particularly in the area of arrhythmia detection. By outperforming other conventional machine-learning and ensemble algorithms, the proposed stack classifier sets a new benchmark for accuracy and reliability in this domain.

This research provides a significant contribution to the field of automated medical diagnostics. The novel stack classifier system, combining XGBoost with PCA-driven feature engineering, offers a powerful tool for the accurate and timely diagnosis of heart arrhythmias. This advancement has the potential to greatly improve patient outcomes by facilitating early detection and treatment, ultimately reducing the mor- tality rate associated with heart arrhythmias. The system’s ability to operate with minimal human intervention also makes it highly scalable and adaptable for use in various clinical settings, further enhancing its practical application in healthcare.

Limitations and future works

This research has certain limitations. Firstly, the classification methods based on image or signal processing techniques were not explored. Secondly, this study’s classification of heart arrhythmia was based solely on 2D data. Thirdly, Transfer learning models, such as pre-trained CNN models, were not investigated due to their reliance on image data. Fourthly, explainable AI, which explains and justifies the AI system predictions, was not explored in this research. Hence, future works may focus on (a) employing image and signal processing techniques with larger sample sizes for the detection of heart arrhythmia from ECG signals, (b) categorizing heart arrhythmia using image data, (c) training and evaluating untested models, and their performances, (d) incor- porating explainable AI with the current research, and (e) conducting a more detailed analysis of the complexities of the models based on time and memory.

Conclusion

Machine learning plays a pivotal role in the precise and timely diagnosis of heart abnormalities, especially in detecting arrhythmias. Its capacity to continuously analyze electrocardiogram (ECG) data allows for the early identification of patterns indicative of arrhythmias, enabling swift intervention. Healthcare professionals may incorporate these machine learning models into their daily practice, enhancing patient care through real-time monitoring and early warning systems.

The need for effective and unbiased analysis of large-scale medical data drives the growing interest in ECG-based cardiac arrhythmia analysis for heart-related studies. Early recognition of heart problems is crucial for prompt treatment and reduced mor- tality rates. However, manual diagnosis of heart conditions is time-consuming and requires expert operators due to the intricacies of the heart’s functions. Thus, the methodology described in this article can be a benchmark for accurate and precise heart arrhythmia classification from ECG signals. The high performances achieved by the proposed methodology demonstrate the validity of this study.