Improving Customer Churn Prediction with Gradient Boosting Machine: A Hybrid Approach of Parameter Optimization and Data Balancing
Keywords:
Customer churn prediction, Gradient boosting machine, Machine learning, Randomized search CV, SMOTEAbstract
Customer churn prediction in the banking industry plays a crucial role in reducing the potential for significant financial losses. By applying machine learning methods, banks can identify patterns and analyze customers who have the potential to churn, thus enabling preventive actions to increase customer loyalty. This research focuses on applying the Gradient Boosting Machine (GBM) algorithm for customer churn prediction, modified with Synthetic Minority Oversampling Technique (SMOTE) in the data preprocessing stage and Randomized Search for parameter optimization. The purpose is to evaluate the effectiveness of the combination of SMOTE and Randomized Search methods in improving the performance of the prediction model. This research uses the CRISP-DM framework as the research framework. The parameter optimization process is performed using Randomized Search, which offers a randomized parameter search approach to optimize accuracy. In the data preprocessing stage, SMOTE was used to handle significant data imbalances to ensure more optimal data quality for the model. The GBM algorithm was tested in two scenarios, with and without the implementation of SMOTE and Randomized Search. Model performance was evaluated using various metrics, including confusion matrix, accuracy, precision, recall, F1-score, and AUC-ROC. The results showed that GBM without Randomized Search and SMOTE resulted in an accuracy of 87.17%. In contrast, the implementation of GBM with Randomized Search and SMOTE increased the accuracy to 90.85%. This study concludes that the implementation of SMOTE and Randomized Search significantly improves the performance of the customer churn prediction model.
References
Bentéjac, C., Csörgő, A., & Martínez-Muñoz, G. (2021). A comparative analysis of gradient boosting algorithms. Artificial Intelligence Review, 54(3), 1937–1967. https://doi.org/10.1007/s10462-020-09896-5
Biau, G., Cadre, B., & Rouvière, L. (2019). Accelerated gradient boosting. Machine Learning, 108(6), 971–992. https://doi.org/10.1007/s10994-019-05787-1
Clementine Aksama, M., & Wahyuniati, A. (2022). Prediksi Churn Nasabah Bank Menggunakan Klasifikasi Naive Bayes dan ID3. Jurnal Processor, 17(1), 9–18. https://doi.org/10.33998/processor.2022.17.1.1170
Dablain, D., Krawczyk, B., & Chawla, N. V. (2023). DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data. IEEE Transactions on Neural Networks and Learning Systems, 34(9), 6390–6404. https://doi.org/10.1109/TNNLS.2021.3136503
Farid Naufal, M., Fernando Susanto, A., Nathaneil Kansil, C., & Huda, S. (2023). Analisis Perbandingan Algoritma Machine Learning untuk Prediksi Potensi Hilangnya Nasabah Bank Application of Machine Learning to Predict Potential Loss of Bank Customer. In Februari (Vol. 22, Issue 1).
Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. In Source: The Annals of Statistics (Vol. 29, Issue 5).
Hafidatus Sholeha, S., Faid, M., & Yaqin, M. A. (2024). Prediksi Perpindahan Pelanggan Pada Toko Online Menggunakan Metode Tree-Based Gradient Boosted Models. Journal of Computer System and Informatics, 5(3), 605–614. https://doi.org/10.47065/josyc.v5i3.5215
Irmanda, H. N., Astriratma, R., & Afrizal, S. (2019). Perbandingan Metode Jaringan Syaraf Tiruan Dan Pohon Keputusan Untuk Prediksi Churn. Jurnal Sistem Informasi, 11, 1817–1825.
Liu, H., Simonyan, K., Vinyals, O., Fernando, C., & Kavukcuoglu, K. (2017). Hierarchical Representations for Efficient Architecture Search. http://arxiv.org/abs/1711.00436
Maretva Cendani, L., & Wibowo, A. (2022). Perbandingan Metode Ensemble Learning pada Klasifikasi Penyakit Diabetes (Vol. 13, Issue 1).
Namira, Slamet, I., & Susanto, I. (2024). Prediksi Nasabah Churn Dengan Algoritma Decision Tree, Random Forest Dan Support Vector Machine. ESCAF, 1045–1053. https://help.sap.com
Rianti, A., Majid, N. W. A., & Fauzi, A. (2023). CRISP-DM: Metodologi Proyek Data Science.
Schröer, C., Kruse, F., & Gómez, J. M. (2021). A systematic literature review on applying CRISP-DM process model. Procedia Computer Science, 181, 526–534. https://doi.org/10.1016/j.procs.2021.01.199
Wang, S., Dai, Y., Shen, J., & Xuan, J. (2021). Research on expansion and classification of imbalanced data based on SMOTE algorithm. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-03430-5
Zhang, Y., & Haghani, A. (2015). A gradient boosting method to improve travel time prediction. Transportation Research Part C: Emerging Technologies, 58, 308–324. https://doi.org/10.1016/j.trc.2015.02.019
