Analyzing the Efficacy of Base Algorithms: Application of Machine Learning Models in Classification Tasks
DOI:
https://doi.org/10.70862/CSIR.2025.0201-12Keywords:
machine learning , deep learning, artificial neural networks, classification, TitanicAbstract
The article presents a comprehensive examination and comparison of several machine learning algorithms, such as Artificial Neural Network, K-Nearest Neighbor, Support Vector Machines, Decision Tree, and Random Forests, with a specific focus on the significance of neural networks in classification tasks. This study employs three distinct datasets: (1) the "Titanic" dataset is about the sinking of the British transatlantic liner, which resulted in the deaths of 1502 of the 2224 people who were on board. On April 15, 1912, the event occurred in the North Atlantic Ocean; (2) the "MNIST" dataset, which consists of 70,000 grayscale images (28×28 pixels) of handwritten digits; (3) and the "Fashion-MNIST" dataset, which showcases images of 10 fashion item classes such as T-shirt/Top, Trouser, Pullover, Dress, Coat, Sandals, Shirt, Sneaker, Bag, Ankle boots which have been classified. The analysis evaluates the strengths and weaknesses of each algorithm based on the research outcomes. The results suggest that for certain classification tasks, the use of machine learning algorithms may be more appropriate, as the accuracy levels may remain relatively unchanged, and vary depending on the size of the dataset used with the features it includes in the classification tasks.
References
Soori, M., Arezoo, B., & Dastres, R. (2023). Artificial intelligence, machine learning and deep learning in advanced robotics: A review. Cognitive Robotics, 3, 54–70. https://doi.org/10.1016/j.cogr.2023.04.001
Salal, Y. K., Hussain, M., & Theodorou, P. (2021). Student next assignment submission prediction using a machine learning approach. In A. A. Radionov & V. R. Gasiyarov (Eds.), Advances in Automation II. RusAutoCon 2020 (Lecture Notes in Electrical Engineering, Vol. 729). Springer. https://doi.org/10.1007/978-3-030-71119-1_38
Hussain, S., Muhsion, Z. F., Salal, Y. K., Theodorou, P., Kurtoğlu, F., & Hazarika, G. C. (2019). Prediction model on student performance based on internal assessment using deep learning. International Journal of Emerging Technologies in Learning, 14(8), 4. https://doi.org/10.3991/ijet.v14i08.10001
Kaggle. (n.d.). Titanic – Machine learning from disaster. Retrieved February 3, 2025, from https://www.kaggle.com/c/titanic/data
Scikit-learn. (n.d.). SVC. Retrieved February 6, 2025, from https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
Scikit-learn. (n.d.). KNeighborsClassifier. Retrieved from https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html
Scikit-learn. (n.d.). DecisionTreeClassifier. Retrieved February 5, 2025, from https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html
Scikit-learn. (n.d.). RandomForestClassifier. Retrieved February 3, 2025, from https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Publ. Springer New York. https://doi.org/10.1007/978-0-387-84858-7
Breiman, L., & Ihaka, R. (1984). Nonlinear discriminant analysis via scaling and ACE (Technical Report No. 40). Department of Statistics, University of California, Berkeley. https://digitalassets.lib.berkeley.edu/sdtr/ucb/text/40.pdf
McCloskey, M., & Cohen, N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24, 109–165. https://doi.org/10.1016/S0079-7421(08)60536-8
Brown, W. M., Gedeon, T. D., Groves, D. I., & Barnes, R. G. (2000). Artificial neural networks: A new method for mineral prospectivity mapping. Australian Journal of Earth Sciences, 47, 757–770. https://doi.org/10.1046/j.1440-0952.2000.00807.x
Rigol-Sanchez, J. P., Chica-Olmo, M., & Abarca-Hernandez, F. (2003). Artificial neural networks as a tool for mineral potential mapping with GIS. International Journal of Remote Sensing, 24, 1151–1156. https://doi.org/10.1080/0143116021000031791
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational Learning Theory (pp. 144–152). https://doi.org/10.1145/130385.130401
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297. https://doi.org/10.1007/BF00994018
Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32. https://doi.org/10.1023/A:1010933404324
Rodriguez-Galiano, V. F., & Chica-Rivas, M. (2014). Evaluation of different machine learning methods for land cover mapping of a Mediterranean area using multi-seasonal Landsat images and digital terrain models. International Journal of Digital Earth, 7, 492–509. https://doi.org/10.1080/17538947.2012.748848
Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and regression trees (Vol. 37). Wadsworth. https://doi.org/10.1201/9781315139470
Al-Anazi, A., & Gates, I. D. (2010). A support vector machine algorithm to classify lithofacies and model permeability in heterogeneous reservoirs. Engineering Geology, 114, 267–277. https://doi.org/10.1016/j.enggeo.2010.05.005
Ben-Hur, A., Horn, D., Siegelmann, H., & Vapnik, V. N. (2001). Support vector clustering. Journal of Machine Learning Research, 2, 125–137.
Chen, W., Xie, X. S., Wang, J. L., Pradhan, B., Hong, H. Y., Bui, D. T., Duan, Z., & Ma, J. Q. (2017). A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility. Catena, 151, 147–160. https://doi.org/10.1016/j.catena.2016.11.032
Gutiérrez, S. L. M., Rivero, M. H., Ramírez, N. C., Hernández, E., & Aranda-Abreu, G. E. (2014). Decision trees for the analysis of genes involved in Alzheimer’s disease pathology. Journal of Theoretical Biology, 357, 21–25. https://doi.org/10.1016/j.jtbi.2014.05.002
Ghimire, B., Rogan, J., Galiano, V. R., Panday, P., & Neeti, N. (2012). An evaluation of bagging, boosting, and random forests for land-cover classification in Cape Cod, Massachusetts, USA. GIScience & Remote Sensing, 49, 5, 623–643. https://doi.org/10.2747/1548-1603.49.5.623
Rodriguez-Galiano, V. F., Ghimire, B., Rogan, J., Chica-Olmo, M., & Rigol-Sanchez, J. P. (2012). An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS Journal of Photogrammetry and Remote Sensing, 67, 93–104. https://doi.org/10.1016/j.isprsjprs.2011.11.002
Pierdicca, R., Malinverni, E. S., Piccinini, F., Paolanti, M., Felicetti, A., & Zingaretti, P. (2018). Deep convolutional neural network for automatic detection of damaged photovoltaic cells. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 893–900. https://doi.org/10.5194/isprs-archives-XLII-2-893-2018
Funaya, H., & Ikeda, K. (2012). A statistical analysis of soft-margin support vector machines for non-separable problems. In 2012 International Joint Conference on Neural Networks (IJCNN) (pp. 1–7). IEEE. https://doi.org/10.1109/IJCNN.2012.6252443
Cohen, G., Afshar, S., Tapson, J., & Van Schaik, A. (2017). EMNIST: Extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2921–2926). IEEE. https://doi.org/10.1109/IJCNN.2017.7966217
Meshkini, K., Platos, J., & Ghassemain, H. (2020). An analysis of convolutional neural network for fashion images classification (fashion-MNIST). In Proceedings of the Fourth International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’19), 4, (pp. 85–95). Springer International Publishing. https://doi.org/10.1007/978-3-030-50097-9_10
Bi, W., & Kwok, J. (2013). Efficient multi-label classification with many labels. In Proceedings of the 30th International Conference on Machine Learning, 28 (pp. 405–413). PMLR.
Zhang, Z. (2018). Improved Adam optimizer for deep neural networks. In 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS) (pp. 1–2). IEEE. https://doi.org/10.1109/IWQoS.2018.8624183
Chauhan, N. K., & Singh, K. (2018, September). A review on conventional machine learning vs deep learning. In 2018 International Conference on Computing, Power and Communication Technologies (GUCON) (pp. 347–352). IEEE. https://doi.org/10.1109/GUCON.2018.8675097
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Dr. Yass K. Salal (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
