Enhancing Classification Performance Through Employing Feature Selection Algorithms

Abstract

Data classification is a pivotal area of research due to its critical importance across a wide range of applications, such as healthcare, finance, and predictive analytics. This study introduces an innovative approach designed to enhance classification accuracy and produce highly precise outputs. The methodology incorporates five distinct feature selection algorithms—Quick Branch and Bound (QBB), Las Vegas Filter (LVF), Branch and Bound (B&B), Focus, and Sequential Floating Forward Search (SFFS)—to identify the most relevant features. These are followed by the application of five classification algorithms—Sequential Minimal Optimization (SMO), Stacking, Random Tree (RT), Naive Bayes (NB), and Decision Table (DT)—to categorize the data. The classifiers, selected from diverse classification families, were rigorously evaluated using seven datasets encompassing various domains. The proposed approach achieved accuracy improvements ranging from 0.843 to 0.966 for Naive Bayes, 0.837 to 0.964 for SMO, 0.806 to 0.962 for Decision Table, 0.783 to 0.941 for Random Tree, and 0.877 to 0.986 for Stacking, demonstrating a significant enhancement compared to traditional methods. Among the tested combinations, the integration of QBB as a feature selection method with the Stacking classification algorithm yielded the highest performance, achieving an accuracy of 99.2% in certain datasets. Ultimately, the combination of the best-performing feature selection algorithm and the most accurate classification algorithm was identified, showcasing the versatility and robustness of this approach for practical applications across multiple fields.