Optimizing clustering efficiency with weighted k-means: a machine learning-driven approach for enhanced accuracy and scalability

Abstract

Data analysis unlocks the hidden, latent patterns and structures within datasets. Clustering algorithms, the cornerstone of any data analysis, are usually challenged by high-dimensionality, complexity, or large-scale data. This research proposes a hybrid model that merges neural networks and clustering techniques to handle these problems. Neural networks are used for feature extraction and dimensionality reduction; raw data will be transformed into a robust, low-dimensional representation. With these refined features, the performance of clustering algorithms improves in terms of scalability, efficiency, and accuracy. The proposed model is tested on diversified datasets such as the wisconsin breast cancer dataset (WBCD), GEO Dataset, and image and text data benchmarks for which substantial improvements in clustering metrics such as silhouette score, purity, and computational efficiency are reported. The results demonstrate the efficacy of the hybrid approach in optimizing clustering applications across domains, such as bioinformatics, health care, and image analysis.