MLFF-Net: a multi-model late feature fusion network for skin disease classification

Abstract

Early diagnosis is paramount to preventing skin diseases and reducing mortality, given their global prevalence. Visual detection by experts using dermoscopy images has become the gold standard for detecting skin cancer. However, a significant challenge in skin cancer detection and classification lies in the similarity of appearance among skin disease lesions and the complexity of dermoscopic images. In response, we developed multi-model late feature fusion network (MLFF-Net), a multi-model late feature fusion network tailored for skin disease detection. Our approach begins with image pre-processing techniques to enhance image quality. We then employ a two-stream network comprising an enhanced densely linked network (DenseNet-121) and a vision transformer (ViTb16). We leverage shallow and deep feature fusion, late fusion, and an attention module to enhance the model’s feature extraction efficiency. The subsequent feature fusion module constructs multi-receptive fields to capture disease information across various scales and uses generalized mean pooling (GeM) pooling to reduce the spatial dimensions of lesion characteristics. Finally, we implement and test our skin lesion categorization model, demonstrating its effectiveness. Despite the combination, convolutional neural network (CNN) outperforms ViT approaches, with our model enhancing the accuracy of the best model by 6.1%.