Multi-visual modality for collaborative filtering-based personalized POI recommendations

Abstract

Point-of-interest (POI) recommendation systems help users discover locations that match their interests. However, these systems often suffer from data sparsity due to limited user check-in history. To address this challenge, this study proposed a novel user profiling framework that incorporates multiple visual modalities derived from user-generated photos. Three types of visual-based user profiles were constructed: image label-based, image feature-based, and a fused profile, combining both modalities through score-level fusion. We conducted extensive experiments on two real-world datasets. The results demonstrate that visual-based profiles, particularly the image feature-based profile, consistently improve recommendation performance under sparse data conditions. Although the fused profile offered stable results, it did not consistently outperform the single modality. Furthermore, performance was sensitive to the number of nearest neighbors and the amount of training data. These findings highlight the importance of modality selection and fusion strategy in visual-based POI recommendation systems.