CALIBRATED POPULARITY RE-RANKING WITH ALTERNATIVE DIVERGENCE MEASURES FOR POPULARITY BIAS MITIGATION
Abstract
Popularity bias significantly limits the effectiveness of recommender systems by disproportionately favoring popular items and reducing exposure to diverse, less-known content. This bias negatively impacts personalization and marginalizes niche users and item providers. To address this challenge, calibrated recommendation methods have gained attention, notably the Calibrated Popularity (CP) approach, due to its simplicity, effectiveness, and model-agnostic nature. Originally, CP employs Jensen–Shannon divergence (JSD) to align the popularity distribution of recommended items with users’ historical interaction patterns. However, the choice of divergence measure substantially impacts calibration effectiveness and recommendation diversity. In this study, we systematically explore several alternative divergence measures, including Chi-Square, Wasserstein, Kullback–Leibler, Hellinger, Total Variation, Bhattacharyya, Cosine, and Renyi divergences, within the CP framework. Additionally, we propose a novel divergence-independent evaluation metric, namely Overall Similarity Error, enabling consistent assessment of calibration quality across divergence measures. Experimental results on two benchmark datasets using two collaborative filtering algorithms highlighted critical insights. More aggressive divergences, particularly Chi-Square, significantly enhanced calibration quality, reduced popularity bias, and increased recommendation diversity, albeit with a modest reduction in accuracy. In contrast, smoother divergences, such as JSD, maintained higher accuracy but provided limited improvements in reducing popularity bias. Also, the performed group-based analysis categorizing users into mainstream, balanced, and niche segments based on their historical popularity preferences revealed distinct patterns: balanced users typically achieved higher accuracy due to their evenly distributed preferences; mainstream users showed superior calibration results benefiting from robust signals of popular items; niche users obtained more diverse and personalized recommendations, clearly benefiting from aggressive divergence measures. These results underscore the complexity of addressing popularity bias and highlight the importance of adopting adaptive, user-aware calibration strategies to effectively balance accuracy, diversity, and fairness in recommender systems.
Keywords
Divergence measures, Popularity bias, Calibrated recommendation, Collaborative filtering, Personalization.
References
- [1] Zangerle E, Bauer C. Evaluating recommender systems: survey and framework. ACM Comput Surv 2022; 55(8): 1-38.
- [2] Ahanger AB, Aalam SW, Bhat MR, Assad A. Popularity bias in recommender systems-a review. In: Int Conf Emerging Technol Comput Eng; February 2022; Cham, Switzerland. Cham: Springer. pp. 431-444.
- [3] Abdollahpouri H, Mansoury M, Burke R, Mobasher B. The unfairness of popularity bias in recommendation. arXiv preprint arXiv:1907.13286.
- [4] Mansoury M, Abdollahpouri H, Pechenizkiy M, Mobasher B, Burke R. Feedback loop and bias amplification in recommender systems. In: 29th ACM Int Conf Inf Knowl Manag; 2020; New York, NY, USA. New York: ACM. pp. 2145-2148.
- [5] Steck H. Calibrated recommendations. In: 12th ACM Conf Recommender Syst; 2018; New York, NY, USA. New York: ACM. pp. 154-162.
- [6] Abdollahpouri H, Mansoury M, Burke R, Mobasher B, Malthouse E. User-centered evaluation of popularity bias in recommender systems. In: 29th ACM Conf User Modeling, Adaptation and Personalization; 2021; New York, NY, USA. New York: ACM. pp. 119-129.
- [7] Yalcin E, Bilge A. Evaluating unfairness of popularity bias in recommender systems: A comprehensive user-centric analysis. Inf Process Manag 2022; 59(6): 103100.
- [8] Pardo L. Statistical Inference Based on Divergence Measures. Boca Raton, FL, USA: Chapman and Hall/CRC, 2018.
- [9] Kaya M, Bridge D. A comparison of calibrated and intent-aware recommendations. In: 13th ACM Conf Recommender Syst; 2019; New York, NY, USA. New York: ACM. pp. 151-159.
- [10] Seymen S, Abdollahpouri H, Malthouse EC. A constrained optimization approach for calibrated recommendations. In: 15th ACM Conf Recommender Syst; 2021; New York, NY, USA. New York: ACM. pp. 607-612.