This page hosts the material for our work A Tutorial on Wikimedia Visual Resources and its Application to Neural Visual Recommender Systems, presented at the 21st IEEE International Conference on Data Mining (IEEE ICDM 2021).
Schedule: 16:00-18:30, Wednesday, December 8, 2021 (GMT+13, Time in Auckland, New Zealand) (See more timezones)
*Corresponding author: Antonio Ossa-Guerra (aaossa[at]uc[dot]cl)
Due to the advancements in deep learning, visual recommendation systems are implemented using visual features from Deep Neural Networks (DNNs) as representations of images. The tutorial focuses on the implementation of visual recommendation systems using deep learning techniques, as well as model evaluation. For this purpose, we present some of the available research resources from the Wikimedia Foundation, introducing a new dataset for image recommendation. The tutorial aims at introducing visual recommendation systems to the data mining community, guiding participants through the complete pipeline of a visual recommendation problem, from data gathering to model evaluation and analysis.
| Duration | Overview | Presenter(s) | Material |
|---|---|---|---|
| 30 mins | Session 1: Introduction to Visual RecSys, datasets and feature extraction with CNNS in Python. Wikimedia Foundation and its available research resources. | Denis Parra & Diego Saez-Trumper & Miriam Redi | Video A, Video B & Slides A, Slides B |
| 20 mins | Session 2: Pipeline for training and testing visual RecSys in Python. | Antonio Ossa-Guerra | Video & Slides |
| 10 mins | BREAK | - | - |
| 25 mins | Session 3: Visual Bayesian Personalized Ranking (VBPR) and Deep Visually-aware Bayesian Personalized Ranking (DVBPR) in Pytorch [2, 3] | Patricio Cerda-Mardini | Video & Slides |
| 20 mins | Session 4: CuratorNet in Pytorch [1] | Manuel Cartagena | Video & Slides |
| 20 mins | Session 5: Attentive Collaborative Filtering (ACF) in Pytorch [4] | Felipe del Río | Video & Slides |
| 15 mins | Live demo of this repository | Isidora Palma | Video |
| 10 mins | Conclusions | Denis Parra | - |
Expected length of tutorial: 2.5 hours (half-day)
Just like you, we have been looking for several years for some datasets to train our models. For instance, the RecSys dataset collection by Prof. Julian McAuley at USCD has datasets, but due to copyright issues he only shares embeddings as .npy and in some cases (such as the Amazon datasets) links to image URLS so you can download them on your own. We need images to test if our recommendations are making sense!
We acknowledge the support of Diego Saez-Trumper from Wikimedia foundation to collect this dataset.
| AUC | RR | R@20 | P@20 | nDCG@20 | R@100 | P@100 | nDCG@100 | |
|---|---|---|---|---|---|---|---|---|
| VisRank | .59216 | .01138 | .01881 | .00111 | .01274 | .03280 | .00039 | .01534 |
| [1] CuratorNet | .61976 | .00931 | .01638 | .00100 | .01051 | .03582 | .00042 | .01403 |
| [2] VBPR | .73062 | .00964 | .01897 | .00113 | .01076 | .06017 | .00069 | .01872 |
| [3] DVBPR | .79573 | .07086 | .03163 | .01809 | .09253 | .10099 | .01155 | .12352 |
| [4] ACF | .77703 | .03547 | .01381 | .00802 | .04792 | .05142 | .00588 | .07886 |
[1] Messina, P., Cartagena, M., Cerda, P., del Rio, F., & Parra, D. (2020). CuratorNet: Visually-aware Recommendation of Art Images. arXiv preprint arXiv:2009.04426.
[2] He, R., & McAuley, J. (2016). VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 30, No. 1).
[3] Kang, W. C., Fang, C., Wang, Z., & McAuley, J. (2017). Visually-aware fashion recommendation and design with generative image models. In 2017 IEEE International Conference on Data Mining (ICDM) (pp. 207-216). IEEE.
[4] Chen, J., Zhang, H., He, X., Nie, L., Liu, W., & Chua, T. S. (2017). Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 335-344).