New Paper
Journal Articles

Affective Image Content Analysis: Two Decades Review and New Perspectives
Sicheng Zhao, Xingxu Yao, Jufeng Yang, Guoli Jia, Guiguang Ding, Tat-Seng Chua, Bjorn W. Schuller, Kurt Keutzer IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021[accepted].

Multimodal Event-Aware Network for Sentiment Analysis in Tourism
Lijuan Wang, Wenya Guo, Xingxu Yao, Yuxiang Zhang, Jufeng Yang
IEEE MultiMedia(
(MM),
2021 [accepted].

LD-MAN: Layout-Driven Multimodal Attention Network for Online News Sentiment Recognition
Wenya Guo, Ying Zhang, Xiangrui Cai, Lei Meng, Jufeng Yang, Xiaojie Yuan IEEE Transactions on Multimedia (TMM), 2021[accepted].

Emotion Recognition From Multiple Modalities: Fundamentals and methodologies
Sicheng Zhao, Guoli Jia, Jufeng Yang, Guiguang Ding, Kurt Keutzer
IEEE Signal Processing Magazine
(SPM),
2021[accepted].

APSE: Attention-aware polarity-sensitive embedding for emotion-based image retrieval
Xingxu Yao, Sicheng Zhao, Yu-Kun Lai, Dongyu She, Jie Liang, Jufeng Yang
IEEE Transcation on Multimedia
(TMM),
2020[accepted].

Adaptive Deep Metric Learning for Affective Image Retrieval and Classification
Xingxu Yao, Dongyu She, Haiwei Zhang, Jufeng Yang, Ming-Ming Cheng and Liang Wang
IEEE Transcation on Multimedia
(TMM),
2020[accepted].

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection
Dongyu She, Jufeng Yang, Ming-Ming Cheng, Yu-Kun Lai, Paul L. Rosin and Liang Wang
IEEE Transcation on Multimedia
(TMM),
2019[accepted].

Learning Discriminative Sentiment Representation from Strongly- and Weakly-Supervised CNNs
Dongyu She, Ming Sun, Jufeng Yang
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)
(TOMM),
2019[accepted].

Visual Sentiment Prediction Based on Automatic Discovery of Affective Regions
Jufeng Yang, Dongyu She, Ming Sun, Ming-Ming Cheng, Paul L. Rosin, and Liang Wang
IEEE Transcation on Multimedia
(TMM),
2018[accepted].
Conference Proceedings

Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal Space
Sicheng Zhao, Yaxian Li, Xingxu Yao, Weizhi Nie, Pengfei Xu, Jufeng Yang, Kurt Keutzer
ACM International Conference on Multimedia(
(ACM MM),
2020[accepted].

An End-to-End Visual-Audio Attention Networt=k for Emotion Recognition in User-Generated Videos
Sicheng Zhao, Yunsheng Ma, Yang Gu, Jufeng Yang, Tengfei Xing, Pengfei Xu, Runbo Hu, Hua Chai, Kurt Keutzer
AAAI Conference on Artificial Intelligence(
(AAAI),
2020[accepted].

Attention-Aware Polarity Sensitive Embedding for Affective Image Retrieval
Xingxu Yao, Dongyu She, Sicheng Zhao, Jie Liang, Yu-Kun Lai, Jufeng Yang
IEEE International Conference on Computer Vision(
(ICCV),
2019[accepted].

Zero-Shot Emotion Recognition via Affective Structural Embedding
Chi Zhan, Dongyu She, Sicheng Zhao, Ming-Ming Cheng, Jufeng Yang
IEEE International Conference on Computer Vision(
(ICCV),
2019[accepted].

Weakly Supervised Coupled Networks for Visual Sentiment Analysis
Jufeng Yang, Dongyu She, Yu-Kun Lai, Paul L. Rosin, Ming-Hsuan Yang
Internaltional Conference on Computer Vision and Pattern Recogintion
(CVPR),
Salt Lake City, Utah, USA,
June. 2018.

Retrieving and Classifying Affective Images via Deep Metric Learning
Jufeng Yang, Dongyu She, Yu-Kun Lai, Ming-Hsuan Yang
AAAI Conference on Artificial Intelligence
(AAAI),
New Orleans, Louisiana, USA,
Feb. 2018.

Understanding Image Impressiveness Inspired by Instantaneous Human Perceptual Cues
Jufeng Yang, Yan Sun, Jie Liang, Yong-Liang Yang, Ming-Ming Cheng
AAAI Conference on Artificial Intelligence
(AAAI),
New Orleans, Louisiana, USA,
Feb. 2018.

Text Emotion Distribution Learning via Multi-Task Convolutional Neural Network
Yuxiang Zhang, Jiamei Fu, Dongyu She, Ying Zhang, Senzhang Wang, and Jufeng Yang
International Joint Conference on Artificial Intelligence
(IJCAI),
2018[accepted].

Deep Coordinated Textual and Visual Network for Sentiment-Oriented Cross-Modal Retrieval
Jiamei Fu, Dongyu She, Xingxu Yao, Yuxiang Zhang, and Jufeng Yang
Pacific Rim International Conferences on Artificial Intelligence
(PRICAI),
2018[accepted].

Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network
Jufeng Yang, Dongyu She, Ming Sun
International Joint Conference on Artificial Intelligence
(IJCAI),
2017[accepted].

Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network
Jufeng Yang, Ming Sun, Xiaoxiao Sun
Internaltional Conference on Computer Vision and Pattern Recogintion
(AAAI),
San Francisco, California, USA,
Feb. 2017.

Discovering Affctive Regions In Deep Convolutional Neural Networks For Visual Sentiment Prediction
Ming Sun, Jufeng Yang, Kai Wang, Hui Shen
International Conference on Multimedia and Expositions
(ICME),
2016[accepted].
Datasets
- Download Abstract and Artphoto dataset click here.
[cite]
@inproceedings{machajdik2010affective,
author = {Machajdik, Jana and Hanbury, Allan},
title = {Affective Image Classification Using Features Inspired by Psychology and Art Theory},
year = {2010},
booktitle = {Proceedings of the 18th ACM International Conference on Multimedia},
doi = {10.1145/1873951.1873965}
}
- Download ArtEmis dataset click here.
[cite]
@InProceedings{Achlioptas_2021_CVPR,
author = {Achlioptas, Panos and Ovsjanikov, Maks and Haydarov, Kilichbek and Elhoseiny, Mohamed and Guibas, Leonidas J.},
title = {ArtEmis: Affective Language for Visual Art},
year = {2021},
month = {June},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
}
- Download B-T4SA dataset click here.
[cite]
@InProceedings{Vadicamo_2017_ICCV,
author = {Vadicamo, Lucia and Carrara, Fabio and Cimino, Andrea and Cresci, Stefano and Dell'Orletta, Felice and Falchi, Fabrizio and Tesconi, Maurizio},
title = {Cross-Media Learning for Image Sentiment Analysis in the Wild},
year = {2017},
month = {Oct},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops}
}
- Download Comics dataset click here.
[cite]
@article{she2019learning,
author = {She, Dongyu and Sun, Ming and Yang, Jufeng},
title = {Learning Discriminative Sentiment Representation from Strongly- and Weakly Supervised CNNs},
year = {2019},
issue_date = {January 2020},
publisher = {Association for Computing Machinery},
volume = {15},
number = {3s},
issn = {1551-6857},
}
- Download EMOd dataset click here.
[cite]
@InProceedings{Fan_2018_CVPR,
author = {Fan, Shaojing and Shen, Zhiqi and Jiang, Ming and Koenig, Bryan L. and Xu, Juan and Kankanhalli, Mohan S. and Zhao, Qi},
title = {Emotional Attention: A Study of Image Sentiment and Visual Attention},
year = {2018},
month = {June},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}
}
- Download EMOTIC dataset click here.
[cite]
@InProceedings{Kosti_2017_CVPR,
author = {Kosti, Ronak and Alvarez, Jose M. and Recasens, Adria and Lapedriza, Agata},
title = {Emotion Recognition in Context},
year = {2017},
month = {July},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}
}
- Download Emotion6 dataset click here.
[cite]
@InProceedings{Peng_2015_CVPR,
author = {Peng, Kuan-Chuan and Chen, Tsuhan and Sadovnik, Amir and Gallagher, Andrew C.},
title = {A Mixed Bag of Emotions: Model, Predict, and Transfer Emotion Distributions},
year = {2015},
month = {June},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}
}
- Download EmotionROI dataset click here.
[cite]
@inproceedings{peng2016emotions,
author={Peng, Kuan-Chuan and Sadovnik, Amir and Gallagher, Andrew and Chen, Tsuhan},
title={Where do emotions come from? Predicting the Emotion Stimuli Map},
year={2016},
booktitle={2016 IEEE International Conference on Image Processing (ICIP)}
}
- Download Emotion-6 dataset click here.
[cite]
@InProceedings{Panda_2018_ECCV,
author = {Panda, Rameswar and Zhang, Jianming and Li, Haoxiang and Lee, Joon-Young and Lu, Xin and Roy-Chowdhury, Amit K.},
title = {Cross-Media Learning for Image Sentiment Analysis in the Wild},
year = {2018},
month = {September},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}
}
- Download Event dataset click here.
[cite]
@inproceedings{ahsan2017towards,
author={Ahsan, Unaiza and De Choudhury, Munmun and Essa, Irfan},
title={Towards using visual attributes to infer image sentiment of social events},
year = {2017},
month = {Oct},
booktitle={2017 International Joint Conference on Neural Networks (IJCNN)},
doi={10.1109/IJCNN.2017.7966013}
}
- Download FI dataset click here.
[cite]
@article{You_Luo_Jin_Yang_2016,
author={You, Quanzeng and Luo, Jiebo and Jin, Hailin and Yang, Jianchao},
title={Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark},
year = {2016},
month = {Feb.},
journal={Proceedings of the AAAI Conference on Artificial Intelligence}
}
- Download flickr and instagram dataset click here.
[cite]
@inproceedings{katsurai2016image,
author={Katsurai, Marie and Satoh, Shin'ichi},
title={Image sentiment analysis using latent correlations among visual, textual, and sentiment views},
year = {2016},
booktitle={2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
doi={10.1109/ICASSP.2016.7472195}
}
- Download FlickrLDL and TwitterLDL dataset click here.
[cite]
@inproceedings{yang2017learning,
author={Yang, Jufeng and Sun, Ming and Sun, Xiaoxiao},
title={Learning visual sentiment distributions via augmented conditional probability neural network},
year = {2017},
booktitle={Thirty-first AAAI conference on artificial intelligence}
}
- Download GAPED dataset click here.
[cite]
@article{dan2011geneva,
author={Dan-Glauser, Elise S and Scherer, Klaus R},
title={The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance},
year = {2011},
journal={Behavior research methods},
publisher={Springer},
volume={43},
number={2},
pages={468--477}
}
- Download IAPSa dataset click here.
[cite]
@article{mikels2005emotional,
author={Mikels, Joseph A and Fredrickson, Barbara L and Larkin, Gregory R and Lindberg, Casey M and Maglio, Sam J and Reuter-Lorenz, Patricia A},
title={Emotional category data on images from the International Affective Picture System},
year = {2005},
journal={Behavior research methods},
publisher={Springer},
volume={37},
number={4},
pages={626--630}
}
- Download SE30K8 dataset click here.
[cite]
@inproceedings{wei2020learning,
author={Wei, Zijun and Zhang, Jianming and Lin, Zhe and Lee, Joon-Young and Balasubramanian, Niranjan and Hoai, Minh and Samaras, Dimitris},
title={Learning visual emotion representations from web data},
year={2020},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}
}
- Download StockEmotion dataset click here.
[cite]
@inproceedings{wei2020learning,
author={Wei, Zijun and Zhang, Jianming and Lin, Zhe and Lee, Joon-Young and Balasubramanian, Niranjan and Hoai, Minh and Samaras, Dimitris},
title={Learning visual emotion representations from web data},
year={2020},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}
}
- Download Twitter1269 dataset click here.
[cite]
@inproceedings{you2015robust,
author={You, Quanzeng and Luo, Jiebo and Jin, Hailin and Yang, Jianchao},
title = {Cross-Media Learning for Image Sentiment Analysis in the Wild},
year = {2015},
booktitle={Twenty-ninth AAAI conference on artificial intelligence}
}
- Download Twitter603 dataset click here.
[cite]
@inproceedings{borth2013large,
author={Borth, Damian and Ji, Rongrong and Chen, Tao and Breuel, Thomas and Chang, Shih-Fu},
title={Large-scale visual sentiment ontology and detectors using adjective noun pairs},
year = {2013},
booktitle={Proceedings of the 21st ACM international conference on Multimedia}
}
- Download UnBiasedEmo dataset click here.
[cite]
@InProceedings{Panda_2018_ECCV,
author = {Panda, Rameswar and Zhang, Jianming and Li, Haoxiang and Lee, Joon-Young and Lu, Xin and Roy-Chowdhury, Amit K.},
title = {Cross-Media Learning for Image Sentiment Analysis in the Wild},
year = {2018},
month = {September},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}
}
- Download VSO dataset click here.
[cite]
@inproceedings{borth2013large,
author={Borth, Damian and Ji, Rongrong and Chen, Tao and Breuel, Thomas and Chang, Shih-Fu},
title={Large-scale visual sentiment ontology and detectors using adjective noun pairs},
year = {2013},
booktitle={Proceedings of the 21st ACM international conference on Multimedia}
}
- Download WEBEmo dataset click here.
[cite]
@InProceedings{Panda_2018_ECCV,
author = {Panda, Rameswar and Zhang, Jianming and Li, Haoxiang and Lee, Joon-Young and Lu, Xin and Roy-Chowdhury, Amit K.},
title = {Cross-Media Learning for Image Sentiment Analysis in the Wild},
year = {2018},
month = {September},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}
}
All Demos

