Addressing Class Imbalance in Scene Graph Parsing by Learning to Contrast and Score
He Huang (University of Illinois at Chicago)*, Shunta Saito (Preferred Networks, Inc.), Yuta Kikuchi (Preferred Networks, Inc.), Eiichi Matsumoto (Preferred Networks, Inc.), Wei Tang (University of Illinois at Chicago), Philip S. Yu (UIC)
Keywords: Applications of Computer Vision, Vision for X
Abstract:
Scene graph parsing aims to detect objects in an image scene and recognize their relations. Recent approaches have achieved high average scores on some popular benchmarks, but fail in detecting rare relations, as the highly long-tailed distribution of data biases the learning towards frequent labels. Motivated by the fact that detecting these rare relations can be critical in real-world applications, this paper introduces a novel integrated framework of classification and ranking to resolve the class imbalance problem in scene graph parsing. Specifically, we design a new Contrasting Cross-Entropy loss, which promotes the detection of rare relations by suppressing incorrect frequent ones. Furthermore, we propose a novel scoring module, termed as Scorer, which learns to rank the relations based on the image features and relation features to improve the recall of predictions. Our framework is simple and effective, and can be incorporated into current scene graph models. Experimental results show that the proposed approach improves the current state-of-the-art methods, with a clear advantage of detecting rare relations.
SlidesLive
Similar Papers
DEAL: Difficulty-aware Active Learning for Semantic Segmentation
Shuai Xie (Zhejiang University), Zunlei Feng (Zhejiang University), Ying chen (Zhejiang University), Songtao Sun (Zhejiang University), Chao Ma (Zhejiang University), Mingli Song (Zhejiang University)*

Webly Supervised Semantic Embeddings for Large Scale Zero-Shot Learning
Yannick Le Cacheux (CEA LIST)*, Adrian Popescu (CEA LIST), Herve Le Borgne (CEA LIST)

Learning to Adapt to Unseen Abnormal Activities under Weak Supervision
JaeYoo Park (Seoul National University)*, Junha Kim (Seoul National University), Bohyung Han (Seoul National University)
