Augmentation Network for Generalised Zero-Shot Learning
RAFAEL FELIX (The University of Adelaide)*, Michele Sasdelli (The University of Adelaide), Ian Reid ("University of Adelaide, Australia"), Gustavo Carneiro (University of Adelaide)
Keywords: Generative models for computer vision
Abstract:
Generalised zero-shot learning (GZSL) is defined by a training process containing a set of visual samples from seen classes and a set of semantic samples from seen and unseen classes, while the testing process consists of the classification of visual samples from the seen and the unseen classes. Current approaches are based on inference processes that rely on the result of a single modality classifier (visual, semantic, or latent joint space) that balances the classification between the seen and unseen classes using gating mechanisms. There are a couple of problems with such approaches: 1) multi-modal classifiers are known to generally be more accurate than single modality classifiers, and 2) gating mechanisms rely on a complex one-class training of an external domain classifier that modulates the seen and unseen classifiers. In this paper, we mitigate these issues by proposing a novel GZSL method -- augmentation network that tackles multi-modal and multi-domain inference for generalised zero-shot learning (AN-GZSL). The multi-modal inference combines visual and semantic classification and automatically balances the seen and unseen classification using temperature calibration, without requiring any gating mechanisms or external domain classifiers. Experiments show that our method produces the new state-of-the-art GZSL results for fine-grained benchmark data sets CUB and FLO and for the large-scale data set ImageNet. We also obtain competitive results for coarse-grained data sets SUN and AWA. We show an ablation study that justifies each stage of the proposed AN-GZSL.
SlidesLive
Similar Papers
MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network
Jiaze Sun (Imperial College London)*, Binod Bhattarai (Imperial College London), Tae-Kyun Kim (Imperial College London)

OpenGAN: Open Set Generative Adversarial Networks
Luke Ditria (Monash University), Benjamin J. Meyer (Monash University)*, Tom Drummond (Monash University)

Exploiting Transferable Knowledge for Fairness-aware Image Classification
sunhee hwang (Yonsei university)*, Sungho Park (Yonsei University), Pilhyeon Lee (Yonsei University), seogkyu jeon (Yonsei university), Dohyung Kim (Yonsei University), Hyeran Byun (Yonsei University)
