Poster
in
Workshop: Mathematical and Empirical Understanding of Foundation Models (ME-FoMo)
Look Globally and Locally: Inter-Intra Contrastive Learning from Unlabeled Videos
David Fan · Deyu Yang · Xinyu Li · Vimal Bhat · Rohith MV
Keywords: [ Video Representation ] [ contrastive learning ] [ action recognition ]
State-of-the-art video contrastive learning methods spatiotemporally augment two clips from the same video as positives. By only sampling positive clips from the same video, these methods neglect other semantically related videos that can also be useful. To address this limitation, we leverage nearest-neighbor videos from the global space as additional positives, thus improving diversity and introducing a more relaxed notion of similarity that extends beyond video and even class boundaries. Our Inter-Intra Video Contrastive Learning (IIVCL) improves performance and generalization on video classification, detection, and retrieval tasks.