Description
Many recent self-supervised learning (SSL) algorithms are based on encoding different "views" of the same image into a common latent space, e.g., BYOL, DINO. These views are typically augmentations of the input, such as rotations, reflections, magnifications, etc. Recent work on self-supervised learning for radio astronomy has shown that the choice of augmentation used for these views can strongly impact model performance when working with astronomical images. In particular, commonly used augmentations such as image crops can be detrimental to model performance when using semantically sparse images such as those recovered from radio telescopes. In this project, we propose using a variational autoencoder (VAE) to learn a variational representation of radio galaxy images that can be used to generate new "views" of each sample stochastically rather than relying on pre-selected augmentations. We then evaluate the performance of various view-based SSL algorithms on radio astronomy data when incorporating this new VAE-based approach.
