From d1dc70ba0bdbb856c1f34679661551fb9ca36fee Mon Sep 17 00:00:00 2001 From: IgorSusmelj Date: Fri, 17 Jan 2025 17:55:36 +0100 Subject: [PATCH] Update SimCLR model docs --- docs/source/examples/simclr.rst | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/source/examples/simclr.rst b/docs/source/examples/simclr.rst index 8f5833cd6..f83047821 100644 --- a/docs/source/examples/simclr.rst +++ b/docs/source/examples/simclr.rst @@ -3,7 +3,21 @@ SimCLR ====== -Example implementation of the SimCLR architecture. +SimCLR is a framework for self-supervised learning of visual representations using contrastive learning. It aims to maximize agreement between different augmented views of the same image. + +Key Components +-------------- + +- **Data Augmentations**: SimCLR uses random cropping, resizing, color jittering, and Gaussian blur to create diverse views of the same image. +- **Backbone**: Convolutional neural networks, such as ResNet, are employed to encode augmented images into feature representations. +- **Projection Head**: A multilayer perceptron (MLP) maps features into a space where contrastive loss is applied, enhancing representation quality. +- **Contrastive Loss**: The normalized temperature-scaled cross-entropy loss (NT-Xent) encourages similar pairs to align and dissimilar pairs to diverge. + +Good to Know +---------------- + +- **Backbone Networks**: SimCLR is specifically optimized for convolutional neural networks, with a focus on ResNet architectures. We do not recommend using it with transformer-based models. +- **Learning Paradigm**: SimCLR is based on contrastive learning which makes it sensitive to the augmentations you pick and the method benefits from larger batch sizes. Reference: `A Simple Framework for Contrastive Learning of Visual Representations, 2020 `_