Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DetConSLoss and DetConBLoss #1771

Merged
merged 68 commits into from
Jan 6, 2025
Merged

Conversation

liopeer
Copy link
Contributor

@liopeer liopeer commented Jan 2, 2025

Description

  • adds the losses DetConBLoss for Detcon with BYOL and DetConSLoss for Detcon with SimCLR
  • adds tests
  • adds the classes to the docs
  • makes them available as lightly.loss.DetConBLoss and lightly.loss.DetConSLoss

How has it been tested?

  • non-distributed: against values gathered from the original JAX implementation
  • distributed: only if it runs through and creates scalar value

Attachments

  • docs screenshot
    Screenshot 2025-01-02 at 12 48 53

Copy link

codecov bot commented Jan 2, 2025

Codecov Report

Attention: Patch coverage is 97.70115% with 2 lines in your changes missing coverage. Please review.

Project coverage is 85.25%. Comparing base (356ae56) to head (3211b88).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
lightly/loss/detcon_loss.py 97.67% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1771      +/-   ##
==========================================
+ Coverage   85.08%   85.25%   +0.16%     
==========================================
  Files         162      163       +1     
  Lines        6747     6834      +87     
==========================================
+ Hits         5741     5826      +85     
- Misses       1006     1008       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lightly/loss/detcon_loss.py Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Show resolved Hide resolved
tests/loss/test_detcon_loss.py Outdated Show resolved Hide resolved
tests/loss/test_detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
@liopeer
Copy link
Contributor Author

liopeer commented Jan 6, 2025

Tests have been completely re-written.

  • now use proper mocking to simulate the distributed behaviour
  • randomized testing against the original implementation (jax.numpy -> numpy) and a few additional small changes (change details are as comments in the code)

Copy link
Contributor

@guarin guarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments but overall looks very good! Let's get this merged :)

lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
tests/loss/test_detcon_loss.py Outdated Show resolved Hide resolved
lightly/loss/detcon_loss.py Outdated Show resolved Hide resolved
@liopeer liopeer enabled auto-merge (squash) January 6, 2025 13:07
Copy link
Contributor

@guarin guarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@liopeer liopeer merged commit 0dcf780 into master Jan 6, 2025
14 checks passed
@liopeer liopeer deleted the lionel-lig-5628-add-detconloss branch January 6, 2025 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants