Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validation is changing every epoch in PathIterable #343

Open
jdeschamps opened this issue Jan 9, 2025 · 2 comments
Open

Validation is changing every epoch in PathIterable #343

jdeschamps opened this issue Jan 9, 2025 · 2 comments
Labels
bug Something isn't working
Milestone
v0.1.0

Comments

@jdeschamps
Copy link
Member

Describe the bug

Not a bug, but a consequence of the choices we've made a long time ago: when training with multiple files, CAREamics will instantiate PathIterable datasets for both training and validation.

PathIterable randomly fetch patches from the images, yielding different validation patches each epoch, which in itself is not how we would want to assess training progress.

@CatEek do you agree with the diagnostic?
@jdeschamps jdeschamps added the bug Something isn't working label Jan 9, 2025
@jdeschamps jdeschamps mentioned this issue Jan 9, 2025
Open
@jdeschamps
Copy link
Member Author

Potentially fixable in #292

@jdeschamps jdeschamps changed the title Validation is changing every epoch Validation is changing every epoch in PathIterable Jan 9, 2025
@jdeschamps jdeschamps added this to the v0.1.0 milestone Jan 20, 2025
@jdeschamps
Copy link
Member Author

Potential quick-fixes:

  • Adding a seed to the patching
  • Forcing InMemoryDataset for validation

@jdeschamps jdeschamps moved this to Backlog in v0.1.0 Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
v0.1.0
Status: Backlog
Milestone
v0.1.0
Development

No branches or pull requests
1 participant
@jdeschamps