Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LVAE dataset: add MultiFileDset #223

Merged
merged 21 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
74cfcd5
extract index manager and switcher to separate files; add empty patch…
veegalinova Aug 23, 2024
60c9c95
move files around
veegalinova Aug 23, 2024
27f06fe
add missing inits
veegalinova Aug 23, 2024
73b2261
add changes from Disentangle for vae_dataset and index_manager
veegalinova Aug 23, 2024
9b54ef0
add multifile dataset
veegalinova Aug 23, 2024
c7310a3
remove biosr and mrc logic
veegalinova Aug 23, 2024
faad30f
add multifile dataset test
veegalinova Aug 23, 2024
386cb2a
Merge branch 'main' into vg/feat/multifile_dataset
veegalinova Aug 24, 2024
ad6e7de
move albumentations import to prevent tests from failing
veegalinova Aug 24, 2024
8ec586e
re-enable tests
veegalinova Aug 24, 2024
a1b1abc
Merge branch 'main' into vg/feat/multifile_dataset
jdeschamps Sep 2, 2024
53047dd
add load_data_fn parameter to datasets
veegalinova Sep 4, 2024
ccafc1f
rename vae dataset back to MultiChDataset for compatibility
veegalinova Sep 4, 2024
e8e76cb
Merge branch 'main' into vg/feat/multifile_dataset
jdeschamps Sep 9, 2024
58c050a
update lvae_tiled_patching with TilingMode parameter
veegalinova Sep 9, 2024
dc3b5c6
merge main branch
veegalinova Sep 9, 2024
7644c37
update new tests with pytest tag
veegalinova Sep 9, 2024
5dd4d30
update multifile dataset with TwoChannelData and MultiChannelData
veegalinova Sep 9, 2024
9bd8c85
Merge branch 'main' into vg/feat/multifile_dataset
jdeschamps Sep 10, 2024
7b4cb1d
update datasets with target normalization; file merge and reorganizat…
veegalinova Sep 18, 2024
d22cd2f
Merge branch 'main' into vg/feat/multifile_dataset
veegalinova Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file.
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
from typing import Optional
from pydantic import ConfigDict

from pydantic import ConfigDict, computed_field

from careamics.lvae_training.dataset.vae_data_config import VaeDatasetConfig
from careamics.lvae_training.dataset.configs.vae_data_config import VaeDatasetConfig


class LCVaeDatasetConfig(VaeDatasetConfig):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,16 @@ class DataSplitType(Enum):
Test = 3


class GridAlignement(Enum):
class TilingMode(Enum):
"""
A patch is formed by padding the grid with content. If the grids are 'Center' aligned, then padding is to done equally on all 4 sides.
On the other hand, if grids are 'LeftTop' aligned, padding is to be done on the right and bottom end of the grid.
In the former case, one needs (patch_size - grid_size)//2 amount of content on the right end of the frame.
In the latter case, one needs patch_size - grid_size amount of content on the right end of the frame.
Enum for the tiling mode.
"""

LeftTop = 0
Center = 1
TrimBoundary = 0
PadBoundary = 1
ShiftBoundary = 2


# TODO: for all bool params check if they are taking different values in Disentangle repo
# TODO: check if any bool logic can be removed
class VaeDatasetConfig(BaseModel):
model_config = ConfigDict(validate_assignment=True)
Expand Down Expand Up @@ -132,15 +129,10 @@ class VaeDatasetConfig(BaseModel):
# TODO: why is this not used?
enable_rotation_aug: Optional[bool] = False

grid_alignment: GridAlignement = GridAlignement.LeftTop

max_val: Optional[float] = None
"""Maximum data in the dataset. Is calculated for train split, and should be
externally set for val and test splits."""

trim_boundary: Optional[bool] = True
"""Whether to trim boundary of the image"""

overlapping_padding_kwargs: Any = None
"""Parameters for np.pad method"""

Expand All @@ -160,6 +152,21 @@ class VaeDatasetConfig(BaseModel):
# TODO: not used?
multiscale_lowres_count: Optional[int] = None

tiling_mode: Optional[TilingMode] = TilingMode.ShiftBoundary

target_separate_normalization: Optional[bool] = True

mode_3D: Optional[bool] = False
"""If training in 3D mode or not"""

trainig_datausage_fraction: Optional[float] = 1.0

validtarget_random_fraction: Optional[float] = None

validation_datausage_fraction: Optional[float] = 1.0

random_flip_z_3D: Optional[bool] = False

@computed_field
@property
def padding_kwargs(self) -> dict:
Expand Down
Loading
Loading