Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create validations for source data #1527

Open
Tracked by #1508
manasaV3 opened this issue Jan 24, 2025 · 0 comments
Open
Tracked by #1508

Create validations for source data #1527

manasaV3 opened this issue Jan 24, 2025 · 0 comments
Assignees
Labels
backend P0 Must have

Comments

@manasaV3
Copy link
Contributor

manasaV3 commented Jan 24, 2025

Relates to: #1418

Rationale

This feature expects the source data to have the required related entities before it can be ingested to surface the frames and ctf metadata. To ensure that the source data is in a valid state, we need to add a validation step that can determine if the data is accurate before the ingestion.

Tasks

Validations for Frames

  1. For every run with Frames there should be an associated mdoc file

Validations for FramesAcquisitionFile

  1. There should never be more than one mdoc associated with a run
  2. The number of sections in the mdoc file should match the number of frames
  3. The SubFramePaths in the mdoc file should match existing frames (Note: Only expect file name to match to the frame file names)
  4. tilt axis angle from mdoc file ~approximately (+/- 10 deg) matches that in tilt series metadata
  5. number of sub-frames listed in mdoc file matches frame files (P1)

Validations for RawTlt Files

  1. The number of rawtlt angles should match the number of z-sections in the tiltseries
  2. number of rawtlt angles should never be greater than the number of mdoc sections
    3.If the is_aligned is not set for a tiltseries, validate that all rawtlt angles are approximately (+/- 1e-2) appearing in mdoc file
  3. The tilt angles in rawtlt file should match the tilt angles in the mdoc file

Validations for TiltSeries

  1. For every run with a tiltseries, there exists a mdoc and raw tlt file

What Done Looks Like?

We are able to run validation for any source data from command line

Notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend P0 Must have
Projects
None yet
Development

No branches or pull requests

2 participants