Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1056 download from daac to s3 #1061

Merged
merged 4 commits into from
Jan 21, 2025
Merged

Conversation

philipjyoon
Copy link
Contributor

Refilsl all the lost DISP-S1 products from ASF back into the S3 bucket. Performs the following:

  • Take in a text file w list of frame IDs
  • Perform CMR query (assuming these are being cataloged to CMR) to find all products for each frame ID from ASF-UAT
  • Compare to see if they exist in the S3 already
  • If they don't exist, then copy the file over

This has been tested for several frames from ASF UAT to INT-POP1 S3 and the PST confirmed that it worked correctly.

Reuses some code from the opera_validator and it had to be refactored a bit to be useful.

Copy link
Collaborator

@collinss-jpl collinss-jpl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, just had one question about a loop that maybe isn't needed?


# NOTE: This time filter isn't really used - we hard-coded it to 1999 to 2099 for now. If at some point we want to
# filter by time as well, we will need to configure this a bit to make it work
for attrib in disp_s1.get("umm").get("AdditionalAttributes"):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this loop over "AdditionalAttributes" is needed. attrib doesn't seem to be used anywhere else.

@philipjyoon philipjyoon merged commit bb06e9a into develop Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants