Do we verify the integrity of input products upon data arrival?

[riverma]

Hi @maseca @chrisjrd @hhlee445 -

We had a discussion (spawned by @hookhua) on how we may be able to verify input products stored on LP.DAAC is byte-by-byte identical to what we pull down via our downloaders. Given we don't see checksums available in CMR for our LP.DAAC input data products, we'd need another method to check for this.

The AWS S3 GET API has something called the x-amz-checksum-mode to auto-retrieve the checksum of a remote file. More documentation here.

Question for the PCM team: do the boto3-based product download jobs already check for checksums via the above mechanism? If not - can they be made to?