Feat(FM): Listing Duplicated Files

[williamputraintan]

There might be cases where files are the same but located in different locations. This could be due to a moving process or different storage tiers. Perhaps having a flag in the API to indicate only files in the best storage tier should be shown when this happens.

[mmalenic]

If querying by portalRunId or ingestId multiple results can be returned for identical files, and I guess that's what's happening in the UI?

Also, by best, would that mean preferencing Standard or instantly retrievable classes over DeepArchive or classes that need a restore first? There is an ?isAccessible=True flag which was introduced recently with #851. Could that solve the issue?

[williamputraintan]

It's not an issue yet, just thinking about the possibility of happening, as during moving files between buckets (copy and delete), then 2 identical files may be listed. Perhaps it is a very small window that the user just needs to refresh and duplicated files are gone. Not sure if we will have any duplicated objects between buckets again where the only difference is just the storage tier, but if it does I am just wondering if we could de-duplicate by listing the preferred tier over the other.

There were cases of duplicated filenames, but it turns out that duplication is due to the workflow having the same file in multiple directories.

Standard or instantly retrievable classes over DeepArchive or classes that need a restore first?

Yes, I think standard or Instant Retrieval is preferred.

?isAccessible=True flag

Thanks for this, the UI should make use of it.

[williamputraintan]

I am happy to close this issue for now if it doesn't make sense as we don't have a strong use case yet.