Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#9590: COG download metadata by default with abort fetch #9687

Merged
merged 4 commits into from
Nov 16, 2023

Conversation

dsuren1
Copy link
Contributor

@dsuren1 dsuren1 commented Nov 9, 2023

Description

This PR allows fetching of COG layer metadata by default on saving the service. Basic metadata informations are added to layer.

Some geotiff source functions are not exported. Hence alias has been created to import the functions.

Please check if the PR fulfills these requirements

What kind of change does this PR introduce? (check one with "x", remove the others)

  • Enhancement

Issue

What is the current behavior?

What is the new behavior?
Allows fetching of COG layer metadata by default and also allows cancelling of the fired request on processing the url and getImage process

Breaking change

Does this PR introduce a breaking change? (check one with "x", remove the other)

  • Yes, and I documented them in migration notes
  • No

Other useful information

@dsuren1 dsuren1 added enhancement Internal BackportNeeded Commits provided for an issue need to be backported to the milestone's stable branch labels Nov 9, 2023
@dsuren1 dsuren1 added this to the 2023.02.01 milestone Nov 9, 2023
@dsuren1 dsuren1 requested a review from offtherailz November 9, 2023 13:27
@dsuren1 dsuren1 self-assigned this Nov 9, 2023
@dsuren1 dsuren1 linked an issue Nov 9, 2023 that may be closed by this pull request
6 tasks
Copy link
Member

@offtherailz offtherailz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Delete button should be disabled, or not visible, while saving, or I'm able to delete the catalog while loading.
Video.del.14-11-2023.12.02.56.webm

@@ -194,7 +194,8 @@ module.exports = (...args) => mapArgumentsToObject(args, ({
jsonix: '@boundlessgeo/jsonix',
// next libs are added because of this issue https://github.com/geosolutions-it/MapStore2/issues/4569
proj4: '@geosolutions/proj4',
"react-joyride": '@geosolutions/react-joyride'
"react-joyride": '@geosolutions/react-joyride',
"geotiff-remote": path.join(paths.base, "node_modules", "geotiff", "dist-module", "source", "remote")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This solution is a big hack...

Did you try to implement it instead?
Like this as documented here, or better using axios (I'd prefer axios solution, so we have also authentication, proxy etc... out of the box, the doc do not show, for brevity, the solution with Promise) ?

const response = await fetch(someUrl);
const arrayBuffer = await response.arrayBuffer();
const tiff = await fromArrayBuffer(arrayBuffer);

If a solution like this is possible, we can have more control on it, without any hack.

Otherwise I think we have to rely on internal functions as you did in this implementation.

Copy link
Contributor Author

@dsuren1 dsuren1 Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I did check.
fromArrayBuffer still doesn't handle signal parameter correctly. It is lost when the passed down to fromSource. Reason it cannot handled with axios is because fromSource makes bigtiff parsing logic to read tif data and converts to image object with required functions necessary for extracting metadata informations.

Maybe we can cancel the promise of fromUrl and getImage but not the actual fetch request action (which is a compromise) and in the process we can avoid the hack and also this will not impact user performance. Kindly let me know if this okay? @offtherailz

@dsuren1
Copy link
Contributor Author

dsuren1 commented Nov 14, 2023

  • Delete button should be disabled, or not visible, while saving, or I'm able to delete the catalog while loading.

Looks like it is an existing issue for all services. I will address it.

@dsuren1 dsuren1 requested a review from offtherailz November 14, 2023 12:32
Copy link
Member

@offtherailz offtherailz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Now bbox information look not to be present anymore.

tested with this: http://localhost:8081/#/viewer/46403

screencast-localhost_8081-2023.11.16-12_57_17.webm
  • Comments inline

signal?.addEventListener("abort", () => abortError(reject));
return fromGeotiffUrl(url)
.then((image)=> image.getImage()) // Fetch and read first image to get medatadata of the tif
.then((image) => resolve(image))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, as discussed with @dsuren1 , we can canel the pending task to make the user continue operating if the "Save" takes too long, but we can not cancel the effective requests pending

This can cause the pending requests to be blocking and slowing down the load on that particular domain in any case (because of bandwitdh and max number of parallel requests allowed to a specific domain by the browser etc...)

Please report the known issue notice, or we loose memory of it.
Having an issue about it linking could be useful to track the problem too.
@tdipisa should we open it ?

I ask @dsuren1 to link the proper issue to github repo of geotiffjs, for completeness.

Suggested change
.then((image) => resolve(image))
.then((image) => resolve(image)) // note: the cacellation will not cancel the request because of issue in

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated note on the function and created an issue in geotiff.js

@dsuren1 dsuren1 requested a review from offtherailz November 16, 2023 12:54
@offtherailz offtherailz merged commit 601b15f into geosolutions-it:master Nov 16, 2023
@ElenaGallo
Copy link
Contributor

Test passed, @dsuren1 please backport to the stable branch. Thanks

dsuren1 added a commit to dsuren1/MapStore2 that referenced this pull request Nov 17, 2023
@ElenaGallo ElenaGallo removed the BackportNeeded Commits provided for an issue need to be backported to the milestone's stable branch label Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make COG download metadata by default
3 participants