Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rqatrend allocationfree #45

Merged
merged 8 commits into from
Jan 15, 2025
Merged

Rqatrend allocationfree #45

merged 8 commits into from
Jan 15, 2025

Conversation

schlichtanders
Copy link
Collaborator

here the code to make rqatrend allocation free

@felixcremer
Copy link
Collaborator

Nice, that works now for me on the VM and I started using it for processing the full tiles.
Let's see, whether this is going to fully work or whether this will also run into memory problems.

@felixcremer
Copy link
Collaborator

I tested this with the main function on the EODC-VM and unfortunately this still runs into out of memory problems.
It seems to be a bit faster than before, but we still would have to run every parameter combination separately or I could play around different thread and worker numbers.

@danlooo
Copy link
Collaborator

danlooo commented Jan 15, 2025

@felixcremer Can you estimate the memory needed to process 1 year with 1 tile? Maybe we need to split the tile to avoid out of memory issues. Our github runner VM currently has only 8GB of RAM. However, there are more resources in other VMs.

@felixcremer
Copy link
Collaborator

That is hard to estimate, because it depends a lot on the number of workers and threads. We would have to experiment on that a bit.

@danlooo danlooo merged commit 322b23f into main Jan 15, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants