Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rsc: Track hidden output dirs #1651

Merged
merged 1 commit into from
Sep 24, 2024
Merged

rsc: Track hidden output dirs #1651

merged 1 commit into from
Sep 24, 2024

Conversation

V-FEXrt
Copy link
Contributor

@V-FEXrt V-FEXrt commented Sep 23, 2024

Updates the RSC and clients to track "hidden" directories which are directories created by a job but not output by the job.

This is necessary as some jobs will output a file and depends on the directory being created without also outputting said directory

Copy link
Contributor

@colinschmidt colinschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me.

| addErrorContext "rsc: Failed to stat files to upload"
def filteredOutputs = output.getRunnerOutputOutputs
def cleanableOutputs = output.getRunnerOutputCleanableOutputs
def hiddenOutputs = subtract scmp cleanableOutputs filteredOutputs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do this subtract after the grouping so one of the lists is smaller?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that helps, we have to stat the full union of filteredOutputs and cleanableOutputs in order to determine which are symlinks and I think that will be the majority of the time spent in the function (fwiw we were always stating the full list so it shouldn't be a time regression)

Base automatically changed from rsc-disallowed-hidden-file to master September 24, 2024 21:30
@V-FEXrt V-FEXrt merged commit b7f811b into master Sep 24, 2024
11 checks passed
@V-FEXrt V-FEXrt deleted the rsc-track-hidden-dirs branch September 24, 2024 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants