Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust knn merge stability testing #14172

Merged
merged 6 commits into from
Jan 29, 2025

Conversation

benwtrent
Copy link
Member

@benwtrent benwtrent commented Jan 27, 2025

With the new connected components work, merging is unstable. It generally seems that between indexing, there might be new connections as vex index size increases.

I am not 100% sure the cause here. Opening this PR as one potential solution. Its interesting indeed that it occurs at all.

closes: #13640

@benwtrent benwtrent requested a review from msokolov January 27, 2025 17:47
Copy link
Contributor

@msokolov msokolov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems OK to mark this as unstable, but something is failing in the checks, maybe related to the print statements

@@ -191,6 +193,9 @@ public PostingsFormat getPostingsFormatForField(String name) {
// Safety:
assert previousMappings.size() < 10000 : "test went insane";
}
if (VERBOSE) {
System.out.println("RandomPostingsFormat: " + name + " -> " + codec);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't System.out forbidden? Or is it OK in tests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's likely forbidden. I found this verbose output helpful in debugging. But I think it can be removed in preference to setting the info stream

I will fix up the PR and merge.

@benwtrent benwtrent requested a review from msokolov January 29, 2025 13:57
@benwtrent benwtrent merged commit de4f07b into apache:main Jan 29, 2025
5 checks passed
@benwtrent benwtrent deleted the test/adj-knn-merge-stability branch January 29, 2025 16:10
benwtrent added a commit that referenced this pull request Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

testMergeStability failing for Knn formats
2 participants