Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In this PR, we fixed a bug in which the creation of the range filter that creates a QuerySpace wasn't taking into account the possibility of column transformations in Spark.
For example, imagine that we index the parameter named
column1
. If we filter by:regex("prefix", "column1", 1) == "a"
Instead of filtering by the regex expression,
QuerySpaceBuilder
would understand that we are trying to filter all the values that containcolumn1 == "a"
Type of change
Describe the change you're making: how it affects the API, and user experience...
Is a bug fix.
Checklist:
Here is the list of things you should do before submitting this pull request:
How Has This Been Tested? (Optional)
Please describe the tests that you ran to verify your changes.
Change test that (unfortunately) passed because the filter range belonged to the same file (
prefix_versace
andversace
have very similar values of hash and had been stored together).Now the test checks if we return the whole dataset in that type of scenarios.
Later on, we can investigate how to manipulate more complex predicates. But let's start with the basics.
Test Configuration: