Skip to content

Commit

Permalink
fix: Run filter null join key optimization once (#3657)
Browse files Browse the repository at this point in the history
Only run the filter null join key optimization once before filter
pushdown, instead of 3 times together with the other rules, otherwise
you end up with something like `[not(is_null(key)) & not(is_null(key))]
& not(is_null(key))`.

Co-authored-by: Colin Ho <[email protected]>
  • Loading branch information
colin-ho and Colin Ho authored Jan 9, 2025
1 parent 43bbbeb commit bb85070
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion src/daft-logical-plan/src/optimization/optimizer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,16 @@ impl Default for OptimizerBuilder {
vec![Box::new(SimplifyExpressionsRule::new())],
RuleExecutionStrategy::FixedPoint(Some(3)),
),
// --- Filter out null join keys ---
// This rule should be run once, before any filter pushdown rules.
RuleBatch::new(
vec![Box::new(FilterNullJoinKey::new())],
RuleExecutionStrategy::Once,
),
// --- Bulk of our rules ---
RuleBatch::new(
vec![
Box::new(DropRepartition::new()),
Box::new(FilterNullJoinKey::new()),
Box::new(PushDownFilter::new()),
Box::new(PushDownProjection::new()),
Box::new(EliminateCrossJoin::new()),
Expand Down

0 comments on commit bb85070

Please sign in to comment.