Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce code repetition in datafusion/functions mod files #10700

Merged
merged 13 commits into from
Jun 3, 2024

Conversation

MohamedAbdeen21
Copy link
Contributor

@MohamedAbdeen21 MohamedAbdeen21 commented May 28, 2024

Which issue does this PR close?

Possibly closes #10397.

Rationale for this change

Extending the export_function! macro to be used in math and core functions modules, reducing code repetition.

Some functions expect a single Expr arg and others expect a single Vec<Expr> args. I took a page from Python's 1-tuples (where (1) is evaluated to int(1), but (1,) is evaluated to tuple(1) and now single args followed by a comma are treated as a Vec<Expr> but single args without trailing commas are treated as Expr.

Variadic functions are unchanged arg1 arg2 ..., but should not contain commas.

I left a comment regarding the special case of get_field.

What changes are included in this PR?

Extending the export_function! macro to accept both variadic functions and functions that take single Vec<Expr> argument. Uses the rust forums link made by @jayzhan211 https://users.rust-lang.org/t/macro-repetition-with-multiple-rules/110816/2?u=jayzhan

Are these changes tested?

Passes existing tests

Are there any user-facing changes?

No

),(
coalesce,
"Returns `coalesce(args...)`, which evaluates to the value of the first expr which is not NULL",
args,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single arguments followed by a single comma indicate a vector argument. The only downside here is that rust-analyzer is unable to identify the type of args and the dev is expected to read the doc of export_functions! to notice that args is a vector and not an Expr.

($(($FUNC:ident, $DOC:expr, $($arg:tt)*)),*) => {
$(
// switch to single-function cases below
export_functions!(single $FUNC, $DOC, $($arg)*);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little hack to avoid making a separate macro.

@MohamedAbdeen21 MohamedAbdeen21 changed the title Reduce repetition in math and functions modules with macros Reduce code repetition in datafusion/functions mod files May 30, 2024
@MohamedAbdeen21 MohamedAbdeen21 force-pushed the reduce-repetition-with-macros branch from 4fc8a18 to 15bfd36 Compare June 1, 2024 20:41
Copy link
Contributor

@jayzhan211 jayzhan211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

)*
};

// single vector argument (a single argument followed by a comma)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is pretty fancy 👍

@alamb alamb merged commit 3aae451 into apache:main Jun 3, 2024
23 checks passed
@MohamedAbdeen21 MohamedAbdeen21 deleted the reduce-repetition-with-macros branch June 3, 2024 18:52
@MohamedAbdeen21
Copy link
Contributor Author

Thanks for the review! @jayzhan211 @alamb

findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
)

* initial reduce repetition using macros

* formatting and docs

* fix docs

* refix doc

* replace math mod too

* fix vec arguments

* fix math variadic args

* apply to functions

* pattern-match hack to avoid second macro

* missed a function

* fix merge conflict

* fix octet_length argument
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reduce repetition in datafusion::functions using macros
3 participants