Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: The polarsArray Join operation does not behave like documented #10910

Open
1 task done
GLeurquin opened this issue Feb 27, 2025 · 3 comments · May be fixed by #10913
Open
1 task done

bug: The polarsArray Join operation does not behave like documented #10910

GLeurquin opened this issue Feb 27, 2025 · 3 comments · May be fixed by #10913
Labels
bug Incorrect behavior inside of ibis

Comments

@GLeurquin
Copy link

What happened?

The polars ArrayJoin operation does not behave like documented

When the array is empty, the expected value is NULL, but the result is an empty string. (ID 3 below).

ibis.set_backend("polars")
table = ibis.memtable(
    [
        {"id": 1, "arr": ["a", "b", "c"], "expected": "a|b|c"},
        {"id": 2, "arr": None, "expected": None},
        {"id": 3, "arr": [], "expected": None},
        {"id": 4, "arr": ["b", None], "expected": "b"},
    ]
)

table.mutate(
    result=_["arr"].join("|"),
).preview()
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┓
┃ id    ┃ arr                ┃ expected ┃ result ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━┩
│ int64 │ array<string>      │ string   │ string │
├───────┼────────────────────┼──────────┼────────┤
│     1 │ ['a', 'b', ... +1] │ a|b|c    │ a|b|c  │
│     2 │ NULL               │ NULL     │ NULL   │
│     3 │ []                 │ NULL     │ ~      │ <-- issue here
│     4 │ ['b', None]        │ b        │ b      │
└───────┴────────────────────┴──────────┴────────┘

Proposed change for code:

@translate.register(ops.ArrayStringJoin)
def array_string_join(op, **kw):
    arg = translate(op.arg, **kw)
    sep = _literal_value(op.sep)
    return (
        pl.when(arg.list.len() > 0)
        .then(arg.list.join(sep))
        .otherwise(None)
    )

What version of ibis are you using?

10.1.0

What backend(s) are you using, if any?

polars

Relevant log output

Code of Conduct

  • I agree to follow this project's Code of Conduct
@GLeurquin GLeurquin added the bug Incorrect behavior inside of ibis label Feb 27, 2025
@cpcloud
Copy link
Member

cpcloud commented Feb 27, 2025

Thanks for the issue, seems like a reasonable change in implementation!

@cpcloud
Copy link
Member

cpcloud commented Feb 27, 2025

@GLeurquin Are you interested in making a PR with that change?

@GLeurquin
Copy link
Author

Here is a PR: #10913

However I don't quite understand how to add the test example to the tests to avoid regression in the future, if you can guide me to where I should add it I can modify the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis
Projects
Status: backlog
Development

Successfully merging a pull request may close this issue.

2 participants