Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for string view to a few functions #13330

Open
6 tasks
timsaucer opened this issue Nov 9, 2024 · 10 comments · May be fixed by #13332
Open
6 tasks

Add support for string view to a few functions #13330

timsaucer opened this issue Nov 9, 2024 · 10 comments · May be fixed by #13332
Labels
enhancement New feature or request

Comments

@timsaucer
Copy link
Contributor

Is your feature request related to a problem or challenge?

These functions are failing when passing in a string view instead of a string. Now that string view is the default type, it is causing some regressions on existing code that have to be modified.

Describe the solution you'd like

Add options of passing in string view to these:

  • ArrayToString
  • NamedStructFunc
  • DateBinFunc (I couldn't quite track down where the Utf->Interval or Timestamp conversion occurs
  • BitLengthFunc
  • ConcatFunc (gives error on optimizer not matching schema)
  • EncodeFunc

Describe alternatives you've considered

No response

Additional context

These were discovered during the datafusion-python migration to DF43.0.0. This commit is where I made adjustments in that repo to account for these: apache/datafusion-python@fcb5f96

@timsaucer
Copy link
Contributor Author

Related: #13347

@Omega359
Copy link
Contributor

regexp_match looks to not support stringview either.

// TODO: Native support Utf8View for regexp_match.
                    TypeSignature::Exact(vec![Utf8, Utf8]),
                    TypeSignature::Exact(vec![LargeUtf8, LargeUtf8]),
                    TypeSignature::Exact(vec![Utf8, Utf8, Utf8]),
                    TypeSignature::Exact(vec![LargeUtf8, LargeUtf8, LargeUtf8]),

@Omega359
Copy link
Contributor

Omega359 commented Nov 11, 2024

regexp_match filed as #13357 - was a duplicate of already filed #11911

@timsaucer
Copy link
Contributor Author

I'm going to recommend a 43.1.0 release after we get all these resolved

@Omega359
Copy link
Contributor

Omega359 commented Nov 11, 2024

array_to_string also does not appear to support Utf8View

@Omega359
Copy link
Contributor

None of the crypto functions support UTF8View directly as a input type. For example digest:

signature: Signature::one_of(
                vec![
                    Exact(vec![Utf8, Utf8]),
                    Exact(vec![LargeUtf8, Utf8]),
                    Exact(vec![Binary, Utf8]),
                    Exact(vec![LargeBinary, Utf8]),
                ],
                Volatility::Immutable,
            ),

@Omega359
Copy link
Contributor

NullIf:

static SUPPORTED_NULLIF_TYPES: &[DataType] = &[
    DataType::Boolean,
    DataType::UInt8,
    DataType::UInt16,
    DataType::UInt32,
    DataType::UInt64,
    DataType::Int8,
    DataType::Int16,
    DataType::Int32,
    DataType::Int64,
    DataType::Float32,
    DataType::Float64,
    DataType::Utf8,
    DataType::LargeUtf8,
];

@Omega359
Copy link
Contributor

nvl:

static SUPPORTED_NVL_TYPES: &[DataType] = &[
    DataType::Boolean,
    DataType::UInt8,
    DataType::UInt16,
    DataType::UInt32,
    DataType::UInt64,
    DataType::Int8,
    DataType::Int16,
    DataType::Int32,
    DataType::Int64,
    DataType::Float32,
    DataType::Float64,
    DataType::Utf8,
    DataType::LargeUtf8,
];

@Omega359
Copy link
Contributor

crypto PR - #13407
array_to_string/string_to_array PR - #13403
nvl PR - #13382
nullif PR - #13380

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants