Support interleave kernel with deep copy buffers for StringViewArray

[2010YOUY01]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

arrow-rs/arrow-select/src/interleave.rs

Line 238 in a0c3186

fn interleave_views<T: ByteViewType>(

Current implementation of interleave kernel on StringViewArray reuses the existing buffers from source arrays, it's only reconstructing views specified by the interleaving operation. It's useful to also support interleave by reconstructing buffers. (For example, if the referenced elements are sparse)

Describe the solution you'd like

Provides an additional interface for StringViewArray to do interleave with deep copy buffers.

Describe alternatives you've considered

Additional context

[tustvold]

I wonder if instead of baking this into the interleave kernel, and potentially others besides, it would be better for applications to call https://docs.rs/arrow-array/latest/arrow_array/array/struct.GenericByteViewArray.html#method.gc when appropriate?

This would have a number of advantages:

• Don't need to modify multiple kernels
• Applications can chain kernels and only perform GC once at the end
• Applications are in control of any heuristics as to when this should take place

[2010YOUY01]

I wonder if instead of baking this into the interleave kernel, and potentially others besides, it would be better for applications to call https://docs.rs/arrow-array/latest/arrow_array/array/struct.GenericByteViewArray.html#method.gc when appropriate?

This would have a number of advantages:

• Don't need to modify multiple kernels
• Applications can chain kernels and only perform GC once at the end
• Applications are in control of any heuristics as to when this should take place

Thanks for the perspectives. We're currently using gc(). Performing the copy within the interleave kernel is just slightly more efficient.
If adding an additional interface inside arrow isn't ideal, we can implement this fused logic at the application level.

[alamb]

Related :

• Memory consumption grows for interleave kernel when input is StringViewArray #7151