Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension types not fully supported in list arrays #45262

Open
chelsea-lin opened this issue Jan 14, 2025 · 0 comments
Open

Extension types not fully supported in list arrays #45262

chelsea-lin opened this issue Jan 14, 2025 · 0 comments

Comments

@chelsea-lin
Copy link

Describe the bug, including details regarding any error messages, version, and platform.

Description

I'm trying to create a list array where each element is an array of a custom extension type, but I'm getting an ArrowNotImplementedError: extension.

Reproduction steps

Taken RelationType created in the pyarrow office doc as example (link),

import pyarrow as pa

class RationalType(pa.ExtensionType):
    # ... (Your RationalType code) ...

rational_type = RationalType(pa.int32())
data1 = [{"numer": 1, "denom": 17}, {"numer": 2, "denom": 13}]
data2 = [{"numer": 3, "denom": 34}, {"numer": 4, "denom": 60}]

pa.array(data1, type=rational_type)  # Works fine
pa.array([data1, data2], type=pa.list_(rational_type))  # Raises ArrowNotImplementedError

Actual behavior:

The code raises the following error:

---------------------------------------------------------------------------
ArrowNotImplementedError                  Traceback (most recent call last)
Cell In[33], [line 1](vscode-notebook-cell:?execution_count=33&line=1)
----> [1](vscode-notebook-cell:?execution_count=33&line=1) pa.array([data1, data2], type=pa.list_(rational_type))

File ~/src/python-db-dtypes-pandas/venv/lib/python3.12/site-packages/pyarrow/array.pxi:370, in pyarrow.lib.array()

File ~/src/python-db-dtypes-pandas/venv/lib/python3.12/site-packages/pyarrow/array.pxi:42, in pyarrow.lib._sequence_to_array()

File ~/src/python-db-dtypes-pandas/venv/lib/python3.12/site-packages/pyarrow/error.pxi:155, in pyarrow.lib.pyarrow_internal_check_status()

File ~/src/python-db-dtypes-pandas/venv/lib/python3.12/site-packages/pyarrow/error.pxi:92, in pyarrow.lib.check_status()

ArrowNotImplementedError: extension

Environment

  • pyarrow version: 18.1.0
  • Python version: 3.12.7

Component(s)

Python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant