Skip to content

Commit

Permalink
cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
Colin Ho authored and Colin Ho committed Jan 17, 2025
1 parent 35adce9 commit e60049a
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 11 deletions.
4 changes: 2 additions & 2 deletions daft/dataframe/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ def iter_rows(
entire partitions of data, see: :meth:`df.iter_partitions() <daft.DataFrame.iter_partitions>`.
By default, Daft will convert the columns to Python lists for easy consumption. However, for nested data such as List or Struct arrays, this can be expensive.
You may wish to set `column_format` to "arrow" such that the nested data is returned as an Arrow array.
You may wish to set `column_format` to "arrow" such that the nested data is returned as Arrow scalars.
.. NOTE::
A quick note on configuring asynchronous/parallel execution using `results_buffer_size`.
Expand Down Expand Up @@ -296,7 +296,7 @@ def iter_rows(
Args:
results_buffer_size: how many partitions to allow in the results buffer (defaults to the total number of CPUs
available on the machine).
column_format: the format of the columns to iterate over. One of "python", "arrow", or "numpy". Defaults to "python".
column_format: the format of the columns to iterate over. One of "python" or "arrow". Defaults to "python".
.. seealso::
:meth:`df.iter_partitions() <daft.DataFrame.iter_partitions>`: iterator over entire partitions instead of single rows
Expand Down
11 changes: 2 additions & 9 deletions tests/dataframe/test_iter.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,18 +101,11 @@ def compare_values(v1, v2):
assert compare_values(actual_row, expected_row)


@pytest.mark.parametrize(
"format",
[
"arrow",
"numpy",
],
)
def test_iter_rows_column_format_not_compatible(format):
def test_iter_rows_arrow_column_format_not_compatible():
df = daft.from_pydict({"a": [object()]}) # Object type is not supported by arrow or numpy

with pytest.raises(ValueError):
list(df.iter_rows(column_format=format))
list(df.iter_rows(column_format="arrow"))


@pytest.mark.parametrize("materialized", [False, True])
Expand Down

0 comments on commit e60049a

Please sign in to comment.