expand test column dtypes to full scale #492
Conversation
hussain-jafari
requested review from
albrja,
patricktnast,
rmudambi and
stevebachmeier
as code owners
| @@ -49,6 +48,25 @@ def test_row_noising_omit_row_or_do_not_respond( | |||
| run_omit_row_or_do_not_respond_tests(dataset_name, config, original_data, noised_data) | |||
|
|
|||
|
|
|||
| def test_column_dtypes( | |||
There was a problem hiding this comment.
I've asked this multiple times but remind me again - how will we run this test as it previously existed (on sample data) but NOT during release-testing? i.e. we need to continue running the previous test every night like we currently are.
There was a problem hiding this comment.
The fixtures are set up to read in sample data if we don't run pytest with the --release flag.
| # str dtype is 'object' | ||
| # Check that they are actually strings and not some other | ||
| # type of object. | ||
| actual_types = noised_data[col.name].dropna().apply(type) |
There was a problem hiding this comment.
Is using apply here vectorized? I don't think it is, but maybe we don't have any other options?
There was a problem hiding this comment.
I believe it's not vectorized, but it might be the fastest way according to this answer:
https://stackoverflow.com/questions/55754713/fastest-way-to-find-all-data-types-in-a-pandas-series
hussain-jafari
deleted the
hjafari/feature/MIC-5866_expand_test_column_dtypes
branch
expand test column dtypes to full scale
Description=
Move test_column_dtypes to release suite framework.
Testing
Ran tests on 5 datasets (acs, cps, wic, ssa and census)