Dataset Recognition

[kozlov721]

Dataset Recognition

• Split parsers into separate classes
  • The parsers are more low-level than before, the high-level usage is provided by the new LuxonisParaser class
• Added validate and validate_split methods for use with dataset recognition
• Added a new LuxonisParser class that automatically recognizes dataset type of given dataset directory and parses its content using the correct parser
• Added test cases for all the supported parsers

Example Usage

parser = LuxonisParser(
    "gs://luxonis-test-bucket/luxonis-ml-test-data/COCO_people_subset",
    dataset_name="coco_test",
    delete_existing=True,
)
dataset = parser.parse()

[conorsim]

Looks great from a code perspective. We'll wait to fix the workflow issue for the tests

[conorsim]

Not sure if it's an issue with my local environment but when I run the last cell with the loader I get

KeyError: <LabelType.BOUNDINGBOX: 'boxes'>

[kozlov721]

You might need to first delete the data directory if it already exists. I changed some things, but it won't re-download if the zip exists.

[github-actions]

Test Results

43 tests   43 ✅  29s ⏱️
 1 suites   0 💤
 1 files     0 ❌

Results for commit 9e14995.

♻️ This comment has been updated with latest results.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
3599	1780	49%	0%	🟢

New Files

File	Coverage	Status
luxonis_ml/data/parsers/init.py	100%	🟢
luxonis_ml/data/parsers/base_parser.py	90%	🟢
luxonis_ml/data/parsers/classification_directory_parser.py	97%	🟢
luxonis_ml/data/parsers/coco_parser.py	80%	🟢
luxonis_ml/data/parsers/create_ml_parser.py	95%	🟢
luxonis_ml/data/parsers/darknet_parser.py	98%	🟢
luxonis_ml/data/parsers/luxonis_parser.py	89%	🟢
luxonis_ml/data/parsers/segmentation_mask_directory_parser.py	93%	🟢
luxonis_ml/data/parsers/tensorflow_csv_parser.py	98%	🟢
luxonis_ml/data/parsers/voc_parser.py	97%	🟢
luxonis_ml/data/parsers/yolov4_parser.py	98%	🟢
luxonis_ml/data/parsers/yolov6_parser.py	96%	🟢
TOTAL	94%	🟢

Modified Files

File	Coverage	Status
luxonis_ml/guard_extras.py	75%	🟢
luxonis_ml/utils/filesystem.py	73%	🟢
TOTAL	74%	🟢

updated for commit: 9e14995 by action🐍