Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support classwise metrics #2121

Open
robmarkcole opened this issue Jun 15, 2024 · 3 comments
Open

Support classwise metrics #2121

robmarkcole opened this issue Jun 15, 2024 · 3 comments
Labels
trainers PyTorch Lightning trainers

Comments

@robmarkcole
Copy link
Contributor

Summary

This FR is to add support for classwise metrics to the segmentation trainer

Rationale

There can be many classes in a dataset and we want class specific metrics

Implementation

Using the approach of @isaaccorley as here with ClasswiseWrapper and enabled by an optional arg to the segmentation trainer, classwise_metrics: bool = False. To support this an additional labels: Optional[List[str]] = None arg will also be added

Alternatives

None

Additional information

No response

@adamjstewart
Copy link
Collaborator

Why not always compute both macro and micro statistics in all trainers? I would welcome a PR that adds this.

@robmarkcole
Copy link
Contributor Author

In addition to class wise or instead of? I think that would be nice to have the complete set - with the classwise as optional otherwise the list of metrics can get very long

@adamjstewart
Copy link
Collaborator

Ah sorry, yeah I meant macro + micro + classwise. As long as that doesn't noticeably slow down training time it should be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
trainers PyTorch Lightning trainers
Projects
None yet
Development

No branches or pull requests

2 participants