Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] merge test_probabilistic_metrics into TestAllDistrMetrics #398

Open
fkiraly opened this issue Jun 17, 2024 · 4 comments
Open

[ENH] merge test_probabilistic_metrics into TestAllDistrMetrics #398

fkiraly opened this issue Jun 17, 2024 · 4 comments
Labels
good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests

Comments

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 17, 2024

The module test_probabilistic_metrics contains tests for interval and quantile distributions with manual collection. These metrics are also automatically collected in TestAllDistrMetrics, which should be the location where these tests are carried out systematically.

Hence, we should move the existing tests in test_probabilistic_metrics to there.

Whether a metric is an interval or quantile metric can be inspectedby reading the "scitype:y_pred" tag.

@fkiraly fkiraly added good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests labels Jun 17, 2024
@fkiraly
Copy link
Collaborator Author

fkiraly commented Jun 17, 2024

FYI @benHeid, turns out skpro is halfway to the point of a fully programmatic test suite for the proba metrics. The quantile and interval metrics will miss out on some interface tests when checked via check_estimator.

@JahnaviDhanaSri
Copy link

Hi @fkiraly,

I am new to open-source contributions, I have understood this issue and would like to work on it. Could you please assign it to me?

@fkiraly
Copy link
Collaborator Author

fkiraly commented Jan 9, 2025

@JahnaviDhanaSri, sure - all yours!

@JahnaviDhanaSri
Copy link

Hi @fkiraly , I have raised the pull request could you please review it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests
Projects
None yet
Development

No branches or pull requests

2 participants