Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dev UI] Available datasets should be scoped to the target being tested #1554

Open
MichaelDoyle opened this issue Dec 19, 2024 · 0 comments
Open
Labels

Comments

@MichaelDoyle
Copy link
Member

Overview

Currently, it's possible to create a "model" dataset, and try to evaluate a "flow" with that dataset. Fundamentally, these are incompatible and will cause an error.

User goal(s)

Help choosing relevant datasets.

Requirements

  1. At minimum, we should scope the target under test to datasets of the same type.
  2. Within the same type (particularly flows) we should consider looking for schema compatibility as well.

Acceptance Criteria

  • 1 When running an evaluation, only relevant/compatible datasets are available

Designs

Inlined screenshots

Notes

After choosing a flow, model datasets are still available in the drop down:

Image

Running the evaluation shows success, but it is really a failure:

Image Image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

No branches or pull requests

1 participant