Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Evaluation used for V.1.1.1 #6

Open
jordiclive opened this issue Jul 24, 2021 · 2 comments
Open

Update Evaluation used for V.1.1.1 #6

jordiclive opened this issue Jul 24, 2021 · 2 comments

Comments

@jordiclive
Copy link

jordiclive commented Jul 24, 2021

The references in /evaluation/dart_reference are not for the current version. Can you replace with the new references and share the tokenization script that is done to the predictions.

I am getting very different BLEU scores depending on tokenization, and how many references I use.
As there are up to ~30 for a few examples.

I would like to directly compare against the README leaderboard.

@lxuechen
Copy link

lxuechen commented Aug 1, 2021

Upvoting this, since having the same issue here.

@LemonQC
Copy link

LemonQC commented Aug 18, 2021

How to run BART in the model. Could you provide more details about running environment and python script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants