Skip to content

Commit

Permalink
update docs on empty reference
Browse files Browse the repository at this point in the history
  • Loading branch information
nikvaessen committed Feb 2, 2025
1 parent 7f2d76e commit cc23b12
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 2 deletions.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,12 @@ error = wer(reference, hypothesis)
There is undefined behaviour when you apply an empty reference and hypothesis pair
to the WER formula, as you get a division by zero.

As of version 4.0, `jiwer` defines the behaviour as follows:
As of version 4.0, `jiwer` defines the behaviour as follows, and thereby also
lifts the requirement for reference strings to be non-empty.
This allows for testing whether models hallucinate on silent audio.
Note that usually, there are multiple reference and hypothesis pairs.
It now supported that one or more of these references are empty, but to test well,
most references should still be non-empty.

```python3
import jiwer
Expand Down
7 changes: 6 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,12 @@ $ pip install jiwer
There is undefined behaviour when you apply an empty reference and hypothesis pair
to the WER formula, as you get a division by zero.

As of version 4.0, `jiwer` defines the behaviour as follows:
As of version 4.0, `jiwer` defines the behaviour as follows, and thereby also
lifts the requirement for reference strings to be non-empty.
This allows for testing whether models hallucinate on silent audio.
Note that usually, there are multiple reference and hypothesis pairs.
It now supported that one or more of these references are empty, but to test well,
most references should still be non-empty.

```python3
import jiwer
Expand Down

0 comments on commit cc23b12

Please sign in to comment.