From cc23b12594ccf4fd1d7662bc222808624a7d3177 Mon Sep 17 00:00:00 2001 From: Nik Vaessen Date: Sun, 2 Feb 2025 11:13:55 -0500 Subject: [PATCH] update docs on empty reference --- README.md | 7 ++++++- docs/index.md | 7 ++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index be94f9f..db3d337 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,12 @@ error = wer(reference, hypothesis) There is undefined behaviour when you apply an empty reference and hypothesis pair to the WER formula, as you get a division by zero. -As of version 4.0, `jiwer` defines the behaviour as follows: +As of version 4.0, `jiwer` defines the behaviour as follows, and thereby also +lifts the requirement for reference strings to be non-empty. +This allows for testing whether models hallucinate on silent audio. +Note that usually, there are multiple reference and hypothesis pairs. +It now supported that one or more of these references are empty, but to test well, +most references should still be non-empty. ```python3 import jiwer diff --git a/docs/index.md b/docs/index.md index 16203cb..415feb4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -31,7 +31,12 @@ $ pip install jiwer There is undefined behaviour when you apply an empty reference and hypothesis pair to the WER formula, as you get a division by zero. -As of version 4.0, `jiwer` defines the behaviour as follows: +As of version 4.0, `jiwer` defines the behaviour as follows, and thereby also +lifts the requirement for reference strings to be non-empty. +This allows for testing whether models hallucinate on silent audio. +Note that usually, there are multiple reference and hypothesis pairs. +It now supported that one or more of these references are empty, but to test well, +most references should still be non-empty. ```python3 import jiwer