Skip to content

Ignore invalid UTF-8 input in the BPE tokenizer #19221

Ignore invalid UTF-8 input in the BPE tokenizer

Ignore invalid UTF-8 input in the BPE tokenizer #19221

Annotations

1 warning

windows-msys2 (UCRT64, ucrt-x86_64, Release)

succeeded Feb 7, 2025 in 6m 36s