Skip to content
Back to pull request #11729

Ignore invalid UTF-8 input in the BPE tokenizer #21546

Sign in to view logs

Ignore invalid UTF-8 input in the BPE tokenizer

Ignore invalid UTF-8 input in the BPE tokenizer #21546

Triggered via pull request February 7, 2025 10:36
@cfillioncfillion
synchronize #11729
cfillion:bpe-tokenize-mojibake-noexcept
Status Success
Total duration 17s
Artifacts –

editorconfig.yml

on: pull_request
editorconfig
9s
editorconfig
Fit to window
Zoom out
Zoom in