Create infer request per inference to enable concurrency#494

Draft

mzegla wants to merge 3 commits intohuggingface:mainfrom mzegla:infer_request_pool

+135-66

Commits on Nov 22, 2023

Commits on Nov 23, 2023

shallow output copy for diffusion
mzegla
committed