-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update setting activation scale for diffusers #1110
update setting activation scale for diffusers #1110
Conversation
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
@IlyasMoutawwakil, @echarlaix, PR is ready for your review. This is a hotfix to solve accuracy issues with FP16. |
I checked that SDXL and FLUX.1 were converted as expected. Thank you so much! |
@e-ddykim export command is the same like for other models:
in case, if you use local directory instead of model id from hub, you need to add --task text-to-image |
@eaidova I think you need to use |
Yes, it works. Thank you! |
I also checked that overflow does not happen on SD3 out-of-box. Thank you! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Waiting for implementation on optimum-intel and GPU sides - huggingface/optimum-intel#1110 - openvinotoolkit/openvino#28449
What does this PR do?
some diffusion models suffer from execution in fp16 due to activations overflow, previously, to fix this issue we changed inference precision to f32 in runtime which may significantly affect performance.
In 2025.0, openvino will provide a more accurate mechanism to inform plugins about such possible overflows. However, this may still affect model perf if it will be set everywhere where there is no need.
This PR introduced approach for setting activation scale only for specific pipeline components based on models analysis result
Before submitting