Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update setting activation scale for diffusers #1110

Merged
merged 4 commits into from
Jan 16, 2025

Conversation

eaidova
Copy link
Collaborator

@eaidova eaidova commented Jan 15, 2025

What does this PR do?

some diffusion models suffer from execution in fp16 due to activations overflow, previously, to fix this issue we changed inference precision to f32 in runtime which may significantly affect performance.
In 2025.0, openvino will provide a more accurate mechanism to inform plugins about such possible overflows. However, this may still affect model perf if it will be set everywhere where there is no need.

This PR introduced approach for setting activation scale only for specific pipeline components based on models analysis result

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@eaidova eaidova requested a review from AlexKoff88 January 15, 2025 09:49
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@AlexKoff88
Copy link
Collaborator

@IlyasMoutawwakil, @echarlaix, PR is ready for your review. This is a hotfix to solve accuracy issues with FP16.

@e-ddykim
Copy link
Contributor

I checked that SDXL and FLUX.1 were converted as expected. Thank you so much!
But, could you let me know how to convert SD3 by optimum-cli?

@eaidova
Copy link
Collaborator Author

eaidova commented Jan 15, 2025

I checked that SDXL and FLUX.1 were converted as expected. Thank you so much! But, could you let me know how to convert SD3 by optimum-cli?

@e-ddykim export command is the same like for other models:

optimum-cli export openvino -m stabilityai/stable-diffusion-3-medium-diffusers sd3

in case, if you use local directory instead of model id from hub, you need to add --task text-to-image

@IlyasMoutawwakil
Copy link
Member

@eaidova I think you need to use stabilityai/stable-diffusion-3-medium-diffusers the diffusers compatible variant

@e-ddykim
Copy link
Contributor

@eaidova I think you need to use stabilityai/stable-diffusion-3-medium-diffusers the diffusers compatible variant

Yes, it works. Thank you!

@e-ddykim
Copy link
Contributor

I also checked that overflow does not happen on SD3 out-of-box. Thank you!

Copy link
Member

@IlyasMoutawwakil IlyasMoutawwakil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IlyasMoutawwakil IlyasMoutawwakil merged commit 726191f into huggingface:main Jan 16, 2025
22 checks passed
ilya-lavrenov added a commit to openvinotoolkit/openvino.genai that referenced this pull request Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants