feat(ui): flux redux canvas #7752
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…p a dependency of it
…we cannot find it This follows the same pattern for IP Adapter w/ its CLIP Vision model. The SigLIP model is unlikely to ever change and we don't want to force the user to select it anywhere. Hardcoding it is safe and makes the UX much nicer. The alternative is a model dropdown that will likely only ever have one valid choice in it.
User facing: When a FLUX main model is selected, users may now add Regional Reference Image layers. When switching between FLUX Redux and FLUX IP Adapter, the settings will change to match the model type. (IP Adapter has weight, begin/end step, but Redux does not.) The image will be retained when switching between the two. Otherwise it works the same way as IP Adapter - both in Global and Regional Reference Image layers. --- Internal state handling: Slightly awkward, but it was easiest to make FLUX Redux a second type of IP Adapter in redux state. Global and regional reference images still have a single `ipAdapter` field, but it can have a type of `ip_adapter` or `flux_redux`. Ideally, this field is called `config` or `settings` or something, but we are past that point. We _could_ do a migration to rename it, but I don't think it's worth the effort. --- Other changes: - Updated canvas layer validators to handle FLUX Redux. - Updated model list loading logic to un-set FLUX Redux models in Canvas if they are not in the list (e.g. if the user deletes the model in the main app). - Updated graph builders - new `addFLUXRedux` util & updated `addRegions` util. - Updated the `buildModelsHook` util to return a hook that accepts a filter callback. This handles a discrepancy: FLUX IP Adapter does not support regional guidance, but FLUX Redux does. The Regional Guidance settings provide the filter to filter out FLUX IP Adapter models from the combined list of IP Adapter ahd Redux models.
psychedelicious
requested review from
blessedcoolant,
maryhipp,
hipsterusername,
lstein,
brandonrising,
RyanJDick and
jazzhaiku
as code owners
github-actions
bot
added
python
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add support for FLUX Redux on Canvas. Choose a FLUX Redux model in a Global or Regional Reference Image layer to use it.
I made a breaking change to the
flux_reduxnode, removing the SigLIP model field (node version bumped to2.0.0to account for this).We don't think there will ever be a choice of SigLIP models, so to make the UX smoother, we will hardcode it and handle it like we do for CLIP Vision models:
flux_reduxnode, if the model is not available, attempt to download it on the spot w/ a 10min timeout and warning in the logsRelated Issues / Discussions
n/a
QA Instructions
Note: It looks like Redux almost totally nullifies text conditioning. I'm not sure if this is expected, but I think @hipsterusername noticed the same thing.
Regions seem to be a bit less strict with FLUX Redux than SDXL IP Adapter. With SDXL IP Adapter, you can get starkly contrasting styles right next to each other. For example, a watercolor-like subject on photograph-like background. I'm having trouble achieving this with Redux. Not suggesting this is a problem - it's different tech, after all - just noting it.
Merge Plan
n/a
Checklist
What's Newcopy (if doing a release after this PR)