Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for combined iterator lineage #2949

Merged
merged 24 commits into from
Jun 16, 2024
Merged

Conversation

joeyballentine
Copy link
Member

@joeyballentine joeyballentine commented Jun 11, 2024

This PR allows iterators (renamed to generators in this PR to match Python's naming scheme) across different lineages to validly connect with each other. As there is no Navi types for sequence size, there is instead a runtime requirement that enforces that iterators must be of the same expected length in order to work properly together. I expect that in the future the UX for this will be better as we add type information to sequences.

The implementation of this was pretty simple, all things considered. I made it so it first splits up the chain into groups, so any connected generators, collectors, and nodes in between in the same chain of nodes get put into their own group. These groups then each get processed one by one. This was necessary to prevent all generators from being run at the same time, even if they had a completely separate lineage. Then, it just collects each of the generators in a group and manually calls next() on them, and caches their outputs the same way that is done now. It also takes all the connected collectors and output nodes from the group and merges them together into a single array for each, which is then run like a single chain. It is then set up to stop iterating once each iterator has reached its StopIteration exception.

From my testing, this all works pretty well, and you can even do things like connect a Load Images node's directory output to another Load Images node, and it just works because it treats them as separate groups.

I'm sure there's edge cases with this that I haven't thought of, but I think it is at least better than our current restricted system. Unfortunately this iteration system still doesn't allow nodes that take in a sequence and return a differently sized sequence, which is what I initially had planned to try to implement in this. It was only at a certain point that I realized I could pretty easily just do what I did here to get this result, which is still a win.

image

Total things done in this PR:

  • Rename the "newIterator" / Iterator node kind to "generator" / Generator, respectively. This is to have a convention more similar to Python's, and will make more sense once we eventually have actual "Iterator" nodes that just take in a sequence and return a sequence.
  • Rename "iterator inputs" and "iterator outputs" to "iterable inputs" and "iterable outputs" to be more accurate
  • Allow combined generators in lineage
  • Remove "Load Image Pairs" node and add migration to just make it two "Load Images" nodes.

Also, apologies for doing a bit too much at once with this PR, renaming to generator probably should have been its own PR first, but I had already gone too deep into this by the time I thought about that.

Copy link
Member

@RunDevelopment RunDevelopment left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. I like that you basically treat all iterator source nodes as a single iterator source node.

A few questions in that regard:

  1. Since there's only one iteration loop, which node's "Stop at first error" option does it use?
  2. How do you group iterators by length? We explicitly don't know the length of some iterables for certain (e.g. Load Video), so what do you group by?
  3. As I see it, you completely disabled the lineage system. And I mean: completely. Yes, the code is still there, but it doesn't do anything anymore AFAICT. Does it still have any effect right now?

iterators (renamed to generators in this PR to match Python's naming scheme)

Iterators and generator are not synonymous. Why did you rename everything to generator?

backend/src/api/api.py Outdated Show resolved Hide resolved
backend/src/util.py Outdated Show resolved Hide resolved
backend/src/api/api.py Outdated Show resolved Hide resolved
@joeyballentine
Copy link
Member Author

joeyballentine commented Jun 11, 2024

Since there's only one iteration loop, which node's "Stop at first error" option does it use?

Whichever is enabled and gets triggered first. Or at least, that's how it should work. That might not be what this code is set up to do. I will double check later when I can.

How do you group iterators by length?

I don't. They're grouped by connectedness and then checked for validity by expected length afterwards. If the length is unknown, then it will be assumed to be the same. Perhaps I should just make the loop stop as soon as we get a stopiteration exception, and just use the shortest connected length in the case where it can't be determined beforehand?

For most videos though, they do have an expected length (which is what gets checked against)

As I see it, you completely disabled the lineage system. And I mean: completely. Yes, the code is still there, but it doesn't do anything anymore AFAICT. Does it still have any effect right now?

There are still some valid checks that take place in that function, but yes the existing iterator lineage checks have been removed as they are now unnecessary. However, I expect that lineage will make a return once we have types for sequence length.

Iterators and generator are not synonymous. Why did you rename everything to generator?

Because they generate iterable values from non-iterable values.

What would you expect the naming system to look like when we have these four cases:

  1. Normal nodes - no iterable inputs or outputs
  2. Collectors - iterable inputs and no iterable outputs
  3. ????? - no iterable inputs and iterable outputs
  4. ????? - iterable inputs and iterable outputs

My proposal is for 3 to be generators and 4 to be iterators, as that makes sense to me.

@joeyballentine
Copy link
Member Author

Since there's only one iteration loop, which node's "Stop at first error" option does it use?

Just double checked. I did account for this. It saves the last generator_output, so when it fails, it checks if the generator that failed has fail fast enabled. If it does, then it errors immediately, otherwise it proceeds.

@joeyballentine
Copy link
Member Author

Also, this is why it had to be a typing.Generator type for the supplier
image

Iterable does not guarantee that it supports next(), but Generator does.

@joeyballentine
Copy link
Member Author

typing.Iterator also guarantees next (which makes sense given next is an iterator thing), so i used that instead.

@joeyballentine
Copy link
Member Author

Added some unit tests to confirm that the set combining function works properly under a few different test cases.

@joeyballentine
Copy link
Member Author

With the added unit tests, I'm pretty confident in that function. It actually does do almost the same thing that I was trying to do, just more efficiently (and fully correct).

@joeyballentine
Copy link
Member Author

I'm good with this the way it is, personally. If you have anything that you want changed, feel free to submit follow up PRs.

@joeyballentine joeyballentine merged commit 410e586 into main Jun 16, 2024
17 checks passed
@joeyballentine joeyballentine deleted the iterators-take-3 branch June 16, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants