Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Operator Kind #288

Open
Max-Meldrum opened this issue Nov 8, 2021 · 1 comment
Open

Operator Kind #288

Max-Meldrum opened this issue Nov 8, 2021 · 1 comment
Labels
Discussion proposal Design proposal for Arcon
Milestone

Comments

@Max-Meldrum
Copy link
Member

The runtime will need some way to distinguish Operators. An example is knowing that a sequence of Operators is of type Non-Blocking and thus can be fused/chained. A straightforward approach would be something like the following:

pub enum OperatorKind {
    /// Map, Filter, Flatmap, ...
    NonBlocking,
    /// Rolling aggregations (sum, max, min)
    NonBlockingPartialAggregator,
    /// Streaming Windows
    Blocking,
}

pub trait Operator: Send + Sized {
    const KIND: OperatorKind;
}
@Max-Meldrum Max-Meldrum added Discussion proposal Design proposal for Arcon labels Nov 8, 2021
@segeljakt
Copy link
Member

segeljakt commented Nov 8, 2021

I think this is a good argument for having an explicit key_by operator #284. If it's implicit then we can't know if an operator changes the key of incoming events before executing the pipeline, so we can't know if it's safe to fuse it.

@Max-Meldrum Max-Meldrum added this to the 0.3.0 milestone Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion proposal Design proposal for Arcon
Projects
None yet
Development

No branches or pull requests

2 participants