Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sync node execution for map over lp #3151

Merged
merged 17 commits into from
Feb 24, 2025
Merged

Conversation

troychiu
Copy link
Member

@troychiu troychiu commented Feb 21, 2025

Why are the changes needed?

Currently if you use flyteremote to sync node execution which contains an array node mapping over a lp, you will get an error

KeyError: LAUNCH_PLAN:troy:development:test_map_over_lp_wf_lp:C1GSRMqmqY7BTENEicqP0A

What changes were proposed in this pull request?

Add case for map over lp in sync node execution.

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Summary by Bito

This PR addresses a bug in FlyteRemote's node execution synchronization by implementing proper handling of launch plan references within array nodes. The changes include improved type hints and interface-based execution synchronization, ensuring correct processing of array nodes containing workflow nodes with launch plan references.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 2

Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
@flyte-bot
Copy link
Contributor

Code Review Agent Run Status

  • Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at [email protected].

Signed-off-by: Troy Chiu <[email protected]>
Signed-off-by: Troy Chiu <[email protected]>
@flyte-bot
Copy link
Contributor

Code Review Agent Run Status

  • Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at [email protected].

@troychiu troychiu changed the title Fix sync for map over lp Fix sync node execution for map over lp Feb 24, 2025
@troychiu troychiu marked this pull request as ready for review February 24, 2025 18:55
Copy link

codecov bot commented Feb 24, 2025

Codecov Report

Attention: Patch coverage is 13.33333% with 13 lines in your changes missing coverage. Please review.

Project coverage is 76.59%. Comparing base (93c87c3) to head (de622c7).
Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
flytekit/remote/remote.py 0.00% 12 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3151      +/-   ##
==========================================
- Coverage   78.18%   76.59%   -1.60%     
==========================================
  Files         211      211              
  Lines       21994    22008      +14     
  Branches     2862     2865       +3     
==========================================
- Hits        17195    16856     -339     
- Misses       3960     4361     +401     
+ Partials      839      791      -48     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flyte-bot
Copy link
Contributor

flyte-bot commented Feb 24, 2025

Code Review Agent Run #f8d228

Actionable Suggestions - 4
  • flytekit/models/core/workflow.py - 1
    • Consider adding property decorator to getter · Line 748-748
  • flytekit/remote/remote.py - 3
Review Details
  • Files reviewed - 3 · Commit Range: 8ba253c..de622c7
    • flytekit/models/core/workflow.py
    • flytekit/remote/entities.py
    • flytekit/remote/remote.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@flyte-bot
Copy link
Contributor

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Bug Fix - Fix Array Node Execution Sync for Launch Plans

workflow.py - Added type hints for launchplan_ref method

entities.py - Added return type hint for FlyteArrayNode

remote.py - Implemented support for syncing array nodes with launch plan references

Copy link
Collaborator

@eapolinario eapolinario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left a comment about a breaking change, but no need to be more accommodating other than just list it in the changelog.

Comment on lines 2680 to +2681
def sync_task_execution(
self, execution: FlyteTaskExecution, entity_definition: typing.Optional[FlyteTask] = None
self, execution: FlyteTaskExecution, entity_interface: typing.Optional[TypedInterface] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a breaking change, right? We should call it out in the changelog.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Thank you for catching this.

@eapolinario eapolinario merged commit 5a17d74 into master Feb 24, 2025
112 of 114 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants