Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DO NOT MERGE] [Only for the long running PW job] Pathways gcsfuse intergation #401

Draft
wants to merge 8 commits into
base: develop
Choose a base branch
from
Draft

[DO NOT MERGE] [Only for the long running PW job] Pathways gcsfuse intergation #401

wants to merge 8 commits into from

Conversation

shauryagup
Copy link
Collaborator

Fixes / Features

  • This enables memory mounted gcsfuse buckets for the large scale cluster
  • Changes the startup policy of the Pathways pods
  • Removes podFailurePolicy from pathways containers

Testing / Documentation

Verified on a large scale run

The startupPolicy and podFailurePolicy related changes can be merged into main

The gcsfuse related changes are not ready to be merged into main and needs further work:

  1. We need to integrate with the xpk storage API to properly plumb gcsfuse related volumes/volume mounts/annotations into PW container definitions.
  2. The changes rely on PV and PVC resources to be already present on the cluster. The storage API integration would also ensure that those resources are created by XPK automatically

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Obliviour Obliviour Awaiting requested review from Obliviour Obliviour will be requested when the pull request is marked ready for review Obliviour is a code owner

@44past4 44past4 Awaiting requested review from 44past4 44past4 will be requested when the pull request is marked ready for review 44past4 is a code owner

@sharabiani sharabiani Awaiting requested review from sharabiani sharabiani will be requested when the pull request is marked ready for review sharabiani is a code owner

@pawloch00 pawloch00 Awaiting requested review from pawloch00 pawloch00 will be requested when the pull request is marked ready for review pawloch00 is a code owner

@BluValor BluValor Awaiting requested review from BluValor BluValor will be requested when the pull request is marked ready for review BluValor is a code owner

@gcie gcie Awaiting requested review from gcie gcie will be requested when the pull request is marked ready for review gcie is a code owner

@RoshaniN RoshaniN Awaiting requested review from RoshaniN RoshaniN will be requested when the pull request is marked ready for review RoshaniN is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@shauryagup @SujeethJinesh