Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Register task-queue workers through worker deployment #7380

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Shivs11
Copy link
Member

@Shivs11 Shivs11 commented Feb 24, 2025

What changed?

  • RegisterWorker will now happen from within WorkerDeployment workflow.

Why?

  • Without this change, a successful DescribeVersion call would not mean that the version was registered in the deployment workflow. Users, and current tests in the SDK, have to further do a DescribeWorkerDeployment and verify if the version was present in the deployment before doing operations like SetCurrent/SetRamping. This was not desirable and this PR fixes that

How did you test it?

  • Passing existing suite of tests work. Tested SDK tests locally to verify too.

Potential risks

Documentation

Is hotfix candidate?

@@ -580,7 +581,7 @@ func (c *physicalTaskQueueManagerImpl) ensureRegisteredInDeploymentVersion(
ctx, namespaceEntry, workerDeployment.SeriesName, workerDeployment.BuildId, c.queue.TaskQueueFamily().Name(), c.queue.TaskType(),
"matching service", uuid.New())
if err != nil {
var errTooMany deployment.ErrMaxTaskQueuesInDeployment
var errTooMany workerdeployment.ErrMaxTaskQueuesInDeployment
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to be an error and is something I directly worked on :(
Sorry! Caught it now and verified things are working by running the test TestDeploymentVersionLimits and having a peek at it's output.

TaskQueueType: taskQueue.Type, // since request doesn't pass through frontend, this field is not automatically populated
// Since request doesn't pass through frontend, this field is not automatically populated.
// Moreover, DescribeTaskQueueEnhanced requires this field to be set to WORKFLOW type.
TaskQueueType: enumspb.TASK_QUEUE_TYPE_WORKFLOW,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be an error on my end - if for example, the missing task-queue was an activity type, sending an activity type in this field will result in the Describe call returning an error : Describe cannot be called on an activity type - in this example, to fetch activity task-queue's stats, you pass the type in the types field (which I am doing

@@ -407,6 +490,27 @@ func (d *WorkflowRunner) validateDeleteVersion(args *deploymentspb.DeleteVersion
return d.validateStateBeforeAcceptingDeleteVersion(args)
}

func (d *WorkflowRunner) deleteVersion(ctx workflow.Context, args *deploymentspb.DeleteVersionArgs) error {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consolidated the main logic of deleting a version into a helper because when an individual delete version request were to come in, we need to acquire the lock vs tryDelete requests spurring up when registering a worker cannot acquire a lock since the workflow, at that point in time, is already holding on to it

PS - Since consolidating into a helper won't necessarily change the execution path of old currently-running worker-deployment workflows, this change won't require patching.

@Shivs11 Shivs11 marked this pull request as ready for review February 24, 2025 22:53
@Shivs11 Shivs11 requested a review from a team as a code owner February 24, 2025 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant