[feature][disaggregated] Support configuration of username and password and multi secrets #312

hechao-ustc · 2024-12-16T04:15:00Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Support configuration of username and password for disaggregated
Support multi secrets for disaggregated

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…rd and multi secret

…ig_username_and_password # Conflicts: # pkg/controller/sub_controller/disaggregated_subcontroller.go

…rd and multi secret

catpineapple · 2024-12-20T06:16:11Z

pkg/common/utils/resource/pod.go

+			})
+		}
+	} else {
+		defaultEnvs = append(defaultEnvs, []corev1.EnvVar{{


This behavior will trigger the restart of the DORIS cluster after upgrading the operator. It is dangerous for the online environment.

Done. Currently, the doris cluster will only be restarted when ddc.Spec.AdminUser is configured.

catpineapple · 2024-12-20T06:22:55Z

pkg/common/utils/resource/pod.go

@@ -729,6 +752,46 @@ func getMultiSecretVolumeAndVolumeMount(bSpec *v1.BaseSpec, componentType v1.Com
 	return volumes, volumeMounts
 }

+func GetMultiSecretVolumeAndVolumeMountWithCommonSpec(cSpec *dv1.CommonSpec, componentType dv1.DisaggregatedComponentType) ([]corev1.Volume, []corev1.VolumeMount) {


componentType is not a must parameter

if only log, the componentType can omit int parameters. log in message on the call function.

It's better to make judgment and print log in the GetMultiSecretVolumeAndVolumeMountWithCommonSpec here. Because there are three statefulsets in the disaggregated cluster, and each statefulset has two methods that call the GetMultiSecretVolumeAndVolumeMountWithCommonSpec, which results in numerous places that need to be modified.

It does not need to output a log. common spec only in fe, be, cn, broker spec, so, if the common spec is not empty, the component type exists.

Done, componentType has been removed.

catpineapple · 2024-12-20T06:30:38Z

pkg/controller/sub_controller/disaggregated_subcontroller.go

+	for _, secret := range secrets {
+		var s corev1.Secret
+		if getErr := d.K8sclient.Get(ctx, types.NamespacedName{Namespace: ddc.Namespace, Name: secret.SecretName}, &s); getErr != nil {
+			errMessage = errMessage + fmt.Sprintf("(name: %s, namespace: %s, err: %s), ", secret.SecretName, ddc.Namespace, getErr.Error())


No Import fmt package, code compilation error

catpineapple · 2024-12-20T06:59:38Z

pkg/common/utils/resource/pod.go

+	for _, secret := range cSpec.Secrets {
+		path := secret.MountPath
+		if secret.MountPath == "" {
+			path = defaultMountPath


path must be unique, You need to add the unique verification of the mounting path. If you do not pass, print it in an wrong log. like this config:

feSpec: secrets: - secretName: secret1 - secretName: secret2

In the controller, it will be judged whether the path is unique or not.

catpineapple · 2024-12-20T07:06:39Z

In the Docker entrypoint.sh script, the current method of getting PWD is only obtained in $Auth_path. By placing Name and PWD directly into ENV, it will be covered by resolve_password_from_secret.

DB_ADMIN_USER=${USER:-"root"}

DB_ADMIN_PASSWD=$PASSWD
AUTH_PATH="/etc/basic_auth"

resolve_password_from_secret()
{
    if [[ -f "$AUTH_PATH/password" ]]; then
        DB_ADMIN_PASSWD=`cat $AUTH_PATH/password`
    fi
    if [[ -f "$AUTH_PATH/username" ]]; then
        DB_ADMIN_USER=`cat $AUTH_PATH/username`
    fi
}

…rd and multi secret

…d password and multi secret" This reverts commit 1b0d139.

…rd and multi secret

hechao added 4 commits December 13, 2024 10:35

[feature][disaggregated] Support configuration of username and passwo…

b4b8cb2

…rd and multi secret

[feature][disaggregated] Support configuration of username and passwo…

2d4276f

…rd and multi secret

Merge remote-tracking branch 'refs/remotes/upstream/master' into conf…

54f29c5

…ig_username_and_password # Conflicts: # pkg/controller/sub_controller/disaggregated_subcontroller.go

[feature][disaggregated] Support configuration of username and passwo…

0e7f85b

…rd and multi secret

catpineapple reviewed Dec 20, 2024

View reviewed changes

intelligentfu8 mentioned this pull request Dec 23, 2024

[Bug] 存算分离模式修改root密码后导致计算组服务挂掉 #316

Closed

3 tasks

hechao added 5 commits December 23, 2024 11:28

[feature][disaggregated] Support configuration of username and passwo…

acdbc28

…rd and multi secret

[feature][disaggregated] Support configuration of username and passwo…

1b0d139

…rd and multi secret

Revert "[feature][disaggregated] Support configuration of username an…

0edece1

…d password and multi secret" This reverts commit 1b0d139.

[feature][disaggregated] Support configuration of username and passwo…

f4753f2

…rd and multi secret

[feature][disaggregated] Support configuration of username and passwo…

2749ea6

…rd and multi secret

intelligentfu8 merged commit a13e1ab into apache:master Dec 24, 2024
1 check passed

catpineapple mentioned this pull request Dec 24, 2024

Release Note 24.2.0 #319

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature][disaggregated] Support configuration of username and password and multi secrets #312

[feature][disaggregated] Support configuration of username and password and multi secrets #312

hechao-ustc commented Dec 16, 2024 •

edited

Loading

catpineapple Dec 20, 2024

hechao-ustc Dec 23, 2024

catpineapple Dec 20, 2024

intelligentfu8 Dec 23, 2024

hechao-ustc Dec 23, 2024

intelligentfu8 Dec 23, 2024 •

edited

Loading

hechao-ustc Dec 23, 2024

catpineapple Dec 20, 2024

hechao-ustc Dec 23, 2024

catpineapple Dec 20, 2024

hechao-ustc Dec 23, 2024

catpineapple commented Dec 20, 2024

[feature][disaggregated] Support configuration of username and password and multi secrets #312

[feature][disaggregated] Support configuration of username and password and multi secrets #312

Conversation

hechao-ustc commented Dec 16, 2024 • edited Loading

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

intelligentfu8 Dec 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

catpineapple commented Dec 20, 2024

hechao-ustc commented Dec 16, 2024 •

edited

Loading

intelligentfu8 Dec 23, 2024 •

edited

Loading