Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve LDAP performances when updating records, especially when many user entries exists #1975

Open
wants to merge 19 commits into
base: dev
Choose a base branch
from

Conversation

zamentur
Copy link
Member

@zamentur zamentur commented Oct 21, 2024

The problem

With 150 users and 12 apps, a yunohost user create take 1min
With 350 users and ~25 apps, a yunohost user create take hours maybe days.

Solution

Replace ldap.modlist.modifyModlist by an optimize code that work for yunohost.

Test in a dev platforme (with ssd 600Mbps) : the 169th user creation is made in ~3s.

PR Status

Need more test to be sure it doesn't break something...

How to test

for i in {1..150}; do echo $i; yunohost user test$i -d ynh.local -F "test test" -p "Thisisadummypa:ssw0rd"; done

@alexAubin alexAubin changed the title Enh perf ldap update Improve LDAP performances when updating records, especially when many user entries exists Oct 21, 2024
src/utils/ldap.py Show resolved Hide resolved
src/utils/ldap.py Outdated Show resolved Hide resolved
@Salamandar
Copy link
Contributor

Not sure about your last commits on [user] -> user, but apart from that, LGTM.

@zamentur zamentur changed the base branch from dev to main January 20, 2025 14:03
@zamentur zamentur changed the base branch from main to dev January 20, 2025 14:04
conf/slapd/permission.ldif Show resolved Hide resolved
@@ -246,7 +246,7 @@ def _apply(
ldap = _get_ldap_interface()
ldap.update(
"cn=admins,ou=sudo",
{"sudoOption": ["!authenticate"] if passwordless_sudo else []},
{"sudoOption": "!authenticate" if passwordless_sudo else set()},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sooo does this mean now the code magically handle the case where the data is a string (and won't interpret a string as a list of char) ?

Comment on lines 488 to +491
# If the requested mail address is already as main address or as an alias by this user
if mail in user["mail"]:
user["mail"].remove(mail)
if mail != user["mail"][0]:
user["mail"].remove(mail)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure to fully understand that bit but my brain is not super fresh right now

Comment on lines 516 to +517
if mail in user["mail"]:
user["mail"].remove(mail)
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also not sure to understand this bit

Comment on lines -446 to +449
new_attr_dict["givenName"] = [firstname] # TODO: Validate
new_attr_dict["cn"] = new_attr_dict["displayName"] = [
(firstname + " " + user["sn"][0]).strip()
]
new_attr_dict["givenName"] = firstname # TODO: Validate
new_attr_dict["cn"] = new_attr_dict["displayName"] = (
firstname + " " + user["sn"][0]
).strip()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are similar stuff in portal.py's portal_update() that need to be adapted too

]
if new_attr_dict["mail"]:
new_attr_dict["objectClass"] = set(group["objectClass"])
new_attr_dict["objectClass"].update({"mailGroup"})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
new_attr_dict["objectClass"].update({"mailGroup"})
new_attr_dict["objectClass"].add("mailGroup")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants