Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Change Proposal] Support 'synthetic_source_keep' in field mapping attributes #861

Open
Tracked by #2376
andrewkroh opened this issue Jan 27, 2025 · 3 comments
Open
Tracked by #2376
Labels
discuss Issue needs discussion

Comments

@andrewkroh
Copy link
Member

synthetic_source_keep is a mapping attribute added to Elasticsearch 8.16 that controls the behavior of a field under sythentic_source and LogsDB.

Integrations need to be able to specify this attribute to optimize the behavior of certain array fields in order to minimize storage costs. For example, an integration that stores a list of IP address would set synthetic_source_keep: none when the value is treated as a unsorted set (ordering does not matter and duplicate values have no meaning). Then the storage of the field can be optimized by not storing the original value in the _source.

@jsoriano
Copy link
Member

jsoriano commented Jan 27, 2025

Do you know if this attribute replaces store? We have found that it is not leveraged in 8.18 when included in component templates, see #860.

@andrewkroh
Copy link
Member Author

I don't know for sure. I don't think it replaces store, but perhaps when synthetic_source is enabled only synthetic_source_keep is honored.

@jsoriano
Copy link
Member

Issue with store resolved. It is enabled by default in text fields since 8.18, and default fields are generally not included in mappings. I am adjusting the test in #860.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Issue needs discussion
Projects
None yet
Development

No branches or pull requests

2 participants