Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Spike] Implement a solution to clean database from bloat data #2189

Open
3 tasks
pacospace opened this issue Jan 27, 2021 · 19 comments
Open
3 tasks

[Spike] Implement a solution to clean database from bloat data #2189

pacospace opened this issue Jan 27, 2021 · 19 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/devsecops Categorizes an issue or PR as relevant to SIG DevSecOps. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@pacospace
Copy link
Contributor

pacospace commented Jan 27, 2021 •
edited by harshad16
Loading

As Database Performance Engineer,

I want to vacuum the database when bloat data are identified.

Acceptance criteria:

  • Understand when data is considered bloated.
  • Investigate the different methods to clean the bloat data.
  • create another issue with the details on implementation.
@goern
Copy link
Member

goern commented Jan 28, 2021

/sig devops
/kind feature
/priority important-longterm

@sesheta sesheta added sig/devops kind/feature Categorizes issue or PR as related to a new feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. labels Jan 28, 2021
@sesheta
Copy link
Member

sesheta commented Apr 28, 2021

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@sesheta sesheta added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 28, 2021
@pacospace
Copy link
Contributor Author

/remove-lifecycle stale

@sesheta sesheta removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 28, 2021
@goern
Copy link
Member

goern commented Apr 30, 2021

/label triage/needs-information

@sesheta sesheta added the triage/needs-information Indicates an issue needs more information in order to work on it. label Apr 30, 2021
@harshad16 harshad16 added sig/knowledge-graph and removed triage/needs-information Indicates an issue needs more information in order to work on it. labels May 12, 2021
@sesheta
Copy link
Member

sesheta commented Jul 15, 2021

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

@sesheta sesheta added the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jul 15, 2021
@pacospace
Copy link
Contributor Author

/remove-lifecycle rotten

@sesheta
Copy link
Member

sesheta commented Aug 24, 2021

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

@sesheta sesheta closed this as completed Aug 24, 2021
@sesheta
Copy link
Member

sesheta commented Aug 24, 2021

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pacospace pacospace reopened this Aug 24, 2021
@pacospace pacospace added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. labels Aug 24, 2021
@sesheta
Copy link
Member

sesheta commented Sep 23, 2021

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

@sesheta
Copy link
Member

sesheta commented Sep 23, 2021

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

@sesheta sesheta removed the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Sep 23, 2021
@sesheta sesheta closed this as completed Sep 23, 2021
@sesheta sesheta added the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Sep 23, 2021
@sesheta
Copy link
Member

sesheta commented Sep 23, 2021

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pacospace pacospace reopened this Sep 27, 2021
@sesheta
Copy link
Member

sesheta commented Oct 27, 2021

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

@sesheta sesheta closed this as completed Oct 27, 2021
@sesheta
Copy link
Member

sesheta commented Oct 27, 2021

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@KPostOffice
Copy link
Member

/reopen
/lifecycle frozen

@sesheta sesheta reopened this Oct 27, 2021
@sesheta
Copy link
Member

sesheta commented Oct 27, 2021

@KPostOffice: Reopened this issue.

In response to this:

/reopen
/lifecycle frozen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@sesheta sesheta added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. labels Oct 27, 2021
@sesheta sesheta added sig/devsecops Categorizes an issue or PR as relevant to SIG DevSecOps. and removed sig/devops labels Jan 19, 2022
@sesheta sesheta added sig/stack-guidance Categorizes an issue or PR as relevant to SIG Stack Guidance. and removed sig/knowledge-graph labels Mar 1, 2022
@goern goern removed the sig/stack-guidance Categorizes an issue or PR as relevant to SIG Stack Guidance. label Apr 4, 2022
@mayaCostantini mayaCostantini changed the title Implement a solution to clean database from bloat data [Spike] Implement a solution to clean database from bloat data Jun 27, 2022
@codificat
Copy link
Member

/sig stack-guidance
/remove-sig devsecops

Reviewed in sig-sg meeting 20220627 - turning this into a spike to determine what "bloated db" means exactly, and what can be done about it

@sesheta sesheta added sig/stack-guidance Categorizes an issue or PR as relevant to SIG Stack Guidance. and removed sig/devsecops Categorizes an issue or PR as relevant to SIG DevSecOps. labels Jun 27, 2022
@codificat
Copy link
Member

/triage accepted

@sesheta sesheta added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Jun 27, 2022
@VannTen
Copy link
Member

VannTen commented Sep 5, 2022

Could the bloated data refer to this : https://github.com/thoth-station/storages/blob/master/thoth/storages/graph/postgres.py#L6842-L7171 ?

This seems to be about postgres db internal machinery (AFAICT it's only used by metrics-exporter https://github.com/search?q=org%3Athoth-station+bloat_data&type=code and it should not be part of storages, rather in a sql-query metrics exporter or something like that)

@codificat codificat moved this to 🆕 New in Planning Board Sep 26, 2022
@mayaCostantini mayaCostantini mentioned this issue Oct 28, 2022
Open
@mayaCostantini mayaCostantini mentioned this issue Nov 14, 2022
Open
2 tasks
@KPostOffice KPostOffice mentioned this issue Jan 27, 2023
Open
@codificat
Copy link
Member

Talking about this in sig-sg 20230130 we believe this is purely related to the database itself (postgres), so putting this back to devsecops
/sig devsecops
/remove-sig stack-guidance

@sesheta sesheta added sig/devsecops Categorizes an issue or PR as relevant to SIG DevSecOps. and removed sig/stack-guidance Categorizes an issue or PR as relevant to SIG Stack Guidance. labels Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/devsecops Categorizes an issue or PR as relevant to SIG DevSecOps. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Planning Board
Status: 🆕 New
Milestone
No milestone
Development

No branches or pull requests
7 participants
@goern @codificat @VannTen @harshad16 @KPostOffice @pacospace @sesheta