Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transaction information to summary #71

Merged
merged 18 commits into from
Nov 27, 2024

Conversation

systay
Copy link
Contributor

@systay systay commented Nov 26, 2024

Overview

Enhances the Markdown report generated by vt summarize by incorporating transaction pattern analysis from vt transactions JSON output.

This enhancement analyzes transaction patterns with a focus on identifying meaningful relationships for sharding key decisions. The analysis specifically targets patterns where queries share predicate values, filtering out patterns that don't contribute to sharding strategy insights.

Here follows an example of this section in the report:

Transaction Patterns

Pattern 1 (Observed 46 times)

Tables Involved: craftable_timesheets, pos_dailies, pos_tickets

Query Patterns

  1. DELETE on craftable_timesheets
    Predicates: craftable_timesheets.store_id = 0 AND craftable_timesheets.rpt_date = 1 AND craftable_timesheets.intraday = 2

  2. DELETE on pos_tickets
    Predicates: pos_tickets.store_id = 0 AND pos_tickets.rpt_date = 1 AND pos_tickets.intraday = 2

  3. DELETE on pos_dailies
    Predicates: pos_dailies.store_id = 0 AND pos_dailies.rpt_date = 1 AND pos_dailies.intraday = 2

Shared Predicate Values

  • Value 0 applied to:
    • craftable_timesheets.store_id
    • pos_tickets.store_id
    • pos_dailies.store_id
  • Value 1 applied to:
    • craftable_timesheets.rpt_date
    • pos_tickets.rpt_date
    • pos_dailies.rpt_date
  • Value 2 applied to:
    • craftable_timesheets.intraday
    • pos_tickets.intraday
    • pos_dailies.intraday

@systay systay changed the title Refactor summarize to make it easier to add more json reports Add transaction information to summary Nov 27, 2024
Signed-off-by: Andres Taylor <[email protected]>
@systay systay marked this pull request as ready for review November 27, 2024 12:20
@systay systay enabled auto-merge November 27, 2024 12:21
Signed-off-by: Andres Taylor <[email protected]>
@systay systay merged commit 367ffb5 into vitessio:main Nov 27, 2024
1 check passed
@frouioui frouioui deleted the tx-summarize branch November 27, 2024 16:50
Copy link
Contributor

@rohit-nayak-ps rohit-nayak-ps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactor looks so good!


for i, tx := range transactions {
var tables []string
for _, query := range tx.Queries {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be inefficient for large number of queries. The usual pattern of using a map to check existence might be faster.

"github.com/vitessio/vt/go/markdown"
)

func renderHotQueries(md *markdown.MarkDown, queries []keys.QueryAnalysisResult, metricReader getMetric) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should think about using a template library like text/html to generate the markdown. That approach has worked well for me in the past.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants