Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow max-execution-time parameter for SyncDiffInspector #845

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Allow max-execution-time parameter for SyncDiffInspector #845

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b1cbb1c
Allow max-execution-time parameter for SyncDiffInspector
zph Aug 14, 2024
d4dfa85
Add test and mark max-execution-time as omitempty to avoid backwards …
zph Feb 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion pkg/dbutil/common.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ type DBConfig struct {

Schema string `toml:"schema" json:"schema"`

Snapshot string `toml:"snapshot" json:"snapshot"`
Snapshot string `toml:"snapshot" json:"snapshot"`
MaxExecutionTime int `toml:"max-execution-time" json:"max-execution-time"`
}

// String returns native format of database configuration
Expand Down
29 changes: 18 additions & 11 deletions sync_diff_inspector/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,12 +108,13 @@ type Security struct {

// DataSource represents the Source Config.
type DataSource struct {
Host string `toml:"host" json:"host"`
Port int `toml:"port" json:"port"`
User string `toml:"user" json:"user"`
Password utils.SecretString `toml:"password" json:"password"`
SqlMode string `toml:"sql-mode" json:"sql-mode"`
Snapshot string `toml:"snapshot" json:"snapshot"`
Host string `toml:"host" json:"host"`
Port int `toml:"port" json:"port"`
User string `toml:"user" json:"user"`
Password utils.SecretString `toml:"password" json:"password"`
SqlMode string `toml:"sql-mode" json:"sql-mode"`
Snapshot string `toml:"snapshot" json:"snapshot"`
MaxExecutionTime int `toml:"max-execution-time" json:"max-execution-time,omitempty"`
Copy link
Contributor

@kennytm kennytm Feb 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are planning to make it accept all session variables (similar to target-database.session in DM), so it can support max_execution_time as well as other options e.g. to tune the optimizer etc.

EDIT: #847 was just submitted.


Security *Security `toml:"security" json:"security"`

Expand All @@ -138,11 +139,12 @@ func (d *DataSource) SetSnapshot(newSnapshot string) {

func (d *DataSource) ToDBConfig() *dbutil.DBConfig {
return &dbutil.DBConfig{
Host: d.Host,
Port: d.Port,
User: d.User,
Password: d.Password.Plain(),
Snapshot: d.Snapshot,
Host: d.Host,
Port: d.Port,
User: d.User,
Password: d.Password.Plain(),
Snapshot: d.Snapshot,
MaxExecutionTime: d.MaxExecutionTime,
}
}

Expand Down Expand Up @@ -191,6 +193,11 @@ func (d *DataSource) ToDriverConfig() *mysql.Config {
if d.Security != nil && len(d.Security.TLSName) > 0 {
cfg.TLSConfig = d.Security.TLSName
}
// 0 indicates unset and max_execution_time=0 is unlimited
// in that case we don't set the max_execution_time at session level
if d.MaxExecutionTime > 0 {
cfg.Params["max_execution_time"] = strconv.Itoa(d.MaxExecutionTime)
}

return cfg
}
Expand Down
1 change: 1 addition & 0 deletions sync_diff_inspector/config/config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ check-struct-only = false
port = 4000
user = "root"
password = ""
max-execution-time = 5

# Support tls connection
# security.ca-path = "..."
Expand Down
1 change: 1 addition & 0 deletions sync_diff_inspector/config/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func TestParseConfig(t *testing.T) {
require.Nil(t, cfg.Parse([]string{"--config", "config.toml"}))
require.Nil(t, cfg.Init())
require.Nil(t, cfg.Task.Init(cfg.DataSources, cfg.TableConfigs))
require.Equal(t, cfg.Task.TargetInstance.MaxExecutionTime, 5)

require.Nil(t, cfg.Parse([]string{"--config", "config_sharding.toml"}))
// we change the config from config.toml to config_sharding.toml
Expand Down