Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix zero timestamp #1

Merged
merged 1 commit into from
Apr 15, 2024
Merged

Fix zero timestamp #1

merged 1 commit into from
Apr 15, 2024

Conversation

agonzale34
Copy link
Collaborator

Description

This PR contains bugfix for timestamp zero issue we are facing when migrating tables that get new inserts, with 0 timestamp value, which get a wrong format.

In case this PR introduced Go code changes:

  • contributed code is using same conventions as original code
  • script/cibuild returns with no formatting errors, build errors or unit test errors.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@pbitty
Copy link

pbitty commented Apr 10, 2024

CLA assistant check Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Where does this come from?

@agonzale34
Copy link
Collaborator Author

CLA assistant check Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Where does this come from?

no idea, maybe some of the files in .github folder?

Copy link

@pbitty pbitty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Noting here that we went over this in a video call and confirmed that timestamp 1970-01-01 00:00:00 isn't valid for MySQL. The lowest value you can put seems to be 1970-01-01 00:00:01.

@pbitty
Copy link

pbitty commented Apr 10, 2024

@agonzale34 , good catch.

Regarding contributing this upstream, it appears this problem has been fixed because they updated the version of the go-mysql/replication library they are using in v1.1.6.

It used to be github.com/siddontang/go-mysql/replication and at some point it became github.com/go-mysql-org/go-mysql/replication. In that version they do have a special case for the zero value: https://github.com/github/gh-ost/blob/732a0642317a544aee533f6b2e0d04de08a1d490/vendor/github.com/go-mysql-org/go-mysql/replication/row_event.go#L1048-L1053.

	case MYSQL_TYPE_TIMESTAMP:
		n = 4
		t := binary.LittleEndian.Uint32(data)
		if t == 0 {
			v = formatZeroTime(0, 0)
		} else {
			// ...

@agonzale34
Copy link
Collaborator Author

@agonzale34 , good catch.

Regarding contributing this upstream, it appears this problem has been fixed because they updated the version of the go-mysql/replication library they are using in v1.1.6.

It used to be github.com/siddontang/go-mysql/replication and at some point it became github.com/go-mysql-org/go-mysql/replication. In that version they do have a special case for the zero value: https://github.com/github/gh-ost/blob/732a0642317a544aee533f6b2e0d04de08a1d490/vendor/github.com/go-mysql-org/go-mysql/replication/row_event.go#L1048-L1053.

	case MYSQL_TYPE_TIMESTAMP:
		n = 4
		t := binary.LittleEndian.Uint32(data)
		if t == 0 {
			v = formatZeroTime(0, 0)
		} else {
			// ...

@pbitty Thanks for going above a beyond with that investigation. I really wish we could use this version with our AWS RDS cluster, sadly the tx_isolation error won't let us use the version:
github#1262 (comment)

@pbitty
Copy link

pbitty commented Apr 10, 2024

I realize that. I just mean that I don't think we need to contribute this fix upstream - it looks like it's been fixed.

It was easy for me to find this because I've worked a lot on our outbox-relay tool, which uses the same library and I could see that gh-ost was using an old version. But it turns out they upgraded it on the latest release.

@agonzale34
Copy link
Collaborator Author

I realize that. I just mean that I don't think we need to contribute this fix upstream - it looks like it's been fixed.

It was easy for me to find this because I've worked a lot on our outbox-relay tool, which uses the same library and I could see that gh-ost was using an old version. But it turns out they upgraded it on the latest release.

Perfect, this PR is just internally for us, while they main project update the issues with the rds.

@agonzale34 agonzale34 merged commit 89a429e into release-1.1.5 Apr 15, 2024
@agonzale34 agonzale34 deleted the fix-zero-timestamp branch April 15, 2024 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants