Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PS-9238: Make MySQL 5.7 compatible with CREATE TABLE AS SELECT [...] START TRANSACTION to improve 8.0 -> 5.7 replication reliability #5362

Open
wants to merge 3 commits into
base: 8.0
Choose a base branch
from

Commits on Jul 25, 2024

  1. PS-9238: Make MySQL 5.7 compatible with

    CREATE TABLE AS SELECT [...] START TRANSACTION to improve 8.0 -> 5.7
    replication reliability
    
    https://perconadev.atlassian.net/browse/PS-9238
    
    Problem:
    5.7 does not understand the new syntax and complains about
    CREATE TABLE ... START TRANSACTION received in binlog.
    
    Cause:
    Commit 4fe3f2f (8.0.21) introduced changes that allow the execution
    of CREATE TABLE AS SELECT as a single transaction. Before that change,
    CREATE TABLE was the 1st transaction, then rows were inserted, which was
    unsafe. To achieve this we don't commit the transaction after
    CREATE TABLE. The commit is done after rows are inserted. For async
    replica to understand this protocol, START TRANSACTION clause was added
    to CREATE TABLE in binlog. When the replica sees this, it knows not to
    commit after CREATE TABLE.
    
    Solution:
    Introduced ctas_compatibility_mode global, read-only variable, OFF by
    default. If switched to ON, it enforces the behavior from before the
    above-mentioned commit.
    
    Implementation details:
    The fix contains 2 parts:
    1. Query_result_create::binlog_show_create_table() - When SE supports
    atomic DDL, CREATE TABLE query was binlogged with
    direct=false / is_trans=true flags. This caused the event to be cached
    in trx_cache instead of stmt_cache.
    MYSQL_BIN_LOG::commit() logic skips trx_cache commit, if this is not
    the transaction commit (it does only stmt_cache commit). Then, when rows
    are inserted, it was detected that there is ongoing transaction
    (trx_cache not empty), so no new BEGIN statement was generated
    in binlog.
    Effectively CREATE TABLE and rows insertion were done in one
    transaction.
    
    The fix is to revert to the default behavior, i.e.,
    Query_result_create::binlog_show_create_table() creates the event in
    stmt_cache, which is committed after table creation. When rows are being
    inserted, it is detected that trx_cache is empty, so BEGIN statement is
    added to binlog.
    
    2. store_create_info()—When executing CTAS, the START TRANSACTION clause
    was added to the rewritten query to inform the replica about what was
    happening.
    
    The fix is not to add the START TRANSACTION clause.
    kamil-holubicki committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    31f5131 View commit details
    Browse the repository at this point in the history
  2. PS-9238: Make MySQL 5.7 compatible with

    CREATE TABLE AS SELECT [...] START TRANSACTION to improve 8.0 -> 5.7
    replication reliability
    
    https://perconadev.atlassian.net/browse/PS-9238
    
    Part 2 of the fix.
    In the scenario source -> replica1 -> replica2, if replica1 is
    configured with ctas_compatibility_mode it should behave accordingly and
    produce the binglog compatible with version 8.0.20 (intermediate commit
    after CREATE TABLE and no START TRANSACTION clause)
    
    This part is sensitive to create_info->m_transactional_ddl flag which is
    set during statement parsing. If ctas_compatibility_mode is set, we
    force this flag not to be set.
    
    Additionally, replica side follows the path of regular CREATE TABLE
    which binlogs the query as-is. if ctas_compatibility_mode is set,
    CREATE TABLE is removed from the query.
    kamil-holubicki committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    966f479 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. PS-9238: Make MySQL 5.7 compatible with

    CREATE TABLE AS SELECT [...] START TRANSACTION to improve 8.0 -> 5.7
    replication reliability
    
    https://perconadev.atlassian.net/browse/PS-9238
    
    Part 2.1 of the fix.
    When ctas_compatibility_mode=OFF the binlog sequence is:
    BEGIN
    CREATE TABLE ... START TRANSACTION
    ROW EVENT
    COMMIT
    
    MTS sees the above as one group.
    
    When the source produces binlog with CREATE TABLE ... START TRANSACTION
    but the intermediate replica is configured with
    ctas_compatibility_mode=ON it has to execute and binlog CTAS in legacy
    mode (pre 8.0.20). It means the intermediate commit will be done after
    CREATE TABLE. This commit will call pre_commit hook, which will call
    Slave_worker::commit_positions() where we check the validity of
    ptr_group->checkpoint_seqno. checkpoint_seqno however is set when group
    end is detected (COMMIT event), but it is too late.
    
    We detect CTAS case and set checkpoint_seqno to be prepared for
    intermediate commit.
    kamil-holubicki committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    78b64a4 View commit details
    Browse the repository at this point in the history