Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to release 8.1(002ebde1e) #386

Merged
merged 212 commits into from
Sep 4, 2024
Merged

Update to release 8.1(002ebde1e) #386

merged 212 commits into from
Sep 4, 2024

Conversation

CalvinNeo
Copy link
Member

What is changed and how it works?

Issue Number: Close #xxx

What's Changed:

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Release note

overvenus and others added 30 commits December 21, 2023 14:51
@overvenus @ti-chi-bot
tikv_util: add handled task counter for LazyWorker (tikv#16196)
3d8f625 
close tikv#16195

Add a metric to inspect the total number of tasks handled by a LazyWorker.

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@gengliqi @ti-chi-bot
expression: cast_duration_as_time should not consider time zone (ti…
669dc79 
…kv#16212)

close tikv#16211

`cast_duration_as_time` should not consider time zone

Signed-off-by: gengliqi <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@overvenus @breezewish
tidb_query_datatype: fix timezone conversion by upgrading chrono-tz (t…
0ec4d33 
…ikv#16221)

ref tikv#16220, close pingcap/tidb#49586

Brazil no longer observes DST since 2020[1]. Updating chrono-tz from
0.5.1 to 0.5.2 bumps the timezone database from 2018i to 2020a, which
includes this change, thus fixes the issue.

[1]: https://en.wikipedia.org/wiki/Daylight_saving_time_in_Brazil

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: Wenxuan <[email protected]>
@crazycs520
*: fix issue of modify resolved-ts.advance-ts-interval from 5s to 2s …
5e0d7b1 
…is not work (tikv#15836)

close tikv#15835

Signed-off-by: crazycs520 <[email protected]>
@SpadeA-Tang
In-memory Engine: implement read flow -- support sequence number (tik…
fc04a50 
…v#16193)

ref tikv#16141

implement read flow -- support sequence number

Signed-off-by: SpadeA-Tang <[email protected]>
@SpadeA-Tang
In-memory Engine: implement read flow -- prefix read (tikv#16237)
9313afa 
ref tikv#16141

implement read flow -- prefix read

Signed-off-by: SpadeA-Tang <[email protected]>
@nolouch
readpool: add priority label for running tasks metrics (tikv#16249)
6a08b98 
close tikv#16248

readpool: add priority label for running tasks

Signed-off-by: nolouch <[email protected]>
@LykxSassinator
metrics: polish the metrics on Raft IO and Raft Propose sections. (ti…
6510959 
…kv#16250)

close tikv#16251

This pr add missing legend on filtering statistics by TiKV instance in Raft IO and Raft Propose sections.

Signed-off-by: lucasliang <[email protected]>
@YuJuncen
added async-backtrace for log backup (tikv#15727)
4702b9b 
ref tikv#15759

added a new http endpoint for querying the asynchronous tasks at `/async_tasks`. 
Used `tracing-active-tree`: `root!` and `frame!` for tracing asynchronous tasks.
see more details in the repo `tikv/tracing-active-tree`.

Signed-off-by: hillium <[email protected]>
Signed-off-by: 山岚 <[email protected]>
@AndreMouche
raftstore/*: Print region information when overlap is detected. (tikv…
1a73761 
…#16241)

close tikv#16240

Signed-off-by: Shirly <[email protected]>
@pingyu @ti-chi-bot
server: fix panic of gRPC threads due to thread group properties not …
369f7be 
…set (tikv#16258)

close tikv#16236

Set thread group properties for gRPC threads to avoid panic when checking "is_shutdown".

Signed-off-by: Ping Yu <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@solotzg @gengliqi
executor: omit truncating error when handling decimal multiplication …
8ba0dac 
…in arithmetic operators (tikv#16187)

close tikv#16268, close pingcap/tidb#48332

omit truncating error when handling decimal multiplication in arithmetic operators

Signed-off-by: TONG, Zhigao <[email protected]>

Co-authored-by: Liqi Geng <[email protected]>
@nolouch @ti-chi-bot
raftstore: fix load base split cannot works in pure follower/stale re…
defc933 
…ad scenario (tikv#16261)

close tikv#15539

raftstore: fix load base split cannot works in pure follower/stale read scenario
- allow split command proposal on non leader peer

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@LykxSassinator
metrics: fix typo errors in the Raft IO panel. (tikv#16266)
0378b24 
ref tikv#16265

Fix typo errors in the Raft IO panel.

Signed-off-by: lucasliang <[email protected]>
@hicqu
cdc: add more metrics about output events queue time (tikv#16281)
7901e0c 
close tikv#16282

Signed-off-by: qupeng <[email protected]>
@Connor1996
config: Enable titan for newly created instance (tikv#16285)
3988614 
ref tikv#16245

Enable titan for newly created instance

Signed-off-by: Connor1996 <[email protected]>
@overvenus @ti-chi-bot
*: Upgrade rust toolchain to nightly-2023-12-10 (tikv#16213)
2605436 
close tikv#15581

Upgrade rust toolchain to nightly-2023-12-10

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@v01dstar
Titan: refine Titan dir empty check (tikv#16296)
3194c4a 
close tikv#16295

Refine Titan dir empty check

Signed-off-by: Yang Zhang <[email protected]>
@YuJuncen @iosmanthus
log-backup: fix a typo (tikv#16257)
df108bf 
ref tikv#15990

Renamed Router::udpate_config to Router::update_config.

Signed-off-by: hillium <[email protected]>

Co-authored-by: iosmanthus <[email protected]>
@cfzjywxk @ti-chi-bot
txn: introduce pessimistic rollback read phase (tikv#16185)
2472fd4 
close tikv#16158

Reduce the cost of expired lock cleaning by introducing pessimsitic rollback read phase, the expired 
pessimistic locks belonging to the same transcation could be cleaned up within one RPC request.

Signed-off-by: cfzjywxk <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@afeinberg
raftstore: fix Display for ReadableOffsetTime (tikv#16300)
e0d7072 
close tikv#16299

Use "%H:%M" to display time for `ReadableOffsetTime`

Signed-off-by: Alex Feinberg <[email protected]>
@overvenus @glorv
engine_rocks: log SST corruption reason (tikv#16253)
7b1e009 
ref tikv#16308

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: glorv <[email protected]>
@3pointer
raftstore: real batch ingest sst files in different region (tikv#16298)
110d9f9 
ref tikv#16267

raftstore: real batch ingest sst files in different region

Signed-off-by: 3pointer <[email protected]>
@LykxSassinator
raft_log_engine: update to latest version. (tikv#16294)
5f5431a 
close tikv#16324

Update `raft_log_engine` lib to the latest version, to fix some issues, including:
- rewrite: optimize the interval of sync when rewriting memtables pingcap#347.
- Return error instead of panicking if rewriting fails pingcap#343.
@hicqu
engine: calculate table properties correctly for Titan (tikv#16320)
65308d6 
close tikv#16319

Signed-off-by: qupeng <[email protected]>
@Connor1996
titan: Fix small blob file can't be gced in fallback mode (tikv#16338)
4a77d18 
close tikv#16336

Fix titan small blbo file can't be gced in fallback mode

Signed-off-by: Connor1996 <[email protected]>
@frew @tonyxuqqi
tikv-ctl: Don't send compact commands to TiFlash stores (tikv#16190)
c041ee3 
close tikv#16189

tikv-ctl compact-cluster now works with clusters that have TiFlash nodes

Signed-off-by: Fred Wulff <[email protected]>

Co-authored-by: tonyxuqqi <[email protected]>
@cfzjywxk @ti-chi-bot
txn: try to fix the possible deadlock caused by scan lock (tikv#16342)
d447120 
ref tikv#16340

Try to fix the possible deadlock caused by scan lock.

Signed-off-by: cfzjywxk <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@overvenus @ti-chi-bot
*: ban unwanted crypto crates (tikv#16352)
7f708a1 
ref tikv#16328

To comply with FIPS 140-2, we should not use crypto algorithms implemented
in rust, instead we should delegate these operation to OpenSSL, including
symmetric encryption, asymmetric encryption, hash functions,
key establishment, digital signatures and random number generators.

Signed-off-by: Neil Shen <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@YuJuncen @ti-chi-bot
snapshot_backup: enhanced prepare stage (tikv#15946)
956c9f3 
close tikv#15739

It contains:
- A coprocessor that can fully reject all admin and ingest commands.
- A new region-leveled wait apply implementation, which allow us to wait all pending commands to be applied.

Signed-off-by: Yu Juncen <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@ti-chi-bot ti-chi-bot bot added the size/XXL label Aug 5, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 5, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@CalvinNeo
Copy link
Member Author

/pull-unit-test

@purelind
Copy link

purelind commented Aug 6, 2024

/test ?

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 6, 2024

@purelind: No presubmit jobs available for pingcap/[email protected]

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@purelind
Copy link

purelind commented Aug 6, 2024

/test ?

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 6, 2024

@purelind: The following commands are available to trigger required jobs:

  • /test pull-unit-test

Use /test all to run all jobs.

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@purelind
Copy link

purelind commented Aug 6, 2024

/test pull-unit-test

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo CalvinNeo force-pushed the update-release-8.1 branch 4 times, most recently from 9c3a9c3 to 46e4b1d Compare September 3, 2024 07:17
@CalvinNeo CalvinNeo changed the title Update to release 8.1 Update to release 8.1(002ebde1e) Sep 3, 2024
@CalvinNeo
Copy link
Member Author

/retest

5 similar comments
@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo CalvinNeo force-pushed the update-release-8.1 branch 3 times, most recently from 0686a8f to d43646a Compare September 4, 2024 03:40
@CalvinNeo
fix clippy
cf047d5 
Signed-off-by: Calvin Neo <[email protected]>
@CalvinNeo CalvinNeo merged commit 5614474 into pingcap:raftstore-proxy-8.1 Sep 4, 2024
3 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
size/XXL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.