New series of ao/co - related cherry-picks #897
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
leborchuk
approved these changes
Jan 28, 2025
x4m
approved these changes
Jan 29, 2025
|
uhhm, ic-isolation2 is flapping? or is it real fail? |
never see those failures . |
|
that a real issue with new test I fought I fixed it |
|
happy new year by the way. |
yjhjstz
reviewed
Feb 11, 2025
yjhjstz
reviewed
Feb 11, 2025
reshke
force-pushed
the
judewjidew
branch
2 times, most recently
from
1b031dd to
1859ea1
Compare
|
cherry-picked greenplum-db/gpdb-archive@1248809 here, should fix CI stale |
reshke
force-pushed
the
judewjidew
branch
2 times, most recently
from
3bd0498 to
55967ca
Compare
Though no special code was written for unique indexes in the context of repeatable read transaction isolation, add a smoke test: * For added coverage * To illustrate how unique index checks cross transaction isolation boundaries.
We don't have to do anything special for partial indexes. Keys not satisfying the partial index predicate are never inserted into the index, and hence there are no uniqueness checks triggered (see ExecInsertIndexTuples()). Also during partial unique index builds, keys that don't satisfy the partial index predicate are never inserted into the index (see *_index_build_range_scan()). Add smoke tests all the same: * For added coverage * For illustrative purposes
Now that unique indexes are supported on AO tables, allow ALTER TABLE SET ACCESS METHOD TO ao_row|ao_column, on heap tables bearing unique indexes.
This enables reporting for the table scan phase of index builds on AO tables. This directly affects: (1) pg_stat_progress_create_index.blocks_total (2) pg_stat_progress_create_index.blocks_done Sample: Connect in utility mode to a segment, while CREATE INDEX is running on an AO table: postgres=# select pid, command, phase, blocks_total, blocks_done from pg_stat_progress_create_index; pid | command | phase | blocks_total | blocks_done --------+--------------+--------------------------------+--------------+------------- 633034 | CREATE INDEX | building index: scanning table | 54993 | 4412 (1 row) This commit introduces: (1) Accounting in AppendOnlyScanDescData to track the total number of bytes read across all segment files. It is updated whenever a new block is read. If the block is compressed, its compressed length is added (uncompressed length otherwise). (2) Reporting code in appendonly_index_build_range_scan() to report in increments of BLCKSZ, the number of blocks scanned so far. The total number of blocks to be scanned is also published up front and is determined by summing the compressed eof values of all segfiles. (3) Marks RelationGuessNumberOfBlocksFromSize() as inline, since it is now being called in a much hotter code path. Note: We currently only support reporting for non-concurrent index builds and serial builds.
This enables reporting for the table scan phase of index builds on AOCO tables. This directly affects: (1) pg_stat_progress_create_index.blocks_total (2) pg_stat_progress_create_index.blocks_done Sample: Connect in utility mode to a segment, while CREATE INDEX is running on an AOCO table: postgres=# select pid, command, phase, blocks_total, blocks_done from pg_stat_progress_create_index; pid | command | phase | blocks_total | blocks_done --------+--------------+--------------------------------+--------------+------------- 644771 | CREATE INDEX | building index: scanning table | 9779 | 6832 This commit introduces: (1) Accounting in AOCSScanDescData to track the total number of bytes read across all segment files. It is updated whenever a new block is read. If the block is compressed, its compressed length is added (uncompressed length otherwise). (2) open_all_datumstreamread_segfiles() is touched since it also reads blocks. It's interface is made simpler by simply passing the scan descriptor, which is also used to get at the total bytes counter. (3) Reporting code in aoco_index_build_range_scan() to report in increments of BLCKSZ, the number of blocks scanned so far. The total number of blocks to be scanned is also published up front and is determined by summing the compressed eof values of all segfiles. The total number of blocks depends on whether a block directory is being built as part of the scan. If it is being built, then the whole table needs to be scanned, whereas only index columns need to be scanned otherwise. A new function has been introduced to summarize the filesegtotals which accepts column projection info: GetAOCSSSegFilesTotalsWithProj(). Note: We currently only support reporting for non-concurrent and serial builds.
|
@yjhjstz Hi! How do you like the latest version? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions