Fix lock order when dropping index #7600

mkindahl · 2025-01-17T10:02:42Z

If an index is dropped, it is necessary to lock the heap table (of
the index) before the index since all normal operations do it in this
order. When dropping an index, we did not take all the necessary locks
in the right order before calling performMultipleDeletions, which can
cause deadlocks when dropping an index on a hypertable at the same time
as running a utility statement that takes heavy locks, e.g., VACUUM or
ANALYZE.

Adding a isolation test as well that will generate a deadlock if the
index and table locks are not taken in the correct order.

codecov · 2025-01-17T10:17:13Z

Codecov Report

Attention: Patch coverage is 50.00000% with 9 lines in your changes missing coverage. Please review.

Project coverage is 81.32%. Comparing base (59f50f2) to head (e584632).
Report is 734 commits behind head on main.

Files with missing lines	Patch %	Lines
src/chunk_index.c	50.00%	3 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7600      +/-   ##
==========================================
+ Coverage   80.06%   81.32%   +1.25%     
==========================================
  Files         190      240      +50     
  Lines       37181    44708    +7527     
  Branches     9450    11164    +1714     
==========================================
+ Hits        29770    36359    +6589     
- Misses       2997     3960     +963     
+ Partials     4414     4389      -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/chunk_index.c

erimatnor

Just nits

tsl/test/isolation/specs/deadlock_drop_index_vacuum.spec

If an index is dropped, it is necessary to lock the heap table (of the index) before the index since all normal operations do it in this order. When dropping an index, we did not take all the necessary locks in the right order before calling `performMultipleDeletions`, which can cause deadlocks when dropping an index on a hypertable at the same time as running a utility statement that takes heavy locks, e.g., VACUUM or ANALYZE. Adding a isolation test as well that will generate a deadlock if the index and table locks are not taken in the correct order.

antekresic

lgtm

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#6884 Match the Postgres NaN comparison behavior in WHERE clause over compressed tables. * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables. * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#6884 Match the Postgres NaN comparison behavior in WHERE clause over compressed tables. * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables. * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * #7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * #7600 Fix lock order when dropping index * #7637 Allow EXPLAIN in read-only mode * #7645 Fix DELETE on compressed chunk with non-btree operators * #7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * #7653 Push down orderby scankeys to Hypercore TAM * #7665 Block merging of frozen chunks * #7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

@bjornuppeke

This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [#7656](#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [#7600](#7600) Fix lock order when dropping index * [#7637](#7637) Allow EXPLAIN in read-only mode * [#7645](#7645) Fix DELETE on compressed chunk with non-btree operators * [#7649](#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [#7653](#7653) Push down orderby scankeys to Hypercore TAM * [#7665](#7665) Block merging of frozen chunks * [#7673](#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [#7653](#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.

mkindahl added the hypertable label Jan 17, 2025

github-actions bot assigned mkindahl Jan 17, 2025

mkindahl added the locking label Jan 17, 2025

mkindahl force-pushed the fix-index-deadlock branch 2 times, most recently from 1f7a183 to 951b92a Compare January 17, 2025 10:04

mkindahl marked this pull request as ready for review January 17, 2025 10:58

mkindahl force-pushed the fix-index-deadlock branch from 951b92a to 0f5b107 Compare January 17, 2025 10:59

mkindahl requested review from erimatnor, fabriziomello and antekresic January 17, 2025 12:24

erimatnor reviewed Jan 20, 2025

View reviewed changes

src/chunk_index.c Outdated Show resolved Hide resolved

mkindahl force-pushed the fix-index-deadlock branch from 431478a to 4504ac8 Compare January 21, 2025 08:24

mkindahl force-pushed the fix-index-deadlock branch from 4504ac8 to ed951cf Compare January 29, 2025 09:34

mkindahl requested a review from erimatnor January 29, 2025 09:39

mkindahl force-pushed the fix-index-deadlock branch 2 times, most recently from a7bd859 to 8f0c997 Compare January 30, 2025 07:39

erimatnor approved these changes Jan 30, 2025

View reviewed changes

mkindahl force-pushed the fix-index-deadlock branch from 019509c to f0e2cd1 Compare January 30, 2025 08:46

mkindahl force-pushed the fix-index-deadlock branch from f0e2cd1 to e584632 Compare January 30, 2025 08:50

antekresic approved these changes Jan 30, 2025

View reviewed changes

mkindahl merged commit 538c443 into timescale:main Jan 30, 2025
51 of 53 checks passed

mkindahl deleted the fix-index-deadlock branch January 30, 2025 12:43

svenklemm added this to the v2.18.1 milestone Feb 1, 2025

timescale-automation added the backported-2.18.x label Feb 6, 2025

svenklemm mentioned this pull request Feb 10, 2025

Release 2.18.1 #7680

Merged

svenklemm mentioned this pull request Feb 10, 2025

Release 2.18.1 #7681

Merged

bayandin mentioned this pull request Feb 11, 2025

timescaledb 2.18.1 bayandin/homebrew-tap#245

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix lock order when dropping index #7600

Fix lock order when dropping index #7600

mkindahl commented Jan 17, 2025 •

edited

Loading

codecov bot commented Jan 17, 2025 •

edited

Loading

erimatnor left a comment

antekresic left a comment

Fix lock order when dropping index #7600

Fix lock order when dropping index #7600

Conversation

mkindahl commented Jan 17, 2025 • edited Loading

codecov bot commented Jan 17, 2025 • edited Loading

Codecov Report

erimatnor left a comment

Choose a reason for hiding this comment

antekresic left a comment

Choose a reason for hiding this comment

mkindahl commented Jan 17, 2025 •

edited

Loading

codecov bot commented Jan 17, 2025 •

edited

Loading