Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#207 rollup compaction simplified #225

Conversation

alexeiakimov
Copy link
Contributor

Description

A simplified approach to multi-block files and the rollup compaction #207. It is not a full solution, but an initial attempt to use multi-block files for indexing and queries. Replication is out of scope.

@alexeiakimov alexeiakimov self-assigned this Oct 30, 2023
@alexeiakimov alexeiakimov changed the title #207 AddFile metadata format is prepared for multiple blocks. #207 rollup compaction simplified Oct 30, 2023
Copy link

codecov bot commented Oct 31, 2023

Codecov Report

Merging #225 (38edfe9) into main (f066acf) will decrease coverage by 3.73%.
The diff coverage is 91.53%.

❗ Current head 38edfe9 differs from pull request most recent head aa1193a. Consider uploading reports for the commit aa1193a to get more accurate results

@@            Coverage Diff             @@
##             main     #225      +/-   ##
==========================================
- Coverage   92.00%   88.27%   -3.73%     
==========================================
  Files          88       94       +6     
  Lines        2214     2414     +200     
  Branches      168      169       +1     
==========================================
+ Hits         2037     2131      +94     
- Misses        177      283     +106     
Files Coverage Δ
.../scala/io/qbeast/core/model/IndexFileBuilder.scala 100.00% <100.00%> (ø)
...cala/io/qbeast/core/model/PointWeightIndexer.scala 100.00% <100.00%> (ø)
...n/scala/io/qbeast/core/model/RevisionClasses.scala 91.11% <ø> (-2.23%) ⬇️
...c/main/scala/io/qbeast/context/QbeastContext.scala 94.73% <100.00%> (ø)
...io/qbeast/core/model/BroadcastedTableChanges.scala 100.00% <100.00%> (ø)
...la/io/qbeast/spark/delta/DeltaMetadataWriter.scala 93.65% <100.00%> (-0.10%) ⬇️
...la/io/qbeast/spark/delta/DeltaQbeastSnapshot.scala 96.66% <100.00%> (+2.91%) ⬆️
...cala/io/qbeast/spark/delta/DeltaStagingUtils.scala 100.00% <100.00%> (ø)
...ala/io/qbeast/spark/delta/IndexStatusBuilder.scala 100.00% <100.00%> (ø)
.../main/scala/io/qbeast/spark/delta/OTreeIndex.scala 86.00% <100.00%> (-1.04%) ⬇️
... and 19 more

... and 7 files with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

@alexeiakimov
Copy link
Contributor Author

alexeiakimov commented Nov 14, 2023

This PR is closed because of incorrect target branch. The correct PR is #232

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants