Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: search objects v2 #626

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: search objects v2 #626

wants to merge 1 commit into from

Conversation

fenos
Copy link
Contributor

@fenos fenos commented Jan 24, 2025

What kind of change does this PR introduce?

Feature

What is the current behavior?

Super slow search functionality when there are millions of objects

What is the new behavior?

Fast, reliable and performant search functionality across hundreds of millions of objects

Copy link

@za-arthur za-arthur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. In overall all looks good to me. I just left couple of comments below.

migrations/tenant/0028-objects-level-index.sql Outdated Show resolved Hide resolved
migrations/tenant/0029-object-bucket-name-sorting.sql Outdated Show resolved Hide resolved
@fenos fenos force-pushed the feat/list-objects-v2 branch from 96b7825 to 831465a Compare January 28, 2025 14:03
@coveralls
Copy link

coveralls commented Jan 28, 2025

Pull Request Test Coverage Report for Build 13197575053

Details

  • 311 of 685 (45.4%) changed or added relevant lines in 18 files are covered.
  • 8 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.8%) to 76.487%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/http/plugins/db.ts 2 3 66.67%
src/storage/events/workers.ts 7 8 87.5%
src/storage/object.ts 77 81 95.06%
src/storage/database/knex.ts 20 27 74.07%
src/internal/database/migrations/progressive.ts 0 9 0.0%
src/storage/events/run-migrations.ts 1 12 8.33%
src/internal/queue/event.ts 1 14 7.14%
src/http/routes/object/listObjectsV2.ts 51 72 70.83%
src/http/routes/admin/migrations.ts 8 37 21.62%
src/http/routes/admin/tenants.ts 15 50 30.0%
Files with Coverage Reduction New Missed Lines %
src/internal/database/migrations/migrate.ts 1 45.79%
src/storage/events/workers.ts 1 56.25%
src/storage/database/knex.ts 6 85.03%
Totals Coverage Status
Change from base Build 12952108566: -0.8%
Covered Lines: 15408
Relevant Lines: 19996

💛 - Coveralls

@fenos fenos force-pushed the feat/list-objects-v2 branch 5 times, most recently from cd1f59e to f7be9ad Compare January 29, 2025 14:18
Copy link

@za-arthur za-arthur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes. LGTM.

@fenos fenos force-pushed the feat/list-objects-v2 branch from f7be9ad to 19df503 Compare February 4, 2025 14:07
Copy link

snyk-io bot commented Feb 4, 2025

Snyk checks have failed. 1 issues have been found so far.

Icon Severity Issues
Critical 0
High 1
Medium 0
Low 0

code/snyk check is complete. 1 issues have been found. (View Details)

@fenos fenos force-pushed the feat/list-objects-v2 branch 4 times, most recently from ca62e66 to 1d2d112 Compare February 7, 2025 08:31
cursor,
})

return response.status(200).send(results)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  Cross-site Scripting (XSS)

Unsanitized input from an HTTP parameter flows into send, where it is used to render an HTML page returned to the user. This may result in a Cross-Site Scripting attack (XSS).

Line 69 | CWE-79 | Priority score 789 | Learn more about this vulnerability
Data flow: 11 steps

Step 1 - 4

const { bucketName } = request.params

Step 5 - 9 src/http/routes/object/listObjectsV2.ts#L62

Step 10 - 11

return response.status(200).send(results)

@fenos fenos force-pushed the feat/list-objects-v2 branch from 1d2d112 to acca57f Compare February 7, 2025 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants