Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow for parent chunks #312

Open
mhordynski opened this issue Jan 27, 2025 · 0 comments
Open

feat: allow for parent chunks #312

mhordynski opened this issue Jan 27, 2025 · 0 comments
Assignees
Labels
feature New feature or request

Comments

@mhordynski
Copy link
Member

Feature description

Ragbits' DocumentSearch should provide a way to store information about parent chunks.

Parent chunks is approach to chunking where you divide one bigger chunk to smaller child chunks. Based on that how many of child chunks from parent chunks were retrieved, pipeline may decide to replace child chunks with full content of parent chunk.

Proposed approach is to leverage VectorStoreEntry metadata fields to store information about parent chunk, and based on that have result postprocessor replacing child chunks with parent when all criteria matches.

Motivation

Parent chunks is method of improving retrieval performance that previously worked well in projects by deepsense.ai

Additional context

No response

@mhordynski mhordynski added the feature New feature or request label Jan 27, 2025
@mhordynski mhordynski moved this to Backlog in ragbits Jan 27, 2025
@mhordynski mhordynski changed the title feat: feat: allow for parent chunks Jan 27, 2025
@mhordynski mhordynski added this to the 9 / Advanced RAG milestone Jan 27, 2025
@mhordynski mhordynski self-assigned this Jan 27, 2025
@mhordynski mhordynski moved this from Backlog to Ready in ragbits Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: Ready
Development

No branches or pull requests

1 participant