Fix error when storing embeddings in localStorage #96
Conversation
Fixes #95 Update `getContextVectorStore` to use `IndexedDB` instead of `localStorage` for storing embeddings. * **src/lib/getMatchedContent.ts** - Import `getFromIndexedDB` and `saveToIndexedDB` from `useStorage.ts`. - Replace `localStorage` usage with `IndexedDB` functions for storing and retrieving embeddings. * **src/hooks/useStorage.ts** - Add `saveToIndexedDB` function to save data to `IndexedDB`. - Add `getFromIndexedDB` function to retrieve data from `IndexedDB`. --- For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/Royal-lobster/Syncia/issues/95?shareId=XXXX-XXXX-XXXX-XXXX).
|
|
🦋 Changeset detectedLatest commit: e3e1e87 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Warning Rate limit exceeded@Royal-lobster has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 7 minutes and 24 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. WalkthroughThe changes introduce two asynchronous functions, Changes
Assessment against linked issues
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Outside diff range and nitpick comments (4)
src/lib/getMatchedContent.ts (2)
Line range hint
11-20: Consider adding error handling for storage operations.Since the function now depends on IndexedDB operations which can fail (e.g., quota exceeded, browser compatibility issues), consider implementing error handling to gracefully handle storage failures.
export const getMatchedContent = async ( query: string, context: string, apiKey: string, baseURL: string, ) => { + try { const vectorStore = await getContextVectorStore(context, apiKey, baseURL) const retriever = vectorStore.asRetriever() const relevantDocs = await retriever.getRelevantDocuments(query) return relevantDocs.map((doc) => doc.pageContent).join('\n') + } catch (error) { + console.error('Failed to process content:', error) + throw new Error('Failed to process content. Please try again.') + } }
Line range hint
23-54: Consider implementing a cache expiration strategy.Since IndexedDB has higher storage limits than localStorage, consider implementing a cache expiration strategy to prevent unlimited growth over time. This could include:
- Timestamp-based expiration
- Size-based cleanup
- LRU (Least Recently Used) eviction policy
Would you like me to provide an implementation example for any of these strategies?
src/hooks/useStorage.ts (2)
111-139: Consider implementing database version management and data cleanup.The current implementation lacks:
- A strategy for database version management
- Cleanup of old/unused embeddings
Consider:
- Implementing a version management system to handle schema changes
- Adding a cleanup mechanism to prevent unlimited storage growth
- Moving database configuration to a separate module
Example configuration module:
// db-config.ts export const DB_CONFIG = { name: 'SynciaDB', version: 1, stores: { embeddings: 'embeddings' } } as const; export interface DBSchema { embeddings: { key: string; value: unknown; timestamp: number; }; }
111-172: Refactor database operations into a reusable service.The current implementation has duplicate code and could benefit from a more structured approach.
Consider creating a database service class:
class IndexedDBService { private readonly dbName = 'SynciaDB'; private readonly version = 1; private readonly storeName = 'embeddings'; private db: IDBDatabase | null = null; private async getConnection(): Promise<IDBDatabase> { if (this.db) return this.db; return new Promise((resolve, reject) => { const request = indexedDB.open(this.dbName, this.version); request.onupgradeneeded = (event) => { const db = (event.target as IDBOpenDBRequest).result; if (!db.objectStoreNames.contains(this.storeName)) { db.createObjectStore(this.storeName); } }; request.onsuccess = () => { this.db = request.result; resolve(request.result); }; request.onerror = () => reject(request.error); }); } async save<T>(key: string, data: T): Promise<void> { const db = await this.getConnection(); const transaction = db.transaction(this.storeName, 'readwrite'); const store = transaction.objectStore(this.storeName); return new Promise((resolve, reject) => { const request = store.put(data, key); request.onsuccess = () => resolve(); request.onerror = () => reject(request.error); }); } async get<T>(key: string): Promise<T | null> { const db = await this.getConnection(); const transaction = db.transaction(this.storeName, 'readonly'); const store = transaction.objectStore(this.storeName); return new Promise((resolve, reject) => { const request = store.get(key); request.onsuccess = () => resolve(request.result ?? null); request.onerror = () => reject(request.error); }); } close(): void { if (this.db) { this.db.close(); this.db = null; } } } const dbService = new IndexedDBService(); export const saveToIndexedDB = <T>(key: string, data: T) => dbService.save(key, data); export const getFromIndexedDB = <T>(key: string) => dbService.get<T>(key);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (2)
src/hooks/useStorage.ts(1 hunks)src/lib/getMatchedContent.ts(3 hunks)
🔇 Additional comments (2)
src/lib/getMatchedContent.ts (1)
5-5: LGTM: Clean import of storage utilities.
The import statement correctly brings in the new IndexedDB utility functions.
src/hooks/useStorage.ts (1)
111-172: Verify the impact of IndexedDB migration.
The transition from localStorage to IndexedDB might affect existing users.
Let's verify the migration impact:
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes #95
Update
getContextVectorStoreto useIndexedDBinstead oflocalStoragefor storing embeddings.src/lib/getMatchedContent.ts
getFromIndexedDBandsaveToIndexedDBfromuseStorage.ts.localStorageusage withIndexedDBfunctions for storing and retrieving embeddings.src/hooks/useStorage.ts
saveToIndexedDBfunction to save data toIndexedDB.getFromIndexedDBfunction to retrieve data fromIndexedDB.For more details, open the Copilot Workspace session.
Summary by CodeRabbit