Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The end of undefined docSync() / await doc() #402

Merged
merged 60 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
7a57d33
merge by removing heads support in find for now
pvh Jan 27, 2025
351ef49
wip -- working on tests. there's an asymmetry to document discovery p…
pvh Dec 31, 2024
3742fb2
wip -- heaps of console logs but fixed a subtle networking race condi…
pvh Dec 31, 2024
6c61a79
one failing test left
pvh Dec 31, 2024
8b7ecfd
fix those two tests
pvh Jan 3, 2025
1d454b1
throw on not ready
pvh Dec 31, 2024
69e118e
remove unhelpful test and some cnosole.logs
pvh Jan 4, 2025
7a22e4b
all tests and tsc passing
pvh Jan 4, 2025
858ae3a
alright, find is now abortable
pvh Jan 4, 2025
f6779f0
added AbortController support -- couple of failing tests but it's too…
pvh Jan 4, 2025
a7979e6
fix abort test
pvh Jan 4, 2025
f54c29f
add a check that aborting also prevents going to unavailable
pvh Jan 4, 2025
d258cbb
add a check that aborting also prevents going to unavailable
pvh Jan 4, 2025
f1bb754
wip of new useDoc/Handle
pvh Jan 4, 2025
89d1905
working on the react-hooks rewrite, still messy in here
pvh Jan 5, 2025
8687593
okay, tests passing. still more to do
pvh Jan 5, 2025
3986420
working on tests
pvh Jan 6, 2025
22a7e68
alright, more passing tests
pvh Jan 6, 2025
cc6e1fb
remove a duplicate test
pvh Jan 6, 2025
0900210
fix the test that was corrupting the shared repo
pvh Jan 6, 2025
b8e790f
usedocument passing all tests too
pvh Jan 6, 2025
d1242d1
continued cleanup
pvh Jan 6, 2025
501d718
cleaned up the comment
pvh Jan 6, 2025
725407c
refactoring .doc away here
pvh Jan 6, 2025
14f0242
failed wip
pvh Jan 6, 2025
80a5e31
useDocuments passing some basic tests
pvh Jan 6, 2025
02d584e
one failing test: suspense false on useDocHandles
pvh Jan 6, 2025
e1cddb5
update react-todo
pvh Jan 6, 2025
fcbaa83
update react-counter example, too
pvh Jan 6, 2025
82a38c1
updating other packages
pvh Jan 6, 2025
38ad944
clean up some warnings
pvh Jan 6, 2025
f8b97a5
fix a couple comments I missed
pvh Jan 6, 2025
7cd6d3c
fix a couple tests broken by moving to std. error boundary
pvh Jan 6, 2025
1a85f61
fix useDocHandle
pvh Jan 7, 2025
ebdb920
okay, more tests passing
pvh Jan 7, 2025
df1eb04
okay, more decent tests for useDocuments
pvh Jan 7, 2025
c575afb
working on an async generator with progress version of find
pvh Jan 7, 2025
4e4b5c8
restore abortcontroller support
pvh Jan 7, 2025
ad779ac
the patch is in a messy state but tests are passing
pvh Jan 8, 2025
f026bad
missed a file
pvh Jan 8, 2025
6a0866b
fix merge damage
pvh Jan 29, 2025
f067ee0
fix tests for urls in heads
pvh Jan 29, 2025
7cd4ce9
working on fixing more tests, as usual
pvh Jan 29, 2025
15bd160
update some dependencies
pvh Jan 29, 2025
d5fff74
test fixing
pvh Jan 29, 2025
ea784cd
updates necessary to boot patchwork
pvh Jan 30, 2025
f5ccf14
fix tests
pvh Jan 30, 2025
d29e59b
quick mucking around with testSetup and just import where we need it
pvh Jan 31, 2025
be4352a
bump to .21
pvh Jan 31, 2025
2166114
implement some style nitpicks spotted by maciek
pvh Jan 31, 2025
33b3044
further nits and add eslint react checking... i think?
pvh Jan 31, 2025
b75b1b4
fix some more linter complaints
pvh Jan 31, 2025
1e4ee1f
fix some asyncs
pvh Jan 31, 2025
6d5d5e5
comments
pvh Jan 31, 2025
28876d4
clean up missing doc handling
pvh Jan 31, 2025
e48009c
fix tests
pvh Jan 31, 2025
caeb6aa
use an ErrorBoundary implementation that doesn't log to console
pvh Jan 31, 2025
147a041
some of our tests require react 18.3 so i'm just going to make it the…
pvh Jan 31, 2025
8fd83be
block node 22 (broadcast channel broke?), try another way of quietyin…
pvh Jan 31, 2025
c92830a
missed an only
pvh Jan 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/react-counter/src/main.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ declare global {
const rootDocUrl = `${document.location.hash.substring(1)}`
let handle
if (isValidAutomergeUrl(rootDocUrl)) {
handle = repo.find(rootDocUrl)
handle = await repo.find(rootDocUrl)
} else {
handle = repo.create<{ count: number }>({ count: 0 })
}
Expand Down
3 changes: 2 additions & 1 deletion examples/react-todo/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@
"classnames": "^2.3.2",
"postcss": "^8.4.21",
"react": "^18.2.0",
"react-dom": "^18.2.0"
"react-dom": "^18.2.0",
"react-error-boundary": "^5.0.0"
},
"devDependencies": {
"tailwindcss": "^3.2.4"
Expand Down
28 changes: 15 additions & 13 deletions examples/react-todo/src/App.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { AutomergeUrl } from "@automerge/automerge-repo"
import { useDocument, useRepo } from "@automerge/automerge-repo-react-hooks"
import cx from "classnames"
import { useRef, useState } from "react"
import { Suspense, useRef, useState } from "react"

import { Todo } from "./Todo.js"
import { ExtendedArray, Filter, State, TodoData } from "./types.js"
Expand All @@ -28,7 +28,7 @@ export function App({ url }: { url: AutomergeUrl }) {
if (!state) return []
return state.todos.filter(async url => {
if (filter === Filter.all) return true
const todo = await repo.find<TodoData>(url).doc()
const todo = (await repo.find<TodoData>(url)).doc()
if (filter === Filter.completed) return todo.completed
if (filter === Filter.incomplete) return !todo.completed
return false
Expand All @@ -38,7 +38,7 @@ export function App({ url }: { url: AutomergeUrl }) {
const destroyCompleted = async () => {
if (!state) return
for (const url of await getFilteredTodos(Filter.completed)) {
const todo = await repo.find<TodoData>(url).doc()
const todo = (await repo.find<TodoData>(url)).doc()
if (todo.completed) destroy(url)
}
}
Expand Down Expand Up @@ -89,16 +89,18 @@ export function App({ url }: { url: AutomergeUrl }) {

{/* todos */}
<section>
<ul className="border-y divide-y divide-solid">
{state.todos.map(url => (
<Todo
key={url}
url={url}
onDestroy={url => destroy(url)}
filter={filter}
/>
))}
</ul>
<Suspense fallback={<li>Loading todo items...</li>}>
<ul className="border-y divide-y divide-solid">
{state.todos.map(url => (
<Todo
key={url}
url={url}
onDestroy={url => destroy(url)}
filter={filter}
/>
))}
</ul>
</Suspense>
</section>

{/* footer tools */}
Expand Down
11 changes: 8 additions & 3 deletions examples/react-todo/src/main.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ import { BroadcastChannelNetworkAdapter } from "@automerge/automerge-repo-networ
import { BrowserWebSocketClientAdapter } from "@automerge/automerge-repo-network-websocket"
import { RepoContext } from "@automerge/automerge-repo-react-hooks"
import { IndexedDBStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb"
import React from "react"
import React, { Suspense } from "react"
import { ErrorBoundary } from "react-error-boundary"
import ReactDOM from "react-dom/client"
import { App } from "./App.js"
import { State } from "./types.js"
Expand All @@ -27,7 +28,7 @@ declare global {
const rootDocUrl = `${document.location.hash.substring(1)}`
let handle
if (isValidAutomergeUrl(rootDocUrl)) {
handle = repo.find(rootDocUrl)
handle = await repo.find(rootDocUrl)
} else {
handle = repo.create<State>({ todos: [] })
}
Expand All @@ -38,7 +39,11 @@ window.repo = repo
ReactDOM.createRoot(document.getElementById("root") as HTMLElement).render(
<RepoContext.Provider value={repo}>
<React.StrictMode>
<App url={docUrl} />
<ErrorBoundary fallback={<div>Something went wrong</div>}>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this officialish error boundary is chatty in the logs. i should silence it and/or replace it with one that is better for tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not much to it—for my project, I ended up just borrowing from it liberally and writing my own. But I think that can wait.

<Suspense fallback={<div>Loading...</div>}>
<App url={docUrl} />
</Suspense>
</ErrorBoundary>
</React.StrictMode>
</RepoContext.Provider>
)
10 changes: 5 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@automerge/automerge-repo-monorepo",
"version": "2.0.0-alpha.20",
"version": "2.0.0-alpha.21",
"description": "Automerge Repo monorepo",
"main": "packages/automerge-repo/dist/index.js",
"repository": "https://github.com/automerge/automerge-repo",
Expand Down Expand Up @@ -46,7 +46,7 @@
"@types/ws": "^8.5.10",
"@typescript-eslint/eslint-plugin": "^7.13.1",
"@typescript-eslint/parser": "^7.13.1",
"@vitejs/plugin-react": "^4.3.1",
"@vitejs/plugin-react": "^4.3.4",
"@vitest/coverage-v8": "^3.0.3",
"@vitest/ui": "^3.0.3",
"c8": "^7.14.0",
Expand All @@ -68,8 +68,8 @@
"ts-node": "^10.9.2",
"typedoc": "^0.25.12",
"typescript": "^5.4.2",
"vite": "^5.2.0",
"vite-plugin-wasm": "^3.3.0",
"vitest": "^3.0.3"
"vite": "^6.0.11",
"vite-plugin-wasm": "^3.4.1",
"vitest": "^3.0.4"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -520,8 +520,7 @@ describe("Websocket adapters", () => {
})

// make a change to the handle on the sync server
const handle = repo.find<{ foo: string }>(url)
await handle.whenReady()
const handle = await repo.find<{ foo: string }>(url)
handle.change(d => (d.foo = "baz"))

// Okay, so now there is a document on both the client and the server
Expand Down
2 changes: 2 additions & 0 deletions packages/automerge-repo-react-hooks/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@
"react-usestateref": "^1.0.8"
},
"devDependencies": {
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^14.0.0",
"jsdom": "^22.1.0",
"react-error-boundary": "^5.0.0",
"rollup-plugin-visualizer": "^5.9.3",
"vite": "^6.0.7",
"vite-plugin-dts": "^3.9.1"
Expand Down
3 changes: 2 additions & 1 deletion packages/automerge-repo-react-hooks/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
*/
export { useDocument } from "./useDocument.js"
export { useDocuments } from "./useDocuments.js"
export { useHandle } from "./useHandle.js"
export { useDocHandle } from "./useDocHandle.js"
export { useDocHandles } from "./useDocHandles.js"
export { RepoContext, useRepo } from "./useRepo.js"
export {
useLocalAwareness,
Expand Down
76 changes: 76 additions & 0 deletions packages/automerge-repo-react-hooks/src/useDocHandle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
import { AnyDocumentId, DocHandle } from "@automerge/automerge-repo/slim"
import { PromiseWrapper, wrapPromise } from "./wrapPromise.js"
import { useRepo } from "./useRepo.js"
import { useEffect, useRef, useState } from "react"

// Shared with useDocHandles
export const wrapperCache = new Map<
AnyDocumentId,
PromiseWrapper<DocHandle<unknown>>
>()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want this global, and not, e.g., living in something that also wraps a RepoContext.Provider?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooooh. that might be better. it would also be a bit more complicated... my immediate instinct is to defer this until someone asks for it. the failure mode would be that if you had the same document in two different repos in your react app they'd share a cache of their state.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would be hard to debug and understand for sure. should this be part of the repo context? i seem to remember in one version of this patch trying to rely on the consistency of the promises from repo.find() ... but then you can't resolve it in two places so i guess actually we need the cache here. hmmm.

i think i'd prefer to leave this as-is because the failure mode seems pretty remote and a robust solution seems complicated but i could be persuaded this is the wrong idea.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my weak compromise was to add the comment below. if you find this too odious, i'm open to improving on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, okay. Though for what it's worth, I use context for stuff like this all the time, though, and I've never run into problems. The odds of anyone running into a problem here are pretty remote, but if they do, it's going to be a fun debugging time for sure. Plus, I think introducing an AutomergeContext with this breaking change lets us put other per-repo state (that does not belong in a repo) in this context in the future without breaking existing APIs at that point. But I don't think it's a huge deal; it'd be a pretty easy migration if necessary.

// NB: this is a global cache that isn't keyed on the Repo
// so if your app uses the same documents in two Repos
// this could cause problems. please let me know if you do.

interface UseDocHandleSuspendingParams {
suspense: true
}
interface UseDocHandleSynchronousParams {
suspense: false
}

type UseDocHandleParams =
| UseDocHandleSuspendingParams
| UseDocHandleSynchronousParams

export function useDocHandle<T>(
id: AnyDocumentId,
params: UseDocHandleSuspendingParams
): DocHandle<T>
export function useDocHandle<T>(
id: AnyDocumentId | undefined,
params?: UseDocHandleSynchronousParams
): DocHandle<T> | undefined
export function useDocHandle<T>(
id: AnyDocumentId | undefined,
{ suspense }: UseDocHandleParams = { suspense: false }
): DocHandle<T> | undefined {
const repo = useRepo()
const controllerRef = useRef<AbortController>()
const [handle, setHandle] = useState<DocHandle<T> | undefined>()

let wrapper = id ? wrapperCache.get(id) : undefined
if (!wrapper && id) {
controllerRef.current?.abort()
controllerRef.current = new AbortController()

const promise = repo.find<T>(id, { signal: controllerRef.current.signal })
wrapper = wrapPromise(promise)
wrapperCache.set(id, wrapper)
}

/* From here we split into two paths: suspense and not.
* In the suspense path, we return the wrapper directly.
* In the non-suspense path, we wait for the promise to resolve
* and then set the handle via setState. Suspense relies on
* re-running this function until it succeeds, whereas the synchronous
* form uses a setState to track the value. */
useEffect(() => {
if (suspense || !wrapper) {
return
}
wrapper.promise
.then(handle => {
setHandle(handle as DocHandle<T>)
})
.catch(() => {
setHandle(undefined)
})
}, [suspense, wrapper])

if (suspense && wrapper) {
return wrapper.read() as DocHandle<T>
} else {
return handle
}
}
75 changes: 75 additions & 0 deletions packages/automerge-repo-react-hooks/src/useDocHandles.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
import { AutomergeUrl, DocHandle } from "@automerge/automerge-repo/slim"
import { useState, useEffect } from "react"
import { useRepo } from "./useRepo.js"
import { PromiseWrapper, wrapPromise } from "./wrapPromise.js"
import { wrapperCache } from "./useDocHandle.js"

interface UseDocHandlesParams {
suspense?: boolean
}

type DocHandleMap<T> = Map<AutomergeUrl, DocHandle<T> | undefined>

export function useDocHandles<T>(
ids: AutomergeUrl[],
{ suspense = false }: UseDocHandlesParams = {}
): DocHandleMap<T> {
const repo = useRepo()
const [handleMap, setHandleMap] = useState<DocHandleMap<T>>(() => new Map())

const pendingPromises: PromiseWrapper<DocHandle<T>>[] = []
const nextHandleMap = new Map<AutomergeUrl, DocHandle<T> | undefined>()

// Check if we need any new wrappers
for (const id of ids) {
let wrapper = wrapperCache.get(id)!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why !? Aren't you testing if (!wrapper) right below?

if (!wrapper) {
try {
const promise = repo.find<T>(id)
wrapper = wrapPromise(promise)
wrapperCache.set(id, wrapper)
} catch (e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this catching?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unavailable. this should probably have a test, now i think about it. i'm not sure what the behaviour would be without thinking hard which suggests they're probably wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that thrown? I'm probably missing something, but as far as I can tell, neither find nor wrapPromise throw synchronously.

continue
}
}

// Try to read each wrapper.
// Update handleMap with any available handles,
// and collect any pending promises
try {
const handle = wrapper.read() as DocHandle<T>
nextHandleMap.set(id, handle)
} catch (e) {
if (e instanceof Promise) {
pendingPromises.push(wrapper as PromiseWrapper<DocHandle<T>>)
} else {
nextHandleMap.set(id, undefined)
}
}
}

// If any promises are pending, suspend with Promise.all
if (suspense && pendingPromises.length > 0) {
throw Promise.all(pendingPromises.map(p => p.promise))
}

useEffect(() => {
if (pendingPromises.length > 0) {
void Promise.allSettled(pendingPromises.map(p => p.promise)).then(
handles => {
handles.forEach(r => {
if (r.status === "fulfilled") {
const h = r.value as DocHandle<T>
nextHandleMap.set(h.url, h)
}
})
setHandleMap(nextHandleMap)
}
)
} else {
setHandleMap(nextHandleMap)
}
}, [suspense, ids])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're omitting pendingPromises and nextHandleMap from the useEffect dependencies. And suspense and ids are there even though they're not actually referenced in the useEffect function body. I think your logic checks out, but if you add eslint-plugin-react-hooks at some point, it will complain. Of course, you can't just add those deps since those variables are re-initialized on every render, but I think this at least deserves a comment for the poor soul who may have to debug this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having that plugin would have saved me a couple stupid hours the other day, so i guess i will install it and fix the gripes!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright, well, that's half an hour of my life. i think the bigger problem is that the effect probably should be managed differently. adding this to my TODO list at least.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're still omitting pendingPromises and nextHandleMap from the dependency array. I think your hook lint isn't running here. I remember when I was in a spot like this that refactoring to avoid that is not trivial, and the logic seems sound, so maybe throw an eslint-ignore-next-line on it once you get linting working for now.


return handleMap
}
Loading