Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

break up actions.AddDataset into remote and qfs #961

Closed
b5 opened this issue Oct 17, 2019 · 0 comments
Closed

break up actions.AddDataset into remote and qfs #961

b5 opened this issue Oct 17, 2019 · 0 comments
Labels
network/p2p peer-2-peer-specific issues, subset of network issues refactor A code change that neither fixes a bug nor adds a feature remote

Comments

@b5
Copy link
Member

b5 commented Oct 17, 2019

actions.AddDataset makes less sense now that we have the remote package, which wants to take over the duty of pushing & pulling a dataset.

AddDataset currently tries both a dsync pull and a p2p fetch in parallel. The registry fetch already uses a remote client, and In practice this means we're only ever pulling from the registry, because the remote client comes back faster. We should change this up so the remote goes first, and if it fails, we fall back to base.FetchDataset.

Long term, we should remove base.FetchDataset entirely, and just rely on the properties of qfs to perform an IPFS add request on our behalf for data we don't have.

One kinda newish requirement shakes out of this: if the remote package is going to fail, it should fail quickly. If a user runs qri add peer/foo --remote peername, and that peer isn't online, we should be moving to p2p fetching ASAP.

Let's close this first qri-io/dag#26. As the first course of action, dsync now bears the responsibility of knowing when no work needs to be done.

@b5 b5 added refactor A code change that neither fixes a bug nor adds a feature network/p2p peer-2-peer-specific issues, subset of network issues remote labels Oct 17, 2019
@b5 b5 added this to the remove actions package milestone Oct 17, 2019
@b5 b5 closed this as completed Oct 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
network/p2p peer-2-peer-specific issues, subset of network issues refactor A code change that neither fixes a bug nor adds a feature remote
Projects
None yet
Development

No branches or pull requests

1 participant