diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9c7aff48ab..384ad3274f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -17,6 +17,14 @@ For licensing reasons and security considerations we try to keep the number of external dependencies (libraries) well-arranged. Try to avoid adding new dependencies unless absolutely necessary. +## Generalized control flow + +![](docs/components.png) + +This flow chart shows a high level control flow common to most operations. +It leaves out a lot of details but should give a better understanding how +information flows within the program and where changes might be necessary. + ## Code Map This section talks briefly about the various directories and some data @@ -112,37 +120,116 @@ during lookup. ### `internal/config` -TODO: backwards compat, loading, overrides, ... +The `config` package implements support for a simple YAML-based configuration +format for `gopass`. Most of the code in this package is for backwards +compatibility. Whenever we introduce or remove a config option we need to +introduce a new fallback version that is automatically attempted when loading +a config file. To resolve ambiguities when parsing different config versions +we use a "catch-all" field to catch any unused keys and check that this is +empty after parsing - otherwise we need to try a different config version. + +NOTE: We did support nested configurations for sub-stores but removed this +because the maintenance cost did not justify the benefits of this feature. ### `internal/cui` -TODO: history, issues, outlook +The name `cui` is an abbreviation for `console-user-interface` and contains +several helper functions to interact with humans over a text based interface. -### `internal/editor` +Most of these ask the user to select some item from a selection or provide +some input. -TODO: security +NOTE: We used to support rich terminal UIs with arrow navigation and such. +However all existing libraries that were available without CGO were either +abandoned or buggy on some platforms and we didn't have any capacity to fix +them. So we had to remove support for this feature. ### `internal/queue` -TODO: motivation, usage +The `queue` package implements a FIFO queue that executes +in the background. This allows for certain operations, like a git push, to be +taken out of the critical path wrt. user interactions. The queue will be fully +processed before the process exits. + +### `internal/store/root` -### `internal/store` +The `root store` package implements an internal password store API that (only) +supports mounting `leaf` stores. It will forward (almost) all operations to +its `leaf` stores (moves across stores being a notable exception) and do the +necessary manipulations of the affected path components (e.g. removing/adding +the mount prefix from the secret name as needed). -TODO: everything +This package makes `gopass` multi-store capable. + +### `internal/store/leaf` + +The `leaf store` package implements a password store that is mostly compatible +with any other password store implementation (while aiming for interoperability, +not at 100% feature parity). The low-level operations like filesystem and / or +version control and crypto operations are passed to the configured `storage` +or `crypto` backend. ### `internal/tree` -TODO: why +The `tree` package implements a simple tree structure that prints an output +similar to the output of the Unix tool `tree`. It does support different +`gopass` specific properties (like mounts or templates) not easily implemented +with other tree packages. ### `internal/updater` -TODO: security +The `updater` package implements a secure and anonymous self updater. + +Note: The self updater contacts GitHub. If this is a concern one should use +other sources, e.g. distro packages. + +It retrieves the latest stable release from GitHub, fetches its metadata +and verifies the signature against the built-in release signing keyring. + +It tries to avoid conflicting with any `gopass` binary managed by the OS +and refuse to update these. -### `pkg/...` +### `pkg/` -TODO: everything +The package `pkg/` contains our public API surface, i.e. packages we want or +have to expose externally. Some packages (e.g. `otp`) are only exposed because +they are being used by some of our integrations. Others (e.g. `pinentry` or +`pwgen`) are designed for wider use. We are considering to split some of the +more widely used packages into their own repositories to work better with +Go module and semantic versioning expectations. + +#### `pkg/appdir` + +The `appdir` package contains a set of [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) +compatible implementations with some `gopass` specifics. For testing purposes +we want to honor the setting of `GOPASS_HOMEDIR` before everything else, so our +implementation has to take this into account before following the XDG spec. + +#### `pkg/clipboard` + +The `clipboard` package is a wrapper around a clipboard package that adds +support for clearing the clipboard. + +#### `pkg/ctxutil` + +The `ctxutil` is the pragmatic (read: non-idiomatic) approach to pass very +specific configuration options through multiple layers of abstraction. This is +arguably not the best design, but it works well and avoids bloated interfaces. + +#### `pkg/gopass` + +This package contains **the** gopass API interface. We provide a concrete +implementation that should work with any properly initialized gopass setup +and a mock for tests. + +This package is designed as the main entry point for any integration that wants +to integrate with gopass. ### `tests` -TODO: integration tests +`gopass` comes with a comprehensive set of integration tests, i.e. tests that +are executed by running a newly compiled gopass binary without access to any +kind of internal state. These tests can't be as exhaustive as the unit tests +but they exist to ensure basic functionality aren't broken by a change. + diff --git a/docs/components.dot b/docs/components.dot new file mode 100644 index 0000000000..58b9338e06 --- /dev/null +++ b/docs/components.dot @@ -0,0 +1,39 @@ +digraph G { + gopass [shape=box,style=filled,color=".2 .2 .6",peripheries=2]; + gopass -> action; + action [label="internal/action"]; + action -> root; + root [label="internal/store/root"]; + root -> leaf; + root -> tree; + tree [label="internal/tree"]; + leaf [label="internal/store/leaf"]; + leaf -> gitfs; + gitfs [label="internal/backend/storage/gitfs"]; + gitfs -> gitcli; + gitcli [label="git binary",shape=Mdiamond]; + leaf -> gpg; + gpg [label="internal/backend/crypto/gpg/cli"]; + leaf -> age [style="dotted"]; + age [label="internal/backend/crypto/age"]; + gpg -> gpgcli; + gpgcli [label="gpg/gpg2 binary",shape=Mdiamond]; + leaf -> secret; + secret [label="pkg/gopass/secrets"]; + secret -> root; + jsonapi [label="gopass-jsonapi",shape=box]; + jsonapi -> api; + api [label="pkg/gopass/api"]; + api -> root; + api -> config; + gopass -> config; + config [label="internal/config"]; + summon -> api; + summon [label="gopass-summon-provider",shape=box]; + hibp -> api; + hibp [label="gopass-hibp",shape=box]; + hibp -> pkghibp; + pkghibp [label="pkg/hibp"]; + gitcreds -> api; + gitcreds [label="git-credential-gopass",shape=box]; +} diff --git a/docs/components.png b/docs/components.png new file mode 100644 index 0000000000..0474b00bcd Binary files /dev/null and b/docs/components.png differ diff --git a/internal/store/leaf/write.go b/internal/store/leaf/write.go index 6f53205562..424a9ad3ac 100644 --- a/internal/store/leaf/write.go +++ b/internal/store/leaf/write.go @@ -60,7 +60,7 @@ func (s *Store) Set(ctx context.Context, name string, sec gopass.Byter) error { } // try to enqueue this task, if the queue is not available - // it will return the task and we will execut it inline + // it will return the task and we will execute it inline t := queue.GetQueue(ctx).Add(func(ctx context.Context) error { return s.gitCommitAndPush(ctx, name) }) diff --git a/internal/tree/root.go b/internal/tree/root.go index e83c16e25f..272b61a42d 100644 --- a/internal/tree/root.go +++ b/internal/tree/root.go @@ -19,9 +19,9 @@ var ( colMount = color.New(color.FgCyan, color.Bold).SprintfFunc() colDir = color.New(color.FgBlue, color.Bold).SprintfFunc() colTpl = color.New(color.FgGreen, color.Bold).SprintfFunc() - //colBin = color.New(color.FgYellow, color.Bold).SprintfFunc() - //colYaml = color.New(color.FgCyan, color.Bold).SprintfFunc() - sep = "/" // this should not be platform-agnostic. this is for the CLI interface + // sep is intentionally NOT platform-agnostic. This is used for the CLI output + // and should always be a regular slash. + sep = "/" ) // Root is the root of a tree