diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 66874065..1e01a996 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -57,7 +57,21 @@ The definition of "trivial" is intentionally slightly ambiguous, and intended to An example of a trivial change is fixing a typo in the README, or bumping a version of a dependency used by the continuous integration suite. If another contributor takes issue with a change merged in this fashion, simply commenting politely that they have concerns about the change (either in an issue or directly) is the right remedy. -## Test-Specific Changes +## Writing Good Tests + +Be familiar with the test structure and assumptions documented in the [README](README.md). + +Test cases should include both valid and invalid instances which exercise the test case schema whenever possible. +Exceptions of course include schemas where only one result is ever possible (such as the `false` schema, or ones using keywords which only produce annotations). + +Schemas should be *minimal*, by which we mean that they should contain only those keywords which are being tested by the specific test case, and should not contain complex values when simpler ones would do. +The same applies to instances -- prefer simpler instances to more complex ones, and when testing string instances, consider using ones which are self-descriptive whenever it aids readability. + +Comments can and should be used to explain tests which are unclear or complex. +The `comment` field is present both for test cases and individual tests for this purpose. +Links to the relevant specification sections are also encouraged, though they can be tedious to maintain from one version to the next. + +When adding test cases, they should be added to all past (and future) versions of the specification which they apply to, potentially with minor modifications (e.g. changing `$id` to `id` or accounting for `$ref` not allowing siblings on older drafts). Changing the schema used in a particular test case should be done with extra caution, though it is not formally discouraged if the change simplifies the schema. Contributors should not generally append *additional* behavior to existing test case schemas, unless doing so has specific justification.