This library provides a flexible way of defining JSON schemas that can be used for validation of JSON objects.
Built on top of yup
schema builder, it provides an unlimited level of flexibility when defining custom data types.
It combines the simplicity of JSON schema definition with the unlimited possibilities for fine-grained custom type definition.
"Schemas" are JSON objects describing a data structure. They can be used to validate input or output data.
import schemaValidation from 'flexible-json-schema'
// Create a validation function.
const validate = schemaValidation({
"name": {
"type": "string",
"description": "User's name",
"example": "John Smith"
},
"age": {
"type": "number",
"description": "User's age",
"example": 18
},
...
})
// Returns the data object if it's valid.
// Throws an error if the data object is not valid.
return validate({
name: 'Alex Jones',
age: 38,
...
})
The data could be an object or it could be a non-object.
The tree structure of a schema should correspond to the tree structure of the data.
Each property in a schema should correspond to the same property in the data, describing it via a "property descriptor" — an object specifying:
- Type descriptor — Describes the type of the value. Could be either a "simple"
type
property having a value like"string"
/"number"
/ etc, or a more "complex" property likearrayOf
/oneOf
/objectOf
/schema
. - Description — A human-readable description of the property, for documentation purposes.
- Required or not? — (optional) A boolean
required
flag. Isrequired: true
by default.required: true
— The property must exist and its value can't benull
or an empty string.required: false
— The property can be omitted and its value can benull
.
- Can be
null
or not? — (optional) (advanced) A booleannullable
flag. Has no default behavior. Can be used to differentiate betweennull
andundefined
values:required: true
andnullable: true
— The property must exist but its value is allowed to benull
.required: false
andnullable: false
— The property can be omitted but its value can't benull
when it exists.
Base value types:
"number"
— A number (could be fractional)."integer"
— An integer."positiveNumber"
— A "positive" number."positiveInteger"
— A "positive" integer."nonNegativeNumber"
— A "non-negative" number."nonNegativeInteger"
— A "non-negative" integer."boolean"
—true
orfalse
."string"
— A string. Specifically, a non-empty string, unlessallowEmptyStrings: true
option is passed."date"
— ADate
instance."dateString"
— ADate
string, in ISO format:YYYY-MM-DDTHH:mm:ss.sssZ
.
Utility value types:
"email"
— An email address. Example:"[email protected]"
."url"
— A valid absolute URL. Example:"https://google.com"
."relativeUrl"
— A valid relative URL. Example:"/users/123"
.
Special case types:
null
— Only allowsnull
value. Can be used in conjunction withrequired: false
to define properties that should be missing or set tonull
.
By default, empty strings aren't allowed. To allow empty strings, pass allowEmptyStrings: true
option. This will allow empty strings, although they'll still be interpreted as "missing values", so if a property should support an empty string value then it should also be declared as required: false
.
const validate = schemaValidation(schema, {
allowEmptyStrings: true
})
By default, all type: "date"
properties are required to be instances of Date
class. However, when JSON objects are stringified, all Date
properties are converted to "date ISO strings" like "2000-01-01T00:00:00.000Z"
. To use type: "date"
for those stringified dates, pass dateStrings: true
option.
const validate = schemaValidation(schema, {
dateStrings: true
})
Another supported date string format is "yyyy-mm-dd"
.
const validate = schemaValidation(schema, {
dateStrings: true,
dateFormat: 'yyyy-mm-dd'
})
The validation function could also convert dates from their stringified representations to Date
instances. To do that, pass convertDates: true
option instead of dateStrings: true
option.
const schema = {
date: {
type: "date",
description: "Date"
}
}
const validateAndConvertDates = schemaValidation(schema, {
convertDates: true,
// dateFormat: 'yyyy-mm-dd'
})
const data = {
date: '2000-01-01T00:00:00.000Z'
// date: '2000-01-01'
}
// Mutates the data object.
validateAndConvertDates(data)
data.date instanceof Date === true
To extend the default list of value types, or to override some of them, one can use useCustomTypes()
function.
To create custom types, one could use the basic yup
type constructors that're re-exported from flexible-json-schema/core
subpackage, along with a few "utility" functions that're exported from flexible-json-schema/type
subpackage.
import schemaValidation, { useCustomTypes } from 'flexible-json-schema'
// These are simply re-exports from `yup`.
//
// See `yup` docs for defining custom types:
// https://www.npmjs.com/package/yup
//
import {
string,
boolean,
number,
date,
object,
array
} from 'flexible-json-schema/core';
// These are utility functions that could be used to create custom types.
import {
arrayOfOneOf,
conditional,
depends,
filter,
oneOf,
regexp
} from 'flexible-json-schema/type';
useCustomTypes({
// A `yup` type definition. See `yup` docs for more info.
"currencyAmount": number().min(0),
// `oneOf()` creates a type that could be any value from the list.
"currencyType": oneOf(["USD", "CAD"]),
// `arrayOfOneOf()` creates a type that could be an array of the values from the list.
// Supported options: `allowEmpty: true` / `nonEmpty: false`.
"colors": arrayOfOneOf(["red", "green", "blue"]),
// `conditional()` dynamically creates a type based on the value.
// Example: Suppose that `rating` could only be `0...10` or `999`.
"rating": conditional((value) => {
if (value >= 0 && value <= 10) {
return number().min(0).max(10);
}
return number().min(999).max(999);
}),
// `depends()` defines a property that depends on another property.
// Example: a list of possible country regions depends on the country.
"country": oneOf(['US', 'CA']),
"countryRegion": depends(['country'], string(), ([country], type) => {
if (country === 'US') {
return oneOf(type, ['TX', 'CA', ...]);
} else {
return oneOf(type, ['ON', 'NS', ...]);
}
}),
// `filter()` defines a type that must satisfy some condition.
"phone": filter(string(), value => value.length === 10),
// `regexp()` defines a string that must match a regular expression.
// By default, empty strings aren't allowed.
// Supported options: `allowEmpty: true` / `nonEmpty: false`.
"url": regexp(/^https?:\/\//)
})
const schema = {
amount: {
type: "currencyAmount",
description: "Currency amount"
},
currency: {
type: "currencyType",
description: "Currency type"
}
}
const validateMoney = schemaValidation(schema)
validateMoney({ amount: 100.50, currency: 'USD' })
When not using the "utility" functions that're expored from flexible-json-schema/type
subpackage, or when there's no suitable function there, one could define custom types using just the functions provided by yup
. This way it would be much more flexible but also more complex.
Refer to yup
documentation for more info on defining custom types.
Common examples:
- An integer between 1 and 10:
number().integer().min(1).max(10)
- An HTTP(S) URL:
string().matches(/^https?:\/\//)
- One of "x", "y", "z" (or empty, when
required: false
):string().oneOf([null, "x", "y", "z"])
- An array of colors:
array().of(string().oneOf("red", "green", "blue"))
Any validation logic via .test()
// Validates US phone numbers written in `+1` notation.
// Example: "+12133734253".
function isValidUsPhoneNumber(value) {
return value.length === 12 && value.startsWith("+1")
}
useCustomTypes({
"phone": string().test(
'isUsPhoneNumber',
'${path} is not a US phone number',
(value) => {
// A workaround for a `yup` quirk:
// `test()` function is also run for `undefined` or `null` values.
// https://github.com/jquense/yup/issues/1055
if (value === undefined || value === null) {
// Ignore `undefined` or `null` values:
// those ones should still pass the test
// when the property is declared as `required: false`.
return true
}
return isValidUsPhoneNumber(value)
}
)
})
The examples above are considered quite simple yup
type definitions. In some cases though, a property type can't be described in such simple terms. For example, a property value could be either a number
or a string
, but both are considered valid in general. Or there might be cases when one property value could depend on some other property value.
In such complex cases, a property type definition could be implemented as a type constructor function:
function (fromYupType, { schemaEntry }) {
return fromYupType(...)
}
fromYupType()
function should wrap the finalyup
type returned from the type contructor function.
schemaEntry
is the "schema entry" for this property. It could be used to read any options from the "schema entry" definition and then act accordingly.
Using yup
lazy()
function to dynamically define property type based on the property value.
useCustomTypes({
"booleanOrObject": (fromYupType) => {
return lazy((value) => {
switch (typeof value) {
case 'boolean':
return fromYupType(boolean())
default:
return fromYupType(object())
}
}
})
Using yup
when()
function to dynamically define property type based on some other property value.
import { useCustomTypes } from "flexible-json-schema"
import { string } from "flexible-json-schema/core"
const STATES = {
US: ["TX", "NY", "CA", ...],
CA: ["NL", "PE", "QC", ...]
}
useCustomTypes({
// `country` can be one of: "US", "CA".
// Adding `null` is required due to `yup`'s internal quirks.
// https://github.com/jquense/yup/issues/104
"country": string().oneOf([null, "US", "CA"]),
// `state` depends on `country`.
"state": (fromYupType) => {
return string().when(["country"], ([country], stringType) => {
// If `country` is specified, then `state` must also be specified.
if (country) {
return fromYupType(stringType.oneOf(STATES[country]))
}
// If `country` is not specified, then `state` should also be not specified.
return fromYupType(stringType.oneOf([null]))
})
}
})
To define a "one of" value type ("enumeration"), use oneOf: [...]
property.
const validateProduct = schemaValidation({
"fruit": {
"oneOf": ["Apple", "Banana", "Coconut"],
"description": "A fruit"
}
})
validateProduct({
fruit: "Banana"
})
oneOf
only supports value types: no arrays, no objects, etc. All values must be of the same type: strings only, numbers only, etc.
To define an array of elements of some type, use arrayOf: <type>
property.
const validateProducts = schemaValidation({
"fruits": {
"arrayOf": "string",
"description": "A list of fruits"
}
})
validateProducts({
fruits: ["Apple", "Banana"]
})
By default, arrays aren't allowed to be empty. To allow any array to be empty, pass allowEmptyArrays: true
option.
const validate = schemaValidation(schema, { allowEmptyArrays: true })
The global allowEmptyArrays
setting can be overridden for individual arrays by specifying allowEmpty: true
or nonEmpty: true
property in the descriptor object:
const schema = {
array: {
arrayOf: 'string',
allowEmpty: true
}
}
To define a uniform-type map, use objectOf: <type>
property.
const validatePerformance = schemaValidation({
scores: {
// Could also be an `objectOf` nested objects, etc.
objectOf: "number",
description: "Player scores"
}
})
validatePerformance({
scores: {
player1: 1.25,
player2: 2.40,
player3: 4.10
}
})
To define an object (for example, a nested object), there're three different approaches:
- Reference another schema by name. See Schema Reference.
- Define the object's schema in a
schema
property. - Define the object's shape.
Referencing another schema by name.
const artistSchema = {
name: {
type: 'string',
description: 'Artist name'
},
discography: {
arrayOf: {
schema: 'album'
}
}
}
const schemas = {
album: {
title: {
type: 'string',
description: 'Album title'
},
year: {
type: 'number',
description: 'Album year'
}
}
}
const validateArtist = schemaValidation(artistSchema, { schemas })
Referencing another schema by name (via extends
keyword) while extending it with custom properties.
const artistSchema = {
name: {
type: 'string',
description: 'Artist name'
},
discography: {
arrayOf: {
extends: 'album',
schema: {
rating: {
type: 'number',
description: 'The album\'s rating among the other albums of the artist'
}
}
}
}
}
const schemas = {
album: {
title: {
type: 'string',
description: 'Album title'
},
year: {
type: 'number',
description: 'Album year'
}
}
}
const validateArtist = schemaValidation(artistSchema, { schemas })
Defining the object's schema in a schema
property.
const artistSchema = {
name: {
type: 'string',
description: 'Artist name'
},
discography: {
arrayOf: {
schema: {
title: {
type: 'string',
description: 'Album title'
},
year: {
type: 'number',
description: 'Album year'
}
}
}
}
}
const validateArtist = schemaValidation(artistSchema)
Define the object's shape.
const artistSchema = {
name: {
type: 'string',
description: 'Artist name'
},
bestAlbum: {
title: {
type: 'string',
description: 'Album title'
},
year: {
type: 'number',
description: 'Album year'
}
}
}
const validateArtist = schemaValidation(artistSchema)
Although supporting "any" type contradicts the concept of defining a schema by itself, it might be useful in cases when a developer wants to "start small" and outline a schema for some legacy undocumented data structure. So "any" type is considered an intermediate rather than a prominent solution.
To define a property of "any" type, use type: "any"
.
Matches any value, any array, any object, etc.
To define an array of type: "any"
items, use arrayOf: "any"
.
To define an object of "any" shape, use an empty object {}
as its schema.
To define an empty object, use an empty object {}
as its schema and empty: true
property.
Example
"Simple" variant:
{
nestedObject: {}
}
"Advanced" variant:
{
nestedObject: {
description: "...",
required: true/false,
schema: {}
}
}
To define a "one of type" property, add a oneOfType
entry containing an array of "type variations".
Each "type variation" should be a standard "property descriptor" object also having:
is
— a javascripttypeof
type.when
— (optional) conditions the property value has to meet in case ofis: "object"
oris: "object[]"
:- In case of
is: "object"
,when
will test the properties of the object. - In case of
is: "object[]"
,when
will test the properties of each object in the array — all of them must match.
- In case of
is
can be one of:
string
number
boolean
object
date
— ADate
instance. IfdateStrings: true
flag is passed, then anystring
.- An array of any of the above:
string[]
number[]
boolean[]
object[]
date[]
any[]
— An array of any elements.
when
condition must be an object describing the value object's properties:
propertyName: propertyValue
— The property value must be equal to a certain value.propertyName: { ...rules }
— The property must adhere to a set of "rules":$exists: true / false
— Whether or not the property should "exist", "exist" meaning "notundefined
ornull
".$notEqual: value
— The property value must not be equal tovalue
.$oneOf: [...]
— The property value must be one of ...$notOneOf: [...]
— The property value must not be one of ...$is: "..."
— The property value must be of type ... (see the list of possibleis
types)
An example of defining a oneOfType
property.
const schema = {
booleanOrStringOrArrayOrObject: {
description: "A boolean, or a string, or an array of strings, or an object with a formula.",
oneOfType: [
{
// "boolean" here is a javascript `typeof` type.
is: "boolean",
type: "boolean",
description: "Can be a boolean"
},
{
// "string" here is a javascript `typeof` type.
is: "string",
oneOf: ["x", "y", "z"],
description: "Can be one of: 'x', 'y', 'z'"
},
{
// "string" here is a javascript `typeof` type.
is: "string[]",
arrayOf: {
oneOf: ["x", "y", "z"]
},
description: "Can be an array of: 'x', 'y', 'z'"
},
{
// "object" here is a javascript `typeof` type.
is: "object",
when: {
formula: {
$exists: true
}
},
description: "Can be an object with a formula",
schema: {
formula: {
type: "string",
description: "Some formula"
}
}
}
]
}
}
Any property in a schema can reference another schema by name.
const schema = {
artist: {
description: 'Artist',
schema: 'artist'
},
discography: {
description: 'Artist\'s discography',
arrayOf: {
schema: 'album'
}
}
}
const schemas = {
artist: {
name: {
type: 'string',
description: 'Artist name'
}
},
album: {
title: {
type: 'string',
description: 'Album title'
},
year: {
type: 'number',
description: 'Album year'
}
}
}
const validateArtist = schemaValidation(schema, { schemas })
Schemas could also be used to "parse" JSON objects with stringified values in order to convert those values from strings to their appropriate "native" type: string, number, boolean, date, etc.
An example of a JSON object with stringified values:
{
"name": "John Smith",
"age": "35",
"married": "true",
"title": "Manager",
"occupation": ["salesman", "marketing"],
"children": [{
"name": "Jane Brown",
"sex": "female",
"age": "4"
}]
}
In that case, each and every "leaf" property of the data tree sctructure must be a string, unless being undefined
or null
.
The most-commonly-used scenarios for parsing JSON objects with stringified values are:
-
Parsing a CSV file.
- First, a CSV file should be parsed into a list rows of cells having a "flat" data structure.
- Example:
[["Column 1", "Column 2", ...], ["Value 1", "Value 2", ...], ...]
.
- Example:
- Then, those "flat" rows of data should be converted from arrays to JSON objects, usually with property nesting. The value of each "leaf" property should stay being a string.
- Example:
[{ "Column 1": "Value 1", "Column 2": "Value 2", ... }, ...]
.
- Example:
- After that, the list of JSON objects should be "parsed" using a schema in order for every "leaf" string property to be converted into an appropriate "native" type:
- string
- number
- boolean
- date
- etc.
- First, a CSV file should be parsed into a list rows of cells having a "flat" data structure.
-
Parsing URL query parameters.
- First, all query parameter values should be extracted from the URL into a
query
object.- Example:
{ "param1": "value1", "param2": "value2", ... }
.
- Example:
- Then, the
query
object should be "parsed" using a schema in order to convert every query parameter's string value into an appropriate "native" type:- string
- number
- boolean
- date
- etc.
- When some URL query parameter values are "complex" data structures such as arrays or objects:
- First, those values should be
JSON.stringify()
-ed before putting them in the URL. For example, if there's a query parameter calledfilters
being an object{...}
then the URL query part should look like:?filters=${JSON.stringify(filters)}
. Same goes for arrays. - Second, one should pass
structure: "flat"
option when "parsing" the extractedquery
object using a schema, which enablesJSON.parse()
-ing such "complex" data structures back from their "stringified" representations. See the example below.
- First, those values should be
- First, all query parameter values should be extracted from the URL into a
When parsing boolean
values, it supports multiple possible formats for convenience:
"true"
or"false"
"true"
→true
"false"
→false
"1"
or"0"
"1"
→true
"0"
→false
"✓"
or"✕"
"✓"
→true
"✕"
→false
Parse HTTP GET Request Query.
import schemaParser from "flexible-json-schema/parse"
const querySchema = {
id: {
type: "number",
description: "A numeric ID"
},
active: {
type: "boolean",
description: "Whether the item is active"
},
status: {
oneOf: ["PENDING", "FINALIZED"],
description: "The item's status"
},
tags: {
arrayOf: "string",
description: "The item's tags"
},
scores: {
arrayOf: "number",
description: "The item's scores"
},
createdAt: {
type: "date",
description: "Creation date"
},
owner: {
id: {
type: "number",
description: "A numeric ID of the item's owner"
}
}
}
// const query = request.query
// const params = request.pathParameters
const query = {
"id": "123",
"active": "true",
"status": "PENDING",
"tags": "[\"home\",\"accessory\"]",
"scores": "[1.5,2.0]",
"createdAt": "2000-01-01T00:00:00.000Z",
"owner": "{\"id\":456}"
}
const parse = schemaParser(schema, {
structure: "flat"
})
parse(query) === {
"id": 123,
"active": true,
"status": "PENDING",
"tags": ["home", "accessory"],
"scores": [1.5, 2],
"createdAt": new Date("2000-01-01T00:00:00.000Z"),
"owner": {
"id": 456
}
}
Parse CSV file data.
import schemaParser from 'flexible-json-schema/parse'
const schema = {
id: {
type: "number",
description: "A numeric ID"
},
name: {
type: "string",
description: "A person's name"
},
dateOfBirth: {
type: "date",
description: "Date of birth"
},
address: {
street: {
type: "string",
description: "Street name"
},
building: {
type: "number",
description: "Building number"
}
}
}
const csvFileContents = `
id,name,dateOfBirth,street,apt
1,John Smith,2000-01-01,Main Ave.,10
`.trim()
// This is a "naive" variant of parsing *.csv file contents.
// A proper implementation should check for escaped commas in cell values.
const [
id,
name,
dateOfBirth,
street,
building
] = csvFileContents.split('\n')[1].split(',')
const person = {
id,
name,
dateOfBirth,
address: {
street,
building
}
}
person === {
id: "1",
name: "John Smith",
dateOfBirth: "2000-01-01",
address: {
street: "Main Ave.",
building: "10"
}
}
const parse = schemaParser(schema, {
inPlace: true,
dateFormat: "yyyy-mm-dd"
})
parse(person) === {
id: 1,
name: "John Smith",
dateOfBirth: new Date("2000-01-01T00:00:00.000Z"),
address: {
street: "Main Ave.",
building: 10
}
}
Parse Custom Types.
Developers can define "custom" types via useCustomTypes()
but those custom type definitions are only used for validation. The parsing function ignores any of those custom type definitions and, by default, leaves those property values as is. If any of those custom types should be parsed in some special way, pass a parseProperty()
function option when creating a schema parsing function.
import schemaParser from 'flexible-json-schema/parse'
const parse = schemaParser(schema, {
parseProperty({ path, value, type, parsePropertyValue, createParseError }) {
// Parse `type: "percent"` properties as numbers.
if (type === "percent") {
return parsePropertyValue({ path, value, type: 'number' })
}
// Parse any other custom-type properties,
// like `type: "phone"` ones, as strings.
return value
}
})
Validating using a schema might throw a SchemaValidationError
.
import { SchemaValidationError } from 'flexible-json-schema'
error.message // Detailed error message.
error.errors // A list of original error messages. To include all errors, pass `returnAllErrors: true` option.
error.type // Error type. One of: "required", "unknown", "ambiguous", "unsupported", `undefined`.
error.path // Example: "somePropertyName.childObjectProperty". Is `undefined` for root path.
error.value // The value of the property that failed validation.
A developer can pass a custom error creation function:
import schemaValidation from 'flexible-json-schema'
const validate = schemaValidation(schema, {
// See TypeScript typings for the list of possible values for the `type` property.
createValidationError({ message, errors, type, path, value }) {
return new Error(message)
}
})
Parsing using a schema might throw a SchemaParseError
.
import { SchemaParseError } from 'flexible-json-schema'
error.message // Detailed error message.
error.errors // A list of original error messages. Only a single error message in the list.
error.type // Error type. One of: "ambiguous", "unsupported", "invalid", "unknown", `undefined`.
error.path // Example: "somePropertyName.childObjectProperty". Is `undefined` for root path.
error.value // The value of the property that failed to be parsed.
A developer can pass a custom error creation function:
import schemaParser from 'flexible-json-schema/parse'
const parse = schemaParser(schema, {
// See TypeScript typings for the list of possible values for the `type` property.
createParseError({ message, errors, type, path, value }) {
return new Error(message)
}
})
By default, schema validation aborts as soon as it encounters the first validation error and then throws an error for that property. If returning all errors is required, pass returnAllErrors: true
property: in that case, a SchemaValidationError
error will have an errors
property (a list of all error messages).
const validate = schemaValidation(schema, {
returnAllErrors: true
})
Conditional Required could be used when a property should be required
depending on some other property.
Example
Suppose there's a "Reason" select for an action performed on a website. The user could select a pre-defined reason from a list of common ones, or they could select "Other" and then describe the reason in a separate text input field (required).
The form data is then sent to the server as a combination of two fields:
reason
— A pre-defined reason code, or"OTHER"
.reasonNotes
— The"OTHER"
reason description (required).
So when reason
is "OTHER"
, reasonNotes
are required. Such "conditional require" could be described as:
{
reason: {
oneOf: [...],
description: "Reason"
},
reasonNotes: {
type: "text",
description: "Reason description",
required: {
when: {
reason: "OTHER"
}
}
}
}
In the schema above, when
describes the conditions that the object's properties must meet.
Besides basic property value equality testing, there also exist different "operators". For example, to test based on some property being present or not:
{
one: {
oneOf: [...],
description: "One"
},
two: {
type: "text",
description: "Two",
required: {
when: {
one: {
$exists: true / false
}
}
}
}
}
Available when
condition "rules":
$exists: true / false
— Whether or not the property should "exist", "exist" meaning "notundefined
ornull
".$notEqual: value
— The property value must not be equal tovalue
.$oneOf: [...]
— The property value must be one of ...$notOneOf: [...]
— The property value must not be one of ...$is: "..."
— The property value must be of type ... (see the list of possibleis
types)
A when
condition could be a combination of conditions imposed on several properties, which would be treated as a logical AND
:
{
one: {
oneOf: [...],
description: "One"
},
two: {
oneOf: [...],
description: "Two"
},
three: {
type: "text",
description: "Three",
required: {
when: {
one: {
$exists: true
},
two: "two"
}
}
}
}
A when
condition could be a combination of OR
conditions:
{
one: {
oneOf: [...],
description: "One"
},
two: {
oneOf: [...],
description: "Two"
},
three: {
type: "text",
description: "Three",
required: {
when: {
$or: [
one: {
$exists: true
},
two: {
$exists: true
}
]
}
}
}
}
To run tests:
npm test