Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow extended filtering on relationships #524

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 20 additions & 8 deletions optimade.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2000,30 +2000,42 @@ A filter on a nested property name consisting of two identifiers :filter-fragmen
- :filter-fragment:`identifier1` references a list of dictionaries that contain as an identifier :filter-fragment:`identifier2` and the filter matches for a flat list containing only the contents of :filter-fragment:`identifier2` for every dictionary in the list.
E.g., if :filter-fragment:`identifier1` is the list :filter-fragment:`[{"identifier2":42, "identifier3":36}, {"identifier2":96, "identifier3":66}]`, then :filter-fragment:`identifier1.identifier2` is understood in the filter as the list :filter-fragment:`[42, 96]`.

- :filter-fragment:`identifier1` references an entry type of a group of related entries each containing a property named after an identifier :filter-fragment:`identifier2` and the filter matches for a flat list containing only the values of :filter-fragment:`identifier2` properties for every related entry.
Support for such queries is OPTIONAL.
E.g., :filter-fragment:`references.doi` is understood in the filter as the list containing values of :property:`doi` for all related entries of type :entry:`references`.
This is explained in more detail in section `Filtering on relationships`_.
Comment on lines +2003 to +2006
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- :filter-fragment:`identifier1` references an entry type of a group of related entries each containing a property named after an identifier :filter-fragment:`identifier2` and the filter matches for a flat list containing only the values of :filter-fragment:`identifier2` properties for every related entry.
Support for such queries is OPTIONAL.
E.g., :filter-fragment:`references.doi` is understood in the filter as the list containing values of :property:`doi` for all related entries of type :entry:`references`.
This is explained in more detail in section `Filtering on relationships`_.
- :filter-fragment:`identifier1` references an entry type of a group of related entries each containing a property named after an identifier :filter-fragment:`identifier2` and the filter matches for a flat list containing only the values of :filter-fragment:`identifier2` properties for every related entry.
In this case, :filter-fragment:`identifier2` is restricted to either :property:`id`, when matching directly on the ID of the related entry, or :property:`target`, which can be further nested to enable queries on the related entry's property.
Support for such queries is OPTIONAL.
E.g., :filter-fragment:`references.target.doi` is understood in the filter as the list containing values of :property:`doi` for all related entries of type :entry:`references`.
This is explained in more detail in section `Filtering on relationships`_.


The API implementation MAY allow this notation to generalize to arbitrary depth.
A nested property name that combines more than one list MUST, if accepted, be interpreted as a completely flattened list.

Filtering on relationships
~~~~~~~~~~~~~~~~~~~~~~~~~~

As described in the section `Relationships`_, it is possible for the API implementation to describe relationships between entries of the same, or different, entry types.
The API implementation MAY support queries on relationships with an entry type :filter-fragment:`<entry type>` by using special nested property names:
The API implementation MAY support queries on relationships with an entry type :filter-fragment:`<entry type>`:

- :filter-fragment:`<entry type>.id` references a list of IDs of relationships with entries of the type :filter-fragment:`<entry type>`.
- :filter-fragment:`<entry type>.description` references a correlated list of the human-readable descriptions of these relationships.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my main review comment, but I think we have to keep this still

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this has to be discussed. There are two options: retain full backwards compatibility by either encoding the special meaning of description or forbidding properties bearing such name, or scrap the special treatment of description altogether. This PR does the latter.

- :filter-fragment:`<entry type>.<property>` references a list of property :property:`<property>` values for related entries of type :filter-fragment:`<entry type>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- :filter-fragment:`<entry type>.<property>` references a list of property :property:`<property>` values for related entries of type :filter-fragment:`<entry type>`.
- :filter-fragment:`<entry type>.target.<property>` references a list of property :property:`<property>` values for related entries (i.e., the :filter-fragment:`target` entry of the relationship) of type :filter-fragment:`<entry type>`.


Hence, the filter language acts as, for every entry type, there is a property with that name which contains a list of dictionaries with two keys, :filter-fragment:`id` and :filter-fragment:`description`.
Hence, the filter language acts as, for every entry type, there is a property with that name which contains a list of dictionaries with entry properties plus :property:`id`.
For example: a client queries the :endpoint:`structures` endpoint with a filter that references :filter-fragment:`calculations.id`.
For a specific structures entry, the nested property behaves as the list :filter-fragment:`["calc-id-43", "calc-id-96"]` and would then, e.g., match the filter :filter:`calculations.id HAS "calc-id-96"`.
This means that the structures entry has a relationship with the calculations entry of that ID.

**Note**: formulating queries on relationships with entries that have specific property values is a multi-step process.
For example, to find all structures with bibliographic references where one of the authors has the last name "Schmidt" is performed by the following two steps:
Support for queries on fields of arbitrary depth is OPTIONAL.
For example, search for all structures related to a publication having DOI 10.1234/1234 could be performed with the following query:

:query-url:`https://example.com/optimade/v1/structures?filter=references.doi=%2210.1234/1234%22`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:query-url:`https://example.com/optimade/v1/structures?filter=references.doi=%2210.1234/1234%22`
:query-url:`https://example.com/optimade/v1/structures?filter=references.target.doi=%2210.1234/1234%22`


Search for all literature references for structures with tantalum:

:query-url:`https://example.com/optimade/v1/references?filter=structures.elements+HAS+%22Ta%22`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:query-url:`https://example.com/optimade/v1/references?filter=structures.elements+HAS+%22Ta%22`
:query-url:`https://example.com/optimade/v1/references?filter=structures.target.elements+HAS+%22Ta%22`


Search for all structures of anonymous formula A2B from year 2024:

- Query the :endpoint:`references` endpoint with a filter :filter:`authors.lastname HAS "Schmidt"` and store the :filter-fragment:`id` values of the returned entries.
- Query the :endpoint:`structures` endpoint with a filter :filter-fragment:`references.id HAS ANY <list-of-IDs>`, where :filter-fragment:`<list-of-IDs>` are the IDs retrieved from the first query separated by commas.
:query-url:`https://example.com/optimade/v1/structures?filter=references.year=2023+AND+chemical_formula_anonymous=%22A2B%22`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:query-url:`https://example.com/optimade/v1/structures?filter=references.year=2023+AND+chemical_formula_anonymous=%22A2B%22`
:query-url:`https://example.com/optimade/v1/structures?filter=references.target.year=2023+AND+chemical_formula_anonymous=%22A2B%22`


(Note: the type of query discussed here corresponds to a "join"-type operation in a relational data model.)
Note: the type of query discussed here corresponds to a "join"-type operation in a relational data model.

Filtering on Properties with an unknown value
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
Loading