Support multiple versions of target dandischema

[yarikoptic]

Inspired by

• Schema mismatch between server and client #1463

by @kabilar , and ongoing work on "vendorization" of DANDI archive codebase to support multiple instances of DANDI.

ATM both dandi-cli and dandi-archive re-use dandischema Python library for manipulating and validating metadata records. As a result, any upgrade of dandischema in server requires upgrade of dandischema in dandi-cli installation which also might require upgrade of dandi-cli itself.

With multiple archive instances in play (already DANDI and LINC and EMBER is coming), synchronizing all of them to align with most recent dandischema would be getting close to infeasible. So I think we should distill a solution which would allow dandi-cli to manipulate different/needed versions of dandischema.

Possible roadmap to achieve that

• migrate pydantic model to linkml (effort ongoing by @candleindark ) + potentially additional "instance specific" restrictions (TODO: link issues in dandi-schema)
• export linkml models into https://github.com/dandi/schema/tree/master/releases
• define dandischema_schema_version -- as long as it is "compatible" in terms of its formulation with dandi-cli. In a similar situation in BIDS we have BIDS_VERSION and its SCHEMA_VERSION, see https://github.com/bids-standard/bids-specification/tree/master/src/schema#version-of-the-schema
• in dandi-cli implement loading of models from linkml models for the version desired by the server:
  • as long as that linkml model is described in compatible version of schema (based on minimal dandischema_schema_version supported by dandi-cli) , we should be good
  • potentially extend the model with instance specific models where needed for validation (most likely the base model would be sufficient to describe)
  • not yet sure what we should do for currently used pydantic models in Python interfaces, might need to populate classes for different versions. So issue similarish to what we discussed with @rly and other nwb developers about nwb versions.

Also attn @jwodder @satra @bendichter @dandi/dandiarchive to potentially also think about enhancement to above plan or just to suggest alternative approaches.

[satra]

the cli has to be compatible with not just schemas but also api processes (e.g. upload flows or parameters) across instances. hence there is a more critical notion of compatibility of cli and server that includes both api version and schema version.