Define/maintain centralized registry of instances

[yarikoptic]

Inspired by

• Add support for LINC staging and production APIs #1519

I wonder if to avoid even the need to modify dandi-cli to add a new instance, we could/should define a centralized registry of instances, e.g. in this repo, and make dandi-cli fetch up-to-date version once in a while?

we could define a versioned schema for the records to verify, and if anything goes wrong -- use "hardcoded" in the code copy.

WDYT @aaronkanzer @kabilar @jwodder

In the longer run we might also allow to register additional instances locally, hence also filed

• Add configuration system #1521

[satra]

indeed an instance could register itself with some service (or json file to start with).

there are many local instances (docker, testing, etc.,.) that would need some exceptions or other approach to register/use

perhaps with this thought we can also consider whether an instance allows certain operations (e.g. publishing), which would disable some components of the server. ideally this should be configurable and configured on the server side in the running instance and fetched through the API.

[aaronkanzer]

@yarikoptic I like the idea you have here -- I think the way the code is, extensibility in the way you are suggesting would be perfectly fine and doable

The one mini-pain point I'll throw out there is all the versioning validation across dandi-cli right now -- albeit it is appropriate to ensure schema uniformity, we'd need to abstract different instances being able to define different schemas and versions a bit more -- happy to provide code references, but a quick grep (or the code commented out in my PR) shows some of the roadblocks.

There is also the overhead of maintaining validation against the etelemetry server -- it works but my brain hurts a bit keeping track of all these things, unless the espresso is strong enough 😄