New API for model classification

[jazzhaiku]

Summary

The goal of this PR is to make it easier to add an new config type.
This scope of this PR is to integrate the API and does not include adding new configs (outside tests) or porting existing ones.

One of the glaring issues of the existing legacy probe is that the logic for each type is spread across multiple classes and intertwined with the other configs. This means that adding a new config type (or modifying an existing one) is complex and error prone.

This PR attempts to remedy this by providing a new API for adding configs that:

• Is backwards compatible with the existing probe.
• Encapsulates fields and logic in a single class, keeping things self-contained and easy to modify safely.

Below is a minimal toy example illustrating the proposed new structure:

class MinimalConfigExample(ModelConfigBase):
    type: ModelType = ModelType.Main
    format: ModelFormat = ModelFormat.Checkpoint
    fun_quote: str

    @classmethod
    def matches(cls, mod: ModelOnDisk) -> bool:
        return mod.path.suffix == ".json"

    @classmethod
    def parse(cls, mod: ModelOnDisk) -> dict[str, Any]:
        with open(mod.path, "r") as f:
            contents = json.load(f)

        return {
            "fun_quote": contents["quote"],
            "base": BaseModelType.Any,
        }

To create a new config type, one needs to inherit from ModelConfigBase and implement its interface:

1. Define fields 'type' and 'format' as class attributes.
2. Override methods 'matches' and 'parse':

• The method matches performs a quick check to determine if the config matches the model.
  This doesn't need to be a perfect test - the aim is to eliminate unlikely matches quickly before parsing.
• The method parse returns a dictionary with any additional fields needed to construct the model.

The code falls back to the legacy model probe for existing models using the old API.
This allows us to incrementally port the configs one by one.

Note: some unrelated linting changes included to make ruff happy.

Related Issues / Discussions

QA Instructions

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[RyanJDick]

I just looked into why the re-formatting was needed. It seems like ruff's format rules changed somewhere between 0.6.2 (the version I was running locally) and 0.9.9 (the current version). I didn't bother to narrow it down further than that.

We should pin the ruff version: "ruff~=0.9.9", "ruff-lsp~=0.0.62". Do you mind splitting out the reformatting into a separate PR to keep the diff on this one clean?