What is the purpose of thinc.Model.id?

[BramVanroy]

Over at datasets we found that loaded spaCy Language objects are not recursively the same in terms of its underlying properties. This leads to an issue in datasets processing where the fingerprint of every script run is different and the processing cannot be cached. Quentin found that nlp1 and nlp2 are not the same. (Not just in memory space, but also in their underlying property values.)

nlp1 = spacy.load("en_core_web_sm")
nlp2 = spacy.load("en_core_web_sm")

Specifically, the id of underlying models differ

nlp1.tagger.model.tok2vec.embed.id, nlp2.tagger.model.tok2vec.embed.id
# (1721, 2243)

I found that indeed this id property for every model that is incremented for every thinc model.

https://github.com/explosion/thinc/blob/68691e303ae68cae4bc803299016f1fc064328bf/thinc/model.py#L96-L98

A lot of the functionality seems to revolve around this unique ID (specifically when getting model parameters), and I am wondering how exactly that works or what its purpose is. ParamServer seems curcial here, but I cannot quite get my head around how it interacts (and requires) a unique id. In other words, why do you need a unique model ID to access parameters instead of a using self?

[honnibal]

There's a couple of reasons, but one is that it lets us run the optimizer without relying on object identity on the arrays. So, let's say you have nlp1 and nlp2 there. The optimizer will need some way of distinguishing that those are different parameters, including across processes. We do that by giving the models a distinct ID, and then parameters are addressed as (model_id, param_id)

[BramVanroy]

I figured it was something about keeping track of the models and its parameters. What is the advantage (or difference) with something like torch where you'd create the optimizer directly on the model's parameters? (If it's too complex to explain, I understand and you can of course leave it at that. No worries.)