Try to add context embeddings

[PonteIneptique]

Hey Enrique,
I want to dig in Contextual embeddings/Language models and in particular Bert. I want to see if the these features would help the lemmatizer for languages close to French (Camembert).

Did you plan to do it ? If so, please tell me, I would not want to steal that from you :)

[emanjavacas]

Hey! This went missing... sorry! In the current setup, if you use the LM loss, you are effectively leveraging contextual embeddings (even before they were cool). Of course you are missing the effect of pre-training on massive datasets. Feel free to explore it. I could also help if you want. For lemmatization, however, I wont expect miracles, since the possibilities in which contextual information helps there are limited (mostly disambiguation).

[PonteIneptique]

Well, I actually did it :) I am figuring some things out, and discovered few bugs with CRF (probably breaking changes that went in at some point...)