You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@RichJackson reported warning when we load the model:
Some weights of the model checkpoint at .../kazu/kazu_model_pack_public-v0.0.25/tinybern were not used when initializing BertForTokenClassification: ['fit_dense.weight', 'fit_dense.bias']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
I think, this layer (fit_dense) comes from legacy code and will not affect performance or functionality.
original comment from @wonjininfo
@RichJackson reported warning when we load the model:
I think, this layer (
fit_dense
) comes from legacy code and will not affect performance or functionality.Related code:
https://github.com/AstraZeneca/KAZU/blob/main/kazu/modelling/distillation/tiny_transformers.py#L56-L57
Todo: I will first double-check it and will delete fit_dense layer!
The text was updated successfully, but these errors were encountered: