RuntimeError: The size of tensor a (1218) must match the size of tensor b (512) at non-singleton dimension 1

[Einsteinder]

I'm trying to train a transformer model using this command:

python -m spacy train  transformer-config.cfg --output ./out --paths.train ./train_2022-05-02_10-47-26.spacy --paths.dev ./validate_2022-05-02_10-47-26.spacy > example.txt

transformer-config.cfg:

[paths]
train = ""
dev = ""
vectors = null
init_tok2vec = ""

[system]
gpu_allocator = "pytorch"
seed = 2021

[nlp]
lang = "en"
pipeline = ["transformer","ner", "beam_ner"]
batch_size = 128
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.beam_ner]
factory = "beam_ner"
beam_density = 0.01
beam_update_prob = 0.5
beam_width = 32
moves = null
update_with_oracle_cut_size = 100

[components.beam_ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = false
nO = null

[components.beam_ner.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0
pooling = {"@layers":"reduce_mean.v1"}
upstream = "*"

[components.ner]
factory = "ner"
moves = null
update_with_oracle_cut_size = 100

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = false
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0
pooling = {"@layers":"reduce_mean.v1"}
upstream = "*"

[components.transformer]
factory = "transformer"
max_batch_items = 1024
set_extra_annotations = {"@annotation_setters":"spacy-transformers.null_annotation_setter.v1"}

[components.transformer.model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "google/mobilebert-uncased"

[components.transformer.model.get_spans]
@span_getters = "spacy-transformers.strided_spans.v1"
window = 128
stride = 96

[components.transformer.model.tokenizer_config]
use_fast = true

[corpora]

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0
gold_preproc = false
limit = 0
augmenter = null

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 0
gold_preproc = false
limit = 0
augmenter = null

[training]
accumulate_gradient = 3
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"
seed = ${system.seed}
gpu_allocator = ${system.gpu_allocator}
dropout = 0.1
patience = 0
max_epochs = 200
max_steps = 50000
eval_frequency = 200
frozen_components = []
before_to_disk = null

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
size = 100
tolerance = 0.2
discard_oversize = false
get_length = null

[training.logger]
@loggers = "spacy.ConsoleLogger.v1"
progress_bar = false

[training.optimizer]
@optimizers = "Adam.v1"
beta1 = 0.9
beta2 = 0.999
L2_is_weight_decay = true
L2 = 0.01
grad_clip = 1.0
use_averages = false
eps = 0.00000001

[training.optimizer.learn_rate]
@schedules = "warmup_linear.v1"
warmup_steps = 250
total_steps = 50000
initial_rate = 0.00005

[training.score_weights]
ents_per_type = null
ents_f = 1.0
ents_p = 0.0
ents_r = 0.0

[pretraining]

[initialize]
vectors = null
init_tok2vec = ${paths.init_tok2vec}
vocab_data = null
lookups = null
before_init = null
after_init = null

[initialize.components]

[initialize.tokenizer]

training sample attached
spacy_file.zip

but it gives error:

  File "/pythonenv/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 872, in forward
    embedding_output = self.embeddings(
  File "/pythonenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/pythonenv/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 218, in forward
    embeddings = inputs_embeds + position_embeddings + token_type_embeddings
RuntimeError: The size of tensor a (1218) must match the size of tensor b (512) at non-singleton dimension 1

It seems related to the length of text, but this is not a crazy lengthy one, anyone can help on this? Thanks in advance

[polm]

There are a couple of weird things going on in your config but I think this is mainly an issue with the Transformer model you're using - if I change that to bert-base-uncased then training runs. Is there a particular reason you're using that model?

Some other notes:

1. I had to edit your config to remove a stray line (edited your post)
2. Why are you training NER and Beam NER at the same time?
3. What version of spaCy are you using? The default version of TransformerModel hasn't been v1 for a while.
4. More generally, how did you generate your config?

[kormilitzin]

If this is still of interest, there is a simple hacky fix to this problem. Add "model_max_length":512 (or any other max length depending on your model) to "tokenizer_config.json" file of a HF custom model.

[blizaga]

Where exactly is the "tokenizer_config.json" file located?

[danieldk]

I think they are referring to the tokenizer_config.json file that come with many (but not all) models on Hugging Face hub. But you can also override the length to the spaCy pipeline configuration. E.g. if the tokenizer configuration in your pipeline looks like this:

[components.transformer.model.tokenizer_config]
use_fast = true

you can add the maximum length:

[components.transformer.model.tokenizer_config]
use_fast = true
model_max_length = 512

[blizaga]

Awesome, Thank you for the information, now my textcat model can be trained.

[kormilitzin]

Thank you @danieldk. Apologise for not being more clear, your right. I also haven't realised you can directly access HF's configs through the spaCy's one, this is by far more convenient and straightforward.