Trouble with labels.json Format for Dependency Parser Training in spaCy v3

[LilitKharatyan]

I'm trying to train a dependency parser using spaCy v3 and I've been facing issues with the setup of my labels.json file.
I've gathered all the unique labels from my training data and organized them into a list.
From my understanding, spaCy v3 expects the labels.json to have a structure: Dict[str, Dict[str, Dict[str, int]]].
I first tried to format my labels as:

{
    "parser": {
        "labels": {
            "nsubj": 0,
            "root": 1,
            "dobj": 2,
            ...
        }
    }
}

However, when referencing this in my config.cfg and starting training, I encounter an error related to the initialization of these labels.
I have also tried to format them in a simple list which works perfectly for the tagger for example, but again I receive an error.

Using the spacy debug config config.cfg command, everything appears to be okay, which suggests that the issue might lie in the way I've formatted my labels.

Questions:

Could anyone clarify the exact expected format of the labels.json for dependency parser training in spaCy v3?
Are there additional considerations I need to account for.
Any insights would be greatly appreciated!

[adrianeboyd]

The labels files aren't intended to be generated by hand, and each pipeline component has its own internal format. You can generate the labels files by using spacy init labels. If you have a very large or streamed/infinite corpus, you can generate the labels based on a representative subset of your training data instead of the whole training corpus.

But be aware that in most cases, the labels file isn't required. spacy train can determine the labels from the provided training corpus in the initialization step. If you have a large training corpus and you're repeatedly training on the same data, the main benefit is that you save some time at the beginning of spacy train by providing the labels.

[LilitKharatyan]

Thank you very much for your response. Additionally, I have a hard time finding a recipe, a code (assistance) any relevant information for training a dependency parser for a new language. There is a lot of info for the exciting one but not for training one from scratch. Do you think there is a relevant piece of information? Thank you.

[adrianeboyd]

The UD corpora are a good place to get started with training dependency parsers, and there's an example project called pipelines/tagger_parser_ud: https://github.com/explosion/projects/tree/v3/pipelines/tagger_parser_ud

To get started, clone it with spacy project clone pipelines/tagger_parser_ud and have a look at the projects documentation: https://spacy.io/usage/projects

[LilitKharatyan]

Thanks, I have tried this too. While I succeed in training only the tagger, I am unable to train the parser (with the same dataset, btw). After the initialization is done it takes ages for it to start the actual training process (actually it never started). My dataset is around 5000 sentences and tagger finishes training in 7-8 min usually. Where can be the issue?

`=========================== Initializing pipeline ===========================
[2023-10-16 14:43:10,266] [INFO] Set up nlp object from config

[2023-10-16 14:43:10,290] [INFO] Pipeline: ['tok2vec', 'parser']

[2023-10-16 14:43:10,297] [INFO] Created vocabulary

[2023-10-16 14:43:10,301] [INFO] Finished initializing nlp object
`
It never goes beyond this stage.

[LilitKharatyan]

Thank you very much for the response.
I actually initialize the config file with the xcl code not the 'hy' (modern Armenian) the rest is the same I guess. As for the language code, it is just one file with the basic settings, no example, or stopword file is included. Do you think the problem can be in it?

[adrianeboyd]

First see if you can run the same conversion + training as shown above, and then change only the language to see if that makes a difference?

[LilitKharatyan]

Thank you very much. I managed to start the training (although the results are not satisfactory even though I used twice as much data as we have released). The only reasonable explanation I find for the error is the way I converted my dataset to the spacy binary format (i used to do it differently and maybe the error arose from it). Thank you for the full support.
Another question: currently xcl (classical Armenian) is not a language supported by spacy. How can we request an addition of it. What should we provide for that?
Thank you

[adrianeboyd]

To add a new language: #13135 (comment)

[LilitKharatyan]

Thanks a lot. I am sorry but I have another question which was never answered in the discussions:
I am training (almost) a full pipeline of models: tokenizer, trainable_lemmatizer, tagger, morphologizer, parser and I see that accuracy differs based on which pipelines are trained together, e.g. I receive the best results when morphologizer and tagger are trained together, while when I add for example parser to this pipeline of training, the results/ accuracy downgrade.
Is it a common behavior, and are there best practices on this issue, or am I doing smth wrong? Thanks