Train SpaCy custom pipelines in AWS Trainium

[Shanoa00]

Hello SpaCy community! First of all, I want to express my gratitude for all the incredible work you've done.

Currently, I'm utilizing an AWS instance for training multiple models. Recently, I came across a new PyTorch-Neuron-based instance called AWS Trainium (you can find more information here: https://aws.amazon.com/machine-learning/trainium/). I'm curious to know whether it's feasible to use SpaCy to train custom pipelines on this specific instance or if there are any plans to implement a SpaCy version that is compatible with it. Thanks for your insights.

[rmitsch]

Hi @Shanoa00, we don't have any plans to add support specifically for Trainium. You should be able to train spaCy models on Trainium instances though (if you run into problems, let us know).

[Shanoa00]

Thanks for your response @rmitsch. I am facing some errors regarding the use of GPU. Since AWS Trainium uses PyTorch-Neuron you don't require CUDA, CuPy libraries. Based on this FAQ (https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/faq/training/neuron-training.html#can-i-use-cuda-libraries-with-aws-trainium) you need to strip the code out, and from that point the Neuron compiler will take the model as is and optimize it at the hardware level.
The specific error is:
`python -m spacy project run train

=================================== train ===================================
Running command: /usr/local/bin/python -m spacy train configs/config.cfg --output training/UD_Spanish-GSD/transformer --gpu-id 0 --paths.train corpus/UD_Spanish-GSD/train.spacy --paths.dev corpus/UD_Spanish-GSD/dev.spacy --nlp.lang=es
ℹ Saving to output directory: training/UD_Spanish-GSD/transformer
ℹ Using GPU: 0
Traceback (most recent call last):
File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/site-packages/spacy/main.py", line 4, in
setup_cli()
File "/usr/local/lib/python3.10/site-packages/spacy/cli/_util.py", line 92, in setup_cli
command(prog_name=COMMAND)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/typer/core.py", line 778, in main
return _main(
File "/usr/local/lib/python3.10/site-packages/typer/core.py", line 216, in _main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 683, in wrapper
return callback(**use_params) # type: ignore
File "/usr/local/lib/python3.10/site-packages/spacy/cli/train.py", line 54, in train_cli
train(config_path, output_path, use_gpu=use_gpu, overrides=overrides)
File "/usr/local/lib/python3.10/site-packages/spacy/cli/train.py", line 76, in train
setup_gpu(use_gpu)
File "/usr/local/lib/python3.10/site-packages/spacy/cli/_util.py", line 599, in setup_gpu
require_gpu(use_gpu)
File "/usr/local/lib/python3.10/site-packages/thinc/util.py", line 230, in require_gpu
raise ValueError("Cannot use GPU, CuPy is not installed")
ValueError: Cannot use GPU, CuPy is not installed`

The instance I am using has:
Ubuntu 20.04.6 LTS
Neuron driver version: 2.13.4.0
Python 3.10.12
spacy 3.7.2
spacy-alignments 0.9.1
spacy-curated-transformers 0.2.0
spacy-experimental 0.6.3
spacy-legacy 3.0.12
spacy-loggers 1.0.5
spacy-transformers 1.3.2
torch.version.cuda: 11.7
torch.version: 1.13.1+cu117
torch.backends.cudnn.version(): 8500
cupy-cuda11x==12.2.0

[rmitsch]

Thanks for bringing this up! We'll discuss AWS Trainium support internally. For now I can only recommend to use non-Trainium instances.