Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More language model links? #162

Open
fcore117 opened this issue Jan 22, 2024 · 2 comments
Open

More language model links? #162

fcore117 opened this issue Jan 22, 2024 · 2 comments

Comments

@fcore117
Copy link

Can someone point me direct download of more languages that can be added to translateLocally as tar.gz, it is pain to find easily. Thanks.

@marco-trovato
Copy link

marco-trovato commented Aug 20, 2024

FYI, I tried those and didn't manage to make them work:
https://github.com/mozilla/firefox-translations-models/tree/main/models/prod

Any help to understand how to use them would be really appreciated!

@marco-trovato
Copy link

Ok i managed to make it work.
Just download any model, i.e. from here: https://github.com/mkiol/dsnote/blob/main/config/models.json
By the way the models are 100% compatible with the one used by:

etc.
So you can download them from any of those programs and then import them into TranslateLocally!

You should get 3 files, as alternative you could even download them manually getting the URLs from models.json.

Example for English-Italian (100% tested and working):

  1. "https://storage.googleapis.com/bergamot-models-sandbox/0.3.4/enit/model.enit.intgemm.alphas.bin"
  2. "https://storage.googleapis.com/bergamot-models-sandbox/0.3.4/enit/lex.50.50.enit.s2t.bin"
  3. "https://storage.googleapis.com/bergamot-models-sandbox/0.3.4/enit/vocab.enit.spm"

Now just create 2 files:

  • model_info.json
{
  "modelName": "English-Italian tiny",
  "shortName": "en-it-tiny",
  "type":      "tiny",
  "src":       "English",
  "trg":       "Italian",
  "srcTags":   {"en": "English"},
  "trgTag":    "it",
  "version":   2.0,
  "API":       1.0
}
  • config.intgemm8bitalpha.yml
relative-paths: true
models:
  - model.enit.intgemm.alphas.bin
vocabs:
  - vocab.enit.spm
  - vocab.enit.spm
shortlist:
    - lex.50.50.enit.s2t.bin
    - false
beam-size: 1
normalize: 1.0
word-penalty: 0
mini-batch: 32
maxi-batch: 1000
maxi-batch-sort: src
workspace: 2000
max-length-factor: 2.5
gemm-precision: int8shiftAlphaAll

Make a tar.gz of this folder, and then from TranslateLocally go to: Edit -> Translator Settings -> Languages tab -> Import model
That's all 🤓 it works!

Source: https://github.com/XapaJIaMnu/translateLocally#basic-model-import

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants