Generate additionnal etymology-free dictionnaries

[BoboTiG]

Related to #1149.

@lasconic that's quite a big PR 😅

Overall:

• Jinja2 for HTML word templates
• Word HTML code now tested for all supported dict types
• The Word "type" is now properly typed to catch any issue elsewhere
• Most of interesting code changes are into convert.py.
• Sorry for Mypy in tests... In the other hand, it 's done and future changes should still be easy to do.

[BoboTiG]

Actually, let me know if Mypy for tests is too much. I will revert it then.

[lasconic]

Looks good. I'm cool with mypy for tests.

Do you prefer to push the dictionaries with etymologies instead of the other ones ? If not, could be better to have "with-etym" for the "old" ones.

[BoboTiG]

Do you prefer to push the dictionaries with etymologies instead of the other ones ? If not, could be better to have "with-etym" for the "old" ones.

I would say we do not touch original dictionaries: they should be the "full" version by default for everyone.

[sourcery-ai]

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 1.33%.

Quality metrics	Before	After	Change
Complexity	15.66 🙂	14.46 🙂	-1.20 👍
Method Length	79.46 🙂	77.69 🙂	-1.77 👍
Working memory	8.54 🙂	8.35 🙂	-0.19 👍
Quality	59.52% 🙂	60.85% 🙂	1.33% 👍

Other metrics	Before	After	Change
Lines	6633	6899	266

Changed files	Quality Before	Quality After	Quality Change
scripts/__main__.py	83.48% ⭐	83.33% ⭐	-0.15% 👎
scripts/de-abk.py	60.87% 🙂	60.87% 🙂	0.00%
scripts/de-langs.py	64.15% 🙂	64.06% 🙂	-0.09% 👎
scripts/el-langs.py	39.16% 😞	39.16% 😞	0.00%
scripts/en-form-of.py	51.96% 🙂	50.82% 🙂	-1.14% 👎
scripts/en-labels.py	39.79% 😞	39.79% 😞	0.00%
scripts/en-langs.py	58.96% 🙂	58.49% 🙂	-0.47% 👎
scripts/es-campos-semanticos.py	72.09% 🙂	72.01% 🙂	-0.08% 👎
scripts/fr-domain-templates.py	74.91% 🙂	74.83% 🙂	-0.08% 👎
scripts/fr-regions.py	75.51% ⭐	75.99% ⭐	0.48% 👍
scripts/pt-gramatica.py	44.13% 😞	43.68% 😞	-0.45% 👎
scripts/scripts_utils.py	59.61% 🙂	59.61% 🙂	0.00%
tests/conftest.py	60.52% 🙂	80.00% ⭐	19.48% 👍
tests/test_0_download.py	77.46% ⭐	77.56% ⭐	0.10% 👍
tests/test_1_parse.py	92.35% ⭐	92.39% ⭐	0.04% 👍
tests/test_2_render.py	89.84% ⭐	90.19% ⭐	0.35% 👍
tests/test_3_convert.py	75.74% ⭐	73.16% 🙂	-2.58% 👎
tests/test_4_check_word.py	87.10% ⭐	87.59% ⭐	0.49% 👍
tests/test_4_check_words.py	94.57% ⭐	94.57% ⭐	0.00%
tests/test_4_find_templates.py	97.05% ⭐	97.05% ⭐	0.00%
tests/test_4_get_word.py	99.25% ⭐	99.25% ⭐	0.00%
tests/test_4_upload.py	81.69% ⭐	81.69% ⭐	0.00%
tests/test_5_gen_dict.py	81.83% ⭐	84.14% ⭐	2.31% 👍
tests/test_ca.py	61.10% 🙂	61.10% 🙂	0.00%
tests/test_de.py	71.69% 🙂	71.69% 🙂	0.00%
tests/test_el.py	74.94% 🙂	74.94% 🙂	0.00%
tests/test_en.py	62.29% 🙂	62.29% 🙂	0.00%
tests/test_es.py	61.05% 🙂	61.05% 🙂	0.00%
tests/test_fr.py	53.08% 🙂	53.08% 🙂	0.00%
tests/test_it.py	75.47% ⭐	75.47% ⭐	0.00%
tests/test_no.py	81.41% ⭐	81.41% ⭐	0.00%
tests/test_pt.py	64.36% 🙂	64.36% 🙂	0.00%
tests/test_ru.py	69.58% 🙂	69.58% 🙂	0.00%
tests/test_sv.py	78.43% ⭐	78.43% ⭐	0.00%
tests/update-data.py	81.64% ⭐	81.35% ⭐	-0.29% 👎
wikidict/check_word.py	34.20% 😞	34.20% 😞	0.00%
wikidict/constants.py	85.92% ⭐	86.39% ⭐	0.47% 👍
wikidict/convert.py	57.27% 🙂	65.18% 🙂	7.91% 👍
wikidict/gen_dict.py	70.32% 🙂	68.79% 🙂	-1.53% 👎
wikidict/get_word.py	35.13% 😞	35.13% 😞	0.00%
wikidict/parse.py	73.05% 🙂	73.05% 🙂	0.00%
wikidict/render.py	37.58% 😞	37.59% 😞	0.01% 👍
wikidict/stubs.py	75.97% ⭐	89.75% ⭐	13.78% 👍
wikidict/user_functions.py	76.84% ⭐	76.84% ⭐	0.00%
wikidict/utils.py	67.18% 🙂	67.18% 🙂	0.00%
wikidict/lang/es/__init__.py	27.39% 😞	27.39% 😞	0.00%

Here are some functions in these files that still need a tune-up:

File	Function	Complexity	Length	Working Memory	Quality	Recommendation
wikidict/check_word.py	filter_html	99 ⛔	685 ⛔	18 ⛔	5.80% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
wikidict/lang/es/__init__.py	last_template_handler	37 ⛔	294 ⛔	25 ⛔	12.13% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
wikidict/render.py	parse_word	58 ⛔	377 ⛔	15 😞	12.17% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
wikidict/convert.py	KoboFormat.handle_word	65 ⛔	315 ⛔	15 😞	12.86% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
wikidict/render.py	find_etymology	44 ⛔	387 ⛔	15 😞	14.47% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

• ⭐ excellent
• 🙂 good
• 😞 poor
• ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

---

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

[BoboTiG]

OK, the CI works perfectly with new files to upload (cf https://github.com/BoboTiG/ebook-reader-dict/releases/).

@lasconic do you want to change something about default dictionaries, or is it OK to merge?

[lasconic]

OK