EPIC: errors in markdown need fixing before merging a change (CI for documentation)

[mi-hol]

Validate Markdown Files With MarkdownLint according to this blog post revealed over 7K errors.
To avoid follow-on errors in other tools I'd suggest to fix them.

Luckily most errors can be fixed automatically.

As a follow-up we should activate this check on each commit.

PS git\rusefi_documentation> markdownlint-cli2 "**/*.md" "#node_modules" 2>.\wiki-tools\mdl-20221224.log
markdownlint-cli2 v0.5.1 (markdownlint v0.26.2)
Finding: **/*.md !node_modules
Linting: 425 file(s)
Summary: 7557 error(s)

mdl-20221224.log

[mi-hol]

after PR #334
markdownlint-cli2-config v0.5.1 (markdownlint v0.26.2)
Finding: **/*.md !node_modules
Linting: 424 file(s)
Summary: 4072 error(s)

[chuckwagoncomputing]

I've been working on this today. I've completed files starting with A-H.

[mi-hol]

@rusefillc sorry this is NOT closed
Again a case for #363

[chuckwagoncomputing]

You can't just re-open an issue without explaining why.

[mi-hol]

markdownlint-cli2 v0.6.0 (markdownlint v0.27.0)
Finding: **/*.md !node_modules
Linting: 426 file(s)
Summary: 3510 error(s)

[mi-hol]

@chuckwagoncomputing
I must say that I find this a rather crappy approach!

[mi-hol]

to avoid misunderstanding I wanted to clarify:
I'm fine to provide a reason, the "crappy approach" is related to "back and forth" as explained further in #363
ANY issue should be fixed completely, ofter multiple iterations are required to achieve that, because of different view points contributors have!

[chuckwagoncomputing]

Please state your view as to why this issue should not be closed.

[mi-hol]

"Documentation as Code" is a concept to apply CI to documentation in order to avoid common errors.
As a "kind of compiler" for md files several tools need to validate the content of a changed file and reject it for merging in case of detected errors & warning.

example:

[mi-hol]

Current implementation validates all markdown files but without any consequence in case of detected error

[chuckwagoncomputing]

Fixed. Details in #268

[mi-hol]

@chuckwagoncomputing does the current implementation prevent references with .md extension?
To my current understand this type of error was corrected manually and could re-occur any time unintentionally.

BTW: I Prefix this issue as "EPIC" (as suggested by Andrey) to remain in control of status

[chuckwagoncomputing]

No it doesn't. An idea I have for how to do that is to modify brokenlinks.sh to accept a flag to make it non-interactive, and just check links.

[mi-hol]

modify brokenlinks.sh to accept a flag to make it non-interactive, and just check links.

not sure if this is the best approach because in my tests here brokenlinks.sh was extremely slow compared to markdown-link-validator.
As I understand it currently markdown-link-validator is actually based on ESLint and consists of just 49 lines of TypeScript code,
Do you happen to know TypeScript?

Another approach could be "custom rule" for markdownlint.

[chuckwagoncomputing]

brokenlinks.sh takes less than a minute on my ~5 year old laptop. It could also be modified to only check the files that were changed in the PR.

If you really want to customize other tools to fit our needs, please feel free to do so. I only recommend not doing so because I don't feel that it's a good use of developer time.

[mi-hol]

brokenlinks.sh takes less than a minute on my ~5 year old laptop

I did just try again and had no result after 30 minutes!
Environment is:
WSL - Ubuntu 22.04.1 LTS running on

Operating System
Windows 11 Pro 64-bit
CPU
Intel Core i5 2520M @ 2.50GHz 60 °C
Sandy Bridge 32nm Technology
RAM
16,0GB Dual-Channel DDR3 (9-9-9-24)
Motherboard
Hewlett-Packard 161D (CPU 1)
Graphics
Generic PnP Monitor (1366x768@60Hz)
Intel HD Graphics 3000 (HP)
Storage
931GB Samsung SSD 870 QVO 1TB (SATA (SSD))

It could also be modified to only check the files that were changed in the PR.

I'd see this change as a must have

[chuckwagoncomputing]

On second thought, only checking the files that were changed isn't sufficient if a file was renamed or deleted. A solution would need to check for that.

[mi-hol]

only checking the files that were changed isn't sufficient if a file was renamed or deleted

From my view the events "file was renamed or deleted" are not handled in current GH actions.

This understanding stems from your comment "Most of the boards in firmware/config/boards have not had a pinouts made for them, which is why they fail in the workflow." in this comment but the fact that pinouts do actually exist (as example see https://rusefi.com/docs/pinouts/BB_V3/) and same error in GH action as I experienced and logged here

Is my understanding correct?

[chuckwagoncomputing]

The same workflow would trigger whether a file was modified or files were renamed or deleted.
You could just check in the 'run' section, using some git command probably, if any files were renamed or deleted, and act accordingly.

Yes, I was wrong. I thought that the board was skipped if the id was missing, turns out the script says:
# todo fail once all pages are compliant exit 1

[mi-hol]

close as script now fails if errors are detected