Add robots exclusions for version specific documentation

[gbartolini]

We should encourage search engines to index only the current version documentation, and exclude the specific version docs.

For example:

• allow https://cloudnative-pg.io/documentation/current/
• deny https://cloudnative-pg.io/documentation/1.17/
• deny https://cloudnative-pg.io/documentation/1.16/
• and so on

[revolunet]

Hi,

When landing on some legacy doc page, would be great to have a banner that says something like

Should that be a mkdocs option to add that kind of banner ?

[gbartolini]

Do you think you can help here with a PR?

[revolunet]

i'd love to :) let me take another look now

[revolunet]

I dont have any good idea to solve this;

I can add some code to hack/import_docs.sh that ensure other versions have that additionnal HTML content in their pages. a dummy regexp search'n'replace at ul.wy-breadcrumbs level for example.