From 6de6ce7727a4323772b6c838c1cc22e94d69b885 Mon Sep 17 00:00:00 2001 From: "Rafid Aslam (refeed)" Date: Sat, 10 Sep 2022 10:14:37 +0700 Subject: [PATCH] README.md: Add step that explains how to use the sitemap.xml properly Followup of 980675c6bb268d7cd222aa84cdc11446de3ba99a --- README.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/README.md b/README.md index 3dea83da89b171..7f6b872612b4d6 100644 --- a/README.md +++ b/README.md @@ -126,6 +126,25 @@ warned that the repository size may explode. Final step is to verify that everything is working as it is supposed to be. You would have to wait for some time since the action is scheduled to run every 20 minutes (or the time you have configured it to be in above step.) You can track the status of the action by visiting `https://github.com///actions`. Once the action completes running, you would be able to visit the archive by opening the link mentioned in the action run log at the end. The link would be usually be of the form `.github.io/` or `/` if you have configured your own personal domain to point to GitHub pages. +### (Optional) Step 7 - Add sitemap.xml to robots.txt + +If you'd like your archive to be indexed properly by search engines like Google, you need to add the generated sitemap.xml path to the `Sitemap` field of your `robots.txt` that is located at the root path of your site. + + +Example: +- The generated sitemap is located in `https://example.github.io/zulip-archive/sitemap.xml` +- Add the sitemap path to `Sitemap` field of the `robots.txt` of the site that is located at `https://example.github.io/robots.txt` + +``` +# robots.txt content +Sitemap: https://example.github.io/zulip-archive/sitemap.xml + +# In case you have another `sitemap.xml` in your site, you can also add it here +Sitemap: https://example.github.io/sitemap.xml +``` + +For more reference regarding to `robots.txt` you can visit [the official documentation of it by Google](https://developers.google.com/search/docs/advanced/robots/create-robots-txt). + ## Running zulip-archive by yourselves For most users, running `zulip-archive` as GitHub actions should be good enough. If you want to run `zulip-archive` in your own server or do something else, see the [instructions](instructions.md) docs. The [hosting docs](hosting.md) also offer a few suggestions for good ways to host the output of this tool.