Fold wiki into repo, make docs with mkdocs

[honno]

Note I'm making this WIP PR in my repo for now, so we can settle on a few things before (hopefully) moving forward and making a PR on the main xianyi/OpenBLAS repo. It also lives in the develop branch here as it just makes it easy to deal with CI, but the actual PR will live in say a branch named mkdocs.

See https://honno.dev/OpenBLAS/ for a live preview.

This PR folds the contents of the wiki into the OpenBLAS repo itself.

I only recently learnt that GitHub wikis are infact git repositories. I opted to use squash all the commits in the wiki repo into one commit and add all contributors as co-authors, as I was worried preserving all the commits as-is was too noisy—let me know if I should preserve all commits. In any case, I'll have to update this dump of the wikis contents as it can get updated (and already has).

After dumping the contents, I moved all of the resulting markdown files to docs/ and did some renaming. I did a simple passthrough of all the file's contents to make things consistent (e.g. consistent header-levels, removing redundant TOCs, removing "top bars" with relevant links", etc.). I still need to update cross-referencing links.

File rename script

#!/usr/bin/bash

# dead files
rm .md
rm Document.md

mkdir -p docs

mv CI-jobs-overview.md docs/ci.md
mv Developer-manual.md docs/developers.md
mv Donation.md docs/donate.md
mv Download.md docs/download.md
mv Faq.md docs/faq.md
mv Fixed-optimized-kernels-To-do-List.md docs/todo.md
mv How-to-generate-import-library-for-MinGW.md docs/import_mingw.md
mv How-to-use-OpenBLAS-in-Microsoft-Visual-Studio.md docs/use_visual_studio.md
mv How-to-use-OpenBLAS-on-Cortex-M.md docs/use_cortex.md
mv Installation-Guide.md docs/install.md
mv Machine-List.md docs/machine_list.md
mv Mailing-List.md docs/mailing_list.md
mv OpenBLAS-Extensions.md docs/extensions.md
mv Precompiled-installation-packages.md docs/packages.md
mv Publications.md docs/publications.md
mv Related-packages-that-use-OpenBLAS.md docs/ecosystem.md
mv User-Manual.md docs/user_manual.md
mv 'WIP---Build-system-overview-(community-made).md' docs/build_system.md
mv Home.md docs/index.md

mkdir -p docs/build

mv How-to-build-OpenBLAS-for-Android.md docs/build/android.md
mv How-to-build-OpenBLAS-for-Windows-on-ARM64.md docs/build/win_arm.md
mv 'How-to-build-OpenBLAS-for-iPhone-iOS-(ARMv8).md' docs/build/ios.md
mv How-to-build-OpenBLAS-for-macOS-M1---arm64.md docs/build/m1.md

Next week I plan to do some more opinionated organisation. Right now the side panel goes alphabetically, where really we'd want high-demand pages like "Download" at the top, and more niche pages like "Building for iOS". I feel some page contents can be folded together too, specifically the download-y/install-y pages. Lots of little things too, although I'll try to just do the obvious and non-controversial stuff.

Big question is where should the HTML output of the docs live? Note the gh-pages branch is already in use, although we could still upload to there in say a docs/ folder. We could use an alternative branch, and hook it to a separate domain. Or maybe a service separate to GitHub actions? Also have to consider live previews support, which I assume would be nice to get in for this PR so is another TODO of mine (I'm quite familiar with the data-apis/array-api model of using artifacts that get pushed to CircleCI FWIW).

cc @rgommers @steppi

[rgommers]

and add all contributors as co-authors, as I was worried preserving all the commits as-is was too noisy—let me know if I should preserve all commits.

No need to do that. adding everyone as co-authors sounds fine to me.

Next week I plan to do some more opinionated organisation. Right now the side panel goes alphabetically, where really we'd want high-demand pages like "Download" at the top, and more niche pages like "Building for iOS". I feel some page contents can be folded together too, specifically the download-y/install-y pages. Lots of little things too, although I'll try to just do the obvious and non-controversial stuff.

Agreed that restructuring is useful. As long as it's just shuffling around whole files and directories, I think making the changes is low-overhead and just as easy as pre-discussing, so you can just go ahead and do it. Before doing more labor-intensive stuff, you may want to just post a bullet-point outline.

Big question is where should the HTML output of the docs live?

It could be pushed to a separate repo, and served from there - that would be ideal I think. A branch doesn't work AFAIK, GitHub Pages can only serve a single branch per repo.

Either way, let's not worry about that just yet, and focus on content first. Too much depends on the constraints around repo permissions right now.

Also have to consider live previews support, which I assume would be nice to get in for this PR so is another TODO of mine

That CircleCI approach is probably the best we've got - but it's another CI integration that needs installing, and it tends to break fairly often.

[rgommers]

Nice work so far @honno, thanks for sharing. Have a great weekend!

[honno]

@rgommers @steppi et. al, here's my second iteration for review. Most of it is uncontroversial and not really noteworthy. The exception to this is my changes to install.md, folding in disparate content from other pages to make it a bit monolithic—to me this makes sense, lemme know if that's something to reconsider/revert.

I think there's other areas which I can streamline, but probably it's all stuff that can be done in future PRs.

[rgommers]

The exception to this is my changes to install.md, folding in disparate content from other pages to make it a bit monolithic—to me this makes sense, lemme know if that's something to reconsider/revert.

That does seem like a good idea to me. It's also something that can then be made more uniform, or easier to navigate in the future via tabs or some such thing (let's not do that now though).

[honno]

Hi @martin-frbg, this was our idea on a start for OpenBLAS docs (https://github.com/Quansight-Labs/sprind-openblas-mgmt/issues/8). As a quick primer, this PR is folding the contents of the wiki into the OpenBLAS repo itself in the docs/ folder, and notably:

• mkdocs is deployed to generate docs.
  • Live preview at https://honno.dev/OpenBLAS/
• All pages were updated to have consistent markup/formatting/styling.
• Disparate install information in the wiki was all folded into install.md ("Install OpenBLAS").

I've tried to be fairly unopinionated so far, but I think interesting future steps would be:

• Fold homepage, "Users of OpenBLAS", "Publications", "Mailing list", "Donation" page contents into a "Project information" page.
  • Homepage could just be a quickstart to using OpenBLAS, nudging readers to look at the "Install" and "User manual" pages.
• Remove "Machine list" and "TODO" pages—they seem very outdated and not that useful in docs.
• Make sure all pages have up-to-date information.

How do you feel about the approach we've employed here? Anything you'd like me to do or try out? Seems good to identify/resolve any major issues here first before I make a PR to xianyi/OpenBLAS.

[martin-frbg]

online content could probably be served from openblas.net (repo for which is openblas.github.io).
wiki pages like donations and the old machine list could simply be left out (unless/until discussed with xianyi - about the last thing I want is to have our cooperation look like a hostile takeover to him and/or any group leaders in the Chinese Academy of Sciences who may be still formally associated with their side of the project)
overall a bit unhappy to see a milestone 5 topic require my attention now when even our recent report had it as not started/not in 2023 unless needed

[rgommers]

online content could probably be served from openblas.net (repo for which is openblas.github.io).

That sounds like a good idea to me.

last thing I want is to have our cooperation look like a hostile takeover

definitely!

overall a bit unhappy to see a milestone 5 topic require my attention now when even our recent report had it as not started/not in 2023 unless needed

Hi Martin, I asked @honno to start on this since he had bandwidth, this task can be done at any time (and I lost some sleep over the last-minute efforts for the previous milestone), and the wiki is something that no one without permissions can contribute to. The idea was to do this on @honno's own fork so it would not generate notifications yet and require only a minimal amount of time from you - basically, I suggested to @honno that we'd like your input on the overall outline only at this point, so we can take any wishes / likes / dont-likes into account. However, if you'd prefer to not spend any time right now and only get a PR for this to https://github.com/xianyi/OpenBLASin early 2024, please say so and we'll plan for that.

[martin-frbg]

sorry, I'm basically juggling two jobs with my regular CAD stuff so I cannot always put in time here.
Just please do not create a PR until I've had a chance to clue in xianyi (if he ever responds to my messages, but there appears to be some RISCV conference and/or trade fair in China now). I'm worried he could suspend my collab status if I interfere too much with his father-of-openblas storyline

[martin-frbg]

xianyi is now on board with this too, so I guess you/we can go ahead as time permits

I agree with you. That is a good idea to use mkdoc or readthedocs.
I think we need a history page to describe the openblas develop
history (forked from GotoBLAS2),
and link to the old "Donations" and GotoBLAS docs.

[rgommers]

I've tried to be fairly unopinionated so far, but I think interesting future steps would be:

* Fold homepage, "Users of OpenBLAS", "Publications", "Mailing list", "Donation" page contents into a "Project information" page.

That sounds reasonable to me. A page like that is often called "About". I'd put the license info there as well, and remove it from the home page. Furthermore, I would remove the full text, but just say that it's 3-clause BSD and link to the actual license file on GitHub, because they're duplicate content which is already non-identical now.

Note also the need for a separate History page from Martin's comment above.

  * Homepage could just be a quickstart to using OpenBLAS, nudging readers to look at the "Install" and "User manual" pages.

A "quickstart" example is a little tricky - would you choose C or Fortran, what about the build step which isn't portable across OSes, etc. I'd suggest just a slightly expanded description of what OpenBLAS is and does, and then linking to a few other key sections (a la https://numpy.org/devdocs/).

* Remove "Machine list" and "TODO" pages—they seem very outdated and not that useful in docs.

agreed

* Make sure all pages have up-to-date information.

definitely.

All these things seem quite common-sense and look uncontroversial. I suggest to make those changes @honno. And I'll promise to review with a much shorter turnaround time after that is done.

[martin-frbg]

Not sure I'd want the license info completely removed from the homepage, as this may be the first question for some potential users - but agree that just calling it "BSD" there should suffice.

[honno]

Thanks both for all the pointers, I think I've covered most things now. Mind there's still a live preview at honno.dev/OpenBLAS.

Note I've left the wiki refs in the README.md for now, as these would depend on the location of the wiki (unless we wanted to reference the doc markdown files e.g. docs/install.md). As mentioned I think just supplanting openblas.net with these docs seems like a good idea. Generally it could be that this PR could go in first, then a future PR can update references to a live documentation, and then we delete the GitHub wiki.

I think we need a history page to describe the openblas develop history (forked from GotoBLAS2)

Something I suggest as a follow-on PR from this (can write an issue after this hypothetically gets merged into the main repo), given I'm not equipped to write about the history myself! This also might be something for the "About" page if it won't be too comprehensive.

[rgommers]

Thanks @honno. I reviewed the last commits and went through the deployed preview from beginning to end. All recent comments from Martin and me seem addressed.

There is a lot to improve in general, however I think this is indeed good to get this merged and deployed in its current form; after that it will be significantly easier to improve content from its current state in a way that is reviewable.

[martin-frbg]

given I'm not equipped to write about the history myself! This also might be something for the "About" page if it won't be too comprehensive.

agree, and while I tried to put something together, I realized I'm not entirely clear on some aspects of the timeline myself - I think there must have been some "dark" phase where Xianyi kept the development private before putting it out as OpenBLAS again, but exactly how and when he first came in contact with the code I don't know (and also how soon after GotoBLAS2 was abandoned by Goto did he pick up the bits ?)

so all I have now is "OpenBLAS is a continuation of the GotoBLAS codebase that was written from scratch by Kazushige Goto, then a research associate at the Texas Advanced Computation Center, in the early 2000s. Goto had to abandon his brainchild in 2010 when he left academia, and subsequently projects like Nakama's SurviveGotoBLAS2 tried to keep it working on newer hardware. When Zhang Xianyi became a postdoc in the same group that had hosted Goto a few years earlier, he combined Goto's work and the various patches, and continued to add to it after his return to the Chinese Academy of Sciences, eventually making it available as OpenBLAS"