[RFC] Approach to hosting Team Manuals, Internal Documentation and Wikis

[paroxp]

[RFC] Approach to hosting Team Manuals, Internal Documentation and Wikis

Problem

With the sunsetting of the GOV.UK PaaS, GDS is faced with a very common problem that needs solving among the teams how and where to host Team Manuals, Runbooks, Internal Documentation, RFCs and ADRS or simple Wiki pages.

Context

Whilst we’re surrounded by productive, smart and creative people, we will notice a huge divergence between different solutions to said problems.

Most common approach to this issue has been building a Middleman Application by each team individually, utilising static HTML generation for the web pages with the use of tech-docs-gem which brings in the GOV.UK look and feel. The generated artefact are generally hosted on GOV.UK PaaS, with the use of the static buildpack and be treated as a sunk cost as PaaS is free for GDS users.

This however introduces a lot of additional concerns and responsibilities for the teams to have.

• What does the deployment pipeline look like?
• Who’s patching the tech-docs-template gem?
• Who’s responsible for upgrading the dependencies and when?
• Should we be using GOV.UK branding on non-GOV.UK domains?
• Where would we host tech-docs-template based docs in a post-PaaS world?

To address some of that, Digital Identity started to utilise its Atlassian Confluence instance instead.

This RFCs aims at an open discussion and an attempt at informing, guiding and advising teams on what the options exposed to us are, and to hopefully build a consensus in advocating for a single approach.

Options

Middleman or alternative application - evolved in hosting strategy

Whilst this is the most common approach, most teams would likely be most comfortable sticking with it.

It introduces no changes in the codebase, little changes to the deployment pipelines, but requires a lot of thought, perhaps centrally to establish the best way of hosting these pages.

It also leaves some concerns around the maintainability of the solution for the teams, or their lack of for more centrally managed documentation such as GDS Way.

It is difficult to estimate the cost of this option as it depends on the possible hosting choices, but would be minimal if, for example, we use GitHub Pages to host these.

More details here

Confluence

This option reduces maintainability; responsibility for hosting and patching; offers fine grained access controls and versioning of the pages; and provides a good user experience for both technical and non-technical colleagues. However, where staff are not already familiar with Confluence, this will involve some upskilling and learning effort.

It does expose GDS to a rather large procurement and Joiners/Movers/Leavers management overhead. There would also be a cost of procuring a central Confluence system.

It would require a substantial amount of effort to move the contents from GitHub repositories onto, and graceful deprecation of existing repositories, pointing users to Confluence in order to avoid confusion. This also means any prior versioning history to date would be lost.

It also loses the branding capabilities.

More details here

GitHub Wiki

Much like Confluence, it reduces the maintainability; responsibility for hosting and patching; and offers fine grained access controls and versioning of the pages.

It does favour the technologists a bit more, as the entry level requires basic understanding of git and version control. It would also require adding more non-technical staff to the GitHub org, consuming more GitHub licences.

It is however already procured for all of the Cabinet Office teams, and integrated into GitHub, allowing us to avoid any additional procurement.

It may expose little amounts of work, removing middleman and deprecating deployment pipelines, but the work should be minimal, without losing already existing versioning history.

Like Confluence, it loses branding capabilities.

More details here

Proposal

Provided:

• the simplicity of Markdown
• version control of Git and perseverance of history
• existing contract with GitHub for a large number of users
• Near zero maintenance overhead
• access management
• no need for deployment pipelines

We’d like to propose the use of GitHub Wiki for any Team Manuals, Runbooks, Internal Documentation and otherwise Wiki pages.

Implications

• We will lose the ability to brand our pages for accessibility (also true with Confluence)
• We may need to onboard less technical colleagues to GitHub (similar to Confluence)
• We are deferring responsibility for uptime to a third party (also true with Confluence)

[jfharden]

Another possibility is github pages. We are now building our product pages as a static site and using github pages to host it (in our case, since it's our product pages, we also have a CloudFront distribution in front of it and serve from www.payments.service.gov.uk, but that's not required).

We plan on doing the same thing with our team manual, tech docs, etc.

[stephengrier]

There are also public product pages to consider; these are by definition public. We have a decent pattern of using the tech-docs-template deployed to GitHub Pages for these, and so there's an argument for using the same pattern for internal documentation too if we're using it for product pages.

[paroxp]

I'm happy to rename that point. But...

Should we also consider things like S3, Heroku, AppRunner or PaaS 2.0 as separate options also?

Maybe I could add sub section listing a few ideas for hosting as oppose to tailoring the option specifically into one of these?

In the same manner, would a re-write from Middleman into Next.js be a consideration, provided GOV.UK Frontend is Node module? Other static site generators?

Just not sure how much of "sub-options" do we want to cover here.

I appreciate we're doing fair bit of solutionising here. But I think we've somewhat accidentally focused on the idea how we do these kinds of static pages as oppose how do we solve individual option 🤔

[pauldougan]

@paroxp not sure its helpful to mention PaaS 2.0, this is not even a thing and is unlikely to ever exist so its more misleading for teams needing some practical assistance between now and the demise data of paas which is Dec 23 2023. Better to focus on actual concrete technologies that can be used now.

[paroxp]

I don't disagree Paul. That is not mentioned in the main body, but in the speculation of what should be in and out of scope. Agree, any future PaaS, without concretes is Out of Scope, which again, above comment meant to highlight the lack of necessity of detailing all possible options.

[stevenjmesser]

I wonder whether this is less about making sure all teams use the same approach and more about ensuring each team understands the pros and cons of each approach. Help them pick something (or make it OK to change their mind later). Not all teams have the same size, mix of roles, etc., therefore a one-size-fits-all approach might not work.

[jfharden]

One reason I especially dislike confluence is it totally removes the code-in-the-open aspect. We can't have our team manuals (and other similar things) in a public space if they are in confluence. It also can make things more difficult where we need external assessors to have access (such as our PCI QSA who requires team manual access to some things which are not-public due to security concerns).

[stephengrier]

I agree, I don't think we should recommend a tool that means all our docs would require an account to view. However, it does look like it is possible to make confluence "spaces" public: https://confluence.atlassian.com/doc/make-a-space-public-829076202.html

[jfharden]

I think overall I'm probably in agreement with your conclusion despite the things I said above :)

However I would hate to see a commandment/mandate to move documentation to there. As a path forward for uncertain teams, and as a way to go if you don't currently have anything, I can agree.

It shouldn't be underestimated though the effort in moving from one system to another, and the number of links in documentation everywhere (so many presentations, google docs, word docs for cabinet office etc) which would now be dead

[jfharden]

On the note of links......and it's a github wiki question.....can you put in redirects for moved pages? We do that quite a bit in pay (or when something is decommissioned, a redirect to the newer version) to ensure things we have codified in PCI documentation etc still lead to valid places. I think the answer to this is you can't do it. This is a definite down side to a documentation site if you can't get moved pages to redirect from their OLD URL (I believe Confluence can do this as long as it's within the same space)

[paroxp]

Yes. I don't think we're in the position of mandating anything... We would need to have case by case plannings and estimations if we were, or some capability performing that work for us.

I am unsure about GitHub redirects. Worth a spike - very good and valid point.

[danworth]

We've been using Github Wiki on Forms for our team manual and internal documentation for a few months now. Overall my opinion is positive and we're not considering a change. Here's our wiki https://github.com/alphagov/forms-team/wiki
Some thoughts from our experience.
Good:

• Its simple to edit and contribute via the Github UI (Github wiki doesn't support PR workflows but it keeps things simple which reduces the barriers to contributing which I think is important)
• We manage just the content (no infra or subscriptions etc).
• Github's own pages on how to use the wiki are pretty good to help those less familiar with github wikis to begin contributing. https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis
• There is a revision history for each page so you can see and compare edits.

Not so good:

• Searching for content across pages isn't great (the browsers search within a page is sufficient)
• We've added a sidebar index type view to help group content. The native pages dropdown just shows the pages in alphabetical order which isn't great by itself.
• Github has reliability issues which means it might not be available. Might be ok for these docs, unless you've got a runbook/wiki page about "what to do if Github is down"
• Since there's no PR workflow its not obvious how to suggest an edit to a page or comment on a change (you can find various ideas for work arounds to enable a "PR like" workflow on a wiki but they introduce unwanted complexity beyond the standard PR process). Our workflow at the moment is make the change or add content then let others edit as necessary which seems good enough for internal things.

[paroxp]

Since there's no PR workflow

Interesting! I knew that there is a *.wiki.git which I suppose I would expect to also have a PR system in place for. I can think of some work arounds, which may be just simple enough to mitigate the issue...

But I would be very keen for us to chat with GitHub themselves about this potential feature request - which I'm sure they've already recorded...

[lfdebrux]

To be explicit, does this proposal also include moving the GDS Way?

[paroxp]

Yes, I believe it touches on the same issues we have with GDS way :)

[sdurnberger]

This is a really important issue, and I just wanted to note a couple of things:

• there was a recent discovery on tech docs tooling at GDS to look at some of these issues and propose next steps
• some of the problems you've mentioned were raised to me on Digital Identity, and I've started some work on looking at considerations - happy to share what I've got if you'd like to see
• the accessibility monitoring team has confirmed that WCAG applies to all internal documentation too, so we need to make sure anything we propose meets upcoming requirements such as WCAG 2.2

Maybe we could get in touch for a chat to at least look at the DI side of things!

[paroxp]

Sounds like a plan! I'm very happy and keen to chat. But we should make an effort to record the discussion on this RFC :)