Create a Gdoc content migration workflow

[ikesau]

Core problem

We currently have no way to programmatically update the content of google documents, if ever we want to migrate an archie component to have a different shape.

Proposed solution

Create some sort of migration tooling that:

1. Takes a migration specification
2. Iterates through all published gdocs
3. Operates on them (probably as raw blocks)
4. Converts them back to archie
5. Replaces the google document content with the new archie
6. Republishes the post

Possible usecases

• Migrating data insights to have only one image (i.e. dropping filename and renaming smallFilename -> filename)
• Updating / removing all the titles in all-charts blocks

Technical considerations

If we operate on raw blocks to update the shape of a component, we'll potentially have to create a "version 1 backup" of the TS interface

e.g.

import { RawBlockImage } from "@ourworldindata/types"

type OldRawBlockImage = {
  // the values this block used to have, before the migration
}

function migrateImage(block: OldRawBlockImage): RawBlockImage {
  // stuff
}

[danyx23]

It would be worth checking out if anyone has solved this problem before (there are a few high profile users of ArchieML in Google Docs)

[danyx23]

We chatted about three different ways of tackling this:

• extending our full ArchieML pipeline all the way back to GDrive AST and patch the document so as not to delete :skip sections
• we could have explicit schema versions as part of the document metadata and have humans manually upgrade those (according to a guide). There could also be an admin UI that makes this part easier.
• we could write migrations that operate on Gdrive AST -> Gdrive AST functions that just delete/add/modify the properties that change