Runtime Migration

[ignazio-bovo]

This is a note to be used as a reference in case of future (storage) migrations

Runtime Side

1. Every pallet should have a version (u32 is fine) in his storage space, I call this on_chain_version
2. Every pallet should have a version (u32 is fine) hardcoded in the module. I call this current_storage_version
   Hence I would like to propose this approach this implementation approach for each pallet
3. 1 + 2 together allows to implement the GetStorage trait for Module<T>. Hence let CHECK = on_chain_version != current_storage_version, if CHECK == true migration is needed.
4. I should have a separate module for migration like

pub mod migration {
  use super::*;

  pub mod v1 {
      // data structures definitions used in previous storage version
  }

  pub fn migrate_to_v2<T: Config>() -> frame_support::weights::Weight {
      if CHECK {
          // translate storageV1 types into storageV2 types
      }
}

5. Having on_runtime_upgrade in runtime/src/runtime_api.rs call the appropriate migrate_to_v2 function

Testing

4 current ways to test the runtime upgrade Sumer -> Giza:

1. two repos: have 2 repos (one with Sumer, the other with Giza) run one test scenario on Sumer in order to create some state to mess up with, then perform the migration and complete assertion launching a test scenario from giza
2. webpack: as a fixer to the previous method we can bundle the whole Sumer augmented api into a sumer.js file using webpack and import it into the Giza test scenarios
3. substrate remote externalities / try_runtime utils: this allowes to download a state snapshot of a live chain at a specified block height. This is more similar to a unit test since it download and executes rust code into a sandbox environment in which you can verify pre/post migration invariants/assertions.
4. fork-off: forkoff allows to turn a state snapshot of a live chain at specified block height into a readable chainspec file.

Fork off tool

Seems the most promising, and it is the workflow that best fit the current integration testing framework.
I edited the forkoff script that clones the provided Sumer storage into a chainspec file. It does not preserve the Set of validator set see Plus a problem on the HTTP endpoint side of the RPC server forces me to download the current state with a curl
query instead of using polkadotJs @rpc/HttpProvider. What the script essentially does is the following: starting with a --dev Sumer-compatible chainspec file copy the state from the storage.json dump into a the specified chainspec. The runtime version is also copied into the
genesis.
I need to put the Giza wasm runtime in the test/data_tmp folder with the generated chain-spec.
fork-off changes the runtime-code value on the chain-spec to the hex dump of the runtime .wasm (so no need to set migration
code via Tx).
Then we can run the
post-migration-scenario

[ignazio-bovo]

Method 4 timeline:

• connect to sumer chain and clone a chainspec
• start chain succesfully with cloned chainspec
• perform runtime upgrade succesfully
• add Migration detection
• run post-upgrade scenario with succesful assertions