Run Missing Migrations

[speeddm]

Is it possible / would it be possible to allow Simple.Migrations to, when migrating to the latest version, run any migration it has not yet run. This would be useful in our scenario where we are developing as a team on various branches with our own local databases and using a timestamp to populate the version on each migration.

So if a migration gets merged into the master branch but a user has already run the migrator on their database then the migrations contained in the branch that got merged in are not run. I understand that this runs migrations out of order, but can be very useful in a team environment and when it comes time to run the migration in any deployed environment then the order is always going to be preserved.

[canton7]

Sounds reasonable.

I can see this working in a number of different ways:

1. Apply all missing migrations before the current migration
2. Un-apply all migrations since the first missing migration, then re-apply everything
3. Either of the above, but going to a specific migration in the future (or the latest migration).

[canton7]

Thinking about this, I think it makes sense to add a "skipped migration behaviour" parameter to the commands which migrate up or down, which can take the values "throw an exception", "apply out of order", or "rewind to the first missing migration and reapply all since". The ConsoleRunner will accept a flag to set the behaviour, and you can set the default programmatically. I think that's enough to cover your use-case which being fairly generic..

[canton7]

Actually, the complication is that I don't currently keep track of the set of migrations which have been applied. The version table forms a history of which migration was applied when, so I guess I can walk that, but I don't currently require that the DatabaseProviders use the version table in that way.

[canton7]

Of course, it would be possible to introduce another table which tracks the currently-applied migrations, but:

1. I have to cope with people who are running Simple.Migrations and don't have this table. That's possible, but I think it's going to be pretty tricky. If I get it wrong, things might start failing in production for people.
2. I'm cluttering up the user's schema with additional tables at this point

It's almost as if I need to track the version of the table used by Simple.Migrations as well as the current migration...

[canton7]

Playing devil's advocate, is there a reason why migrating down to 0 and back to the latest version won't work for you?

[speeddm]

We don't tend to use the down section when writing migrations. And with the branches I can still foresee issues - when you pull in a branch that contains other migrations wont the migrator try and apply the down migration on migrations that have never had actually been applied.

Generally how we operate (and this may not be the way all teams do or indeed should) is that we simply run the migration to get our local DB to the latest point (including any migrations that have come in from other branches) and if we do require a certain version of the DB for debugging etc then we would look at using a backup (obviously not ideal).

Isn't there enough information in the VersionInfo table to work out which migrations have already been applied?

[canton7]

- when you pull in a branch that contains other migrations wont the migrator try and apply the down migration on migrations that have never had actually been applied

Not if it knows which migrations have been run, which is a prerequisite to any solution to your issue. It's possible to figure out which migrations have been run from the version table, but not ideal. Database providers are not required to keep a history of the versions of the database: the built in ones do, but ones which users have written might not. It requires walking the table and reconstructing the state of the schema at each point: an operation which takes longer the more migrations you've applied (this applies more to people who *do* use the down migrations, rather than backup, to restore their development database to an earlier version).

…

On 28 August 2017 00:52:47 BST, speeddm ***@***.***> wrote: We don't tend to use the down section when writing migrations. And with the branches I can still foresee issues - when you pull in a branch that contains other migrations wont the migrator try and apply the down migration on migrations that have never had actually been applied. Generally how we operate (and this may not be the way all teams do or indeed should) is that we simply run the migration to get our local DB to the latest point (including any migrations that have come in from other branches) and if we do require a certain version of the DB for debugging etc then we would look at using a backup (obviously not ideal). Isn't there enough information in the VersionInfo table to work out which migrations have already been applied? -- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: #25 (comment)

[gregbrant2]

I realise this is an old ticket but I'm about 99.9999% in the same use case as @speeddm.

A thought regarding the connundrum of does the provider support the VersionInfo table. Could you introduce a property into the provider base type bool SupportsVersionTable {get;set;} = false; and so the provider author has to opt-in setting it true in their constructor, and hopefully reading the docs and understanding why they've set it true, and implementing the appropriate functionality. Then, if SkippedMigrationBehaviour is set to ApplyOutOfOrder" or anything else that relies on SupportsVersionTable, you could pre-flight check that SupportsVersionTable == true and throw if it's not supported.