Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor store.service Add support for relations, enums, subscription on Schema Migration #2251

Merged
merged 23 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f39f985
update schema migration with raw query execution, instead of sequeliz…
bz888 Feb 5, 2024
629b57c
fix tests
bz888 Feb 5, 2024
6fdcc36
update changelog
bz888 Feb 5, 2024
ba27ace
tidy up test for historical db sync, update statement generators
bz888 Feb 7, 2024
cc23277
update topo logic, add tests, add sync logic with cyclic support
bz888 Feb 8, 2024
23658a1
revert bad rebase on migeration class
bz888 Feb 8, 2024
225d464
revert bad rebase on migeration class
bz888 Feb 8, 2024
cdeeec9
refacotr toposort, updated sync-helper tests
bz888 Feb 8, 2024
ab0785f
update schema migration with raw query execution, instead of sequeliz…
bz888 Feb 5, 2024
2b1e969
add relational support on schema migration, fix table comments
bz888 Feb 1, 2024
76d1946
fix tableComments, refactor createTable and createModel, update creat…
bz888 Feb 5, 2024
c3cd09b
add drop relations, add check for fkey on creation relations, add tests
bz888 Feb 8, 2024
8c20565
refactor drop table and column with relational foreign keys, fix test…
bz888 Feb 10, 2024
d63efae
update syncSchema logic with migration.run()
bz888 Feb 11, 2024
4c5fee5
added test for enum creation, and enum support on drop and creation
bz888 Feb 12, 2024
7dbb468
clean up comments and logs
bz888 Feb 12, 2024
d0b6c36
update based on review, add idempotency to db queries
bz888 Feb 13, 2024
3a8c55a
check matching ddl, fixed relation comments, benchmarked init and res…
bz888 Feb 13, 2024
fc0fa74
update logic for get exisitng f key and enumMap, refactor based on re…
bz888 Feb 14, 2024
026a794
update logger
bz888 Feb 14, 2024
1ef14fc
update tests, clean up todo
bz888 Feb 15, 2024
4f47d7a
update tests, clean up todo, update improves
bz888 Feb 15, 2024
a7cb813
fix dropEnum and dropEnum test
bz888 Feb 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion packages/node-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).


## [Unreleased]
### Added
- Schema Migration support for Enums, Relations, Subscription (#2251)
-
### Fixed
- Fixed non-atomic schema migration execution (#2244)

## [7.2.1] - 2024-02-07
### Added
Expand Down
1 change: 1 addition & 0 deletions packages/node-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
"prom-client": "^14.0.1",
"source-map": "^0.7.4",
"tar": "^6.1.11",
"toposort-class": "^1.0.1",
"vm2": "^3.9.19",
"yargs": "^16.2.0"
},
Expand Down
3 changes: 1 addition & 2 deletions packages/node-core/src/configure/ProjectUpgrade.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ import {ParentProject} from '@subql/types-core';
import {getAllEntitiesRelations} from '@subql/utils';
import {Sequelize, Transaction} from '@subql/x-sequelize';
import {findLast, last, parseInt} from 'lodash';
import {SchemaMigrationService} from '../db';
import {ISubqueryProject, StoreCacheService, StoreService} from '../indexer';
import {getLogger} from '../logger';
import {getStartHeight, mainThreadOnly} from '../utils';
import {BlockHeightMap} from '../utils/blockHeightMap';
import {SchemaMigrationService} from './migration-service';
import {NodeConfig} from './NodeConfig';

type OnProjectUpgradeCallback<P> = (height: number, project: P) => void | Promise<void>;
Expand Down Expand Up @@ -230,7 +230,6 @@ export class ProjectUpgradeSevice<P extends ISubqueryProject = ISubqueryProject>
assert(this.migrationService, 'MigrationService is undefined');
if (this.config.allowSchemaMigration) {
const modifiedModels = await this.migrationService.run(project.schema, newProject.schema, transaction);

if (modifiedModels) {
this.#storeCache?.updateModels(modifiedModels);
}
Expand Down
1 change: 0 additions & 1 deletion packages/node-core/src/configure/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,3 @@
export * from './configure.module';
export * from './NodeConfig';
export * from './ProjectUpgrade.service';
export * from './migration-service';

This file was deleted.

191 changes: 0 additions & 191 deletions packages/node-core/src/configure/migration-service/migration.ts

This file was deleted.

2 changes: 2 additions & 0 deletions packages/node-core/src/db/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@
// SPDX-License-Identifier: GPL-3.0

export * from './db.module';
export * from './migration-service';
export * from './sync-helper';
bz888 marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading