Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tough async #3566

Merged
merged 3 commits into from
Nov 30, 2023
Merged

tough async #3566

merged 3 commits into from
Nov 30, 2023

Conversation

webern
Copy link
Contributor

@webern webern commented Nov 2, 2023 •
edited
Loading

Issue number:

Description of changes:

Use tough in its async incarnation.

Testing done:

  • Before the PR landed
    • Run an image
    • Downgrade with migrations
    • Upgrade with migrations
  • Repeat the above once we have a released version of tough with these changes.

Note: during the re-test, when I looked at migrator's jounral entries I saw some errors on downgrade. This is was v1.15.1 which was built from the tag (and thus not running the new async code).

Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.1_updog-network-affected.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_schnauzer-v2-generators.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_public-control-container-v0-7-5.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_aws-control-container-v0-7-5.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [ERROR] Migration stderr: 'settings.host-containers.control.source' has no 'template' key in metadata
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_public-admin-container-v0-11-1.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_aws-admin-container-v0-11-1.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [ERROR] Migration stderr: 'settings.host-containers.admin.source' has no 'template' key in metadata
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-settings.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-restart.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-files.lz4'
Nov 29 19:33:13 localhost migrator[941]: 19:33:13 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-configs.lz4'

On upgrade (running the new async code) I did not see these errors. For the purposes of these PRs, the testing shows that migrator is working correctly. Someone may want to look into the host container migrations on downgrade to see what's up.

Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-configs.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-files.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-restart.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_kernel-modules-autoload-settings.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_aws-admin-container-v0-11-1.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_public-admin-container-v0-11-1.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_aws-control-container-v0-7-5.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_public-control-container-v0-7-5.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.0_schnauzer-v2-generators.lz4'
Nov 29 22:58:47 localhost migrator[942]: 22:58:47 [INFO] Running migration 'migrate_v1.16.1_updog-network-affected.lz4'

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@webern webern mentioned this pull request Nov 2, 2023
Merged
@webern
Copy link
Contributor Author

webern commented Nov 11, 2023

https://github.com/bottlerocket-os/bottlerocket/compare/c4e1ebd0b32c7fbf4e097451ae1eaf60bd45d7f7..7ae81a9cb8e1e4c531288d4b99b279cc1cd1e18a

Update PR to released version of tough async.

@webern
migrator: refactor test code
fd6e774 
This refactor of test code will make it easier to
switch to async since the tokio::fs::read_dir API does
not match the std::fs::read_dir API.
@webern
migrator: update tough to async version
1d9ca9d
@webern
Copy link
Contributor Author

webern commented Nov 29, 2023

https://github.com/bottlerocket-os/bottlerocket/compare/8f4dd999204c568b55ce43ae5c0912a0d27620de..4bd61e9de85fece5a81146b6542d8eb1055bdb96

Rebase

@webern
Copy link
Contributor Author

webern commented Nov 29, 2023

https://github.com/bottlerocket-os/bottlerocket/compare/4bd61e9de85fece5a81146b6542d8eb1055bdb96..3fae5483acefa97c3fa89e5b510853966f63324c

Fix an issue in updog.

@webern webern changed the title Tough async tough async Nov 29, 2023
@webern webern marked this pull request as ready for review November 29, 2023 23:04
@webern webern requested review from etungsten and zmrow November 29, 2023 23:06
@etungsten
Copy link
Contributor

Would be good to triage the errors on downgrade for the host-containers migration.

@webern
Copy link
Contributor Author

webern commented Nov 30, 2023

Would be good to triage the errors on downgrade for the host-containers migration.

I have confirmed that this happens when running a v1.16.1 Bottlerocket built from develop (b375585) and downgrading to an image built at the v1.15.1 tag. I created an issue to track this unrelated problem: #3629

@webern webern merged commit 6c9d9a3 into bottlerocket-os:develop Nov 30, 2023
46 checks passed
@webern webern deleted the tough-async branch November 30, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@etungsten etungsten etungsten approved these changes

@zmrow zmrow zmrow approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

use tough async in migrator and updog
3 participants
@webern @etungsten @zmrow