-
-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(windows): test State Machine in each "state" with each "event" #13059
Open
rc-swag
wants to merge
2
commits into
beta
Choose a base branch
from
feat/windows/test-state-transitions
base: beta
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
User Test ResultsTest specification and instructions
Results Template
Test Artifacts |
ermshiperete
approved these changes
Jan 29, 2025
…ansitions # Keyman Conventional Commit suggestions: # # - Link to a Sentry issue with git trailer: # Fixes: _MODULE_-_ID_ # - Give credit to co-authors: # Co-authored-by: _Name_ <_email_> # - Use imperative, present tense ('attach' not 'attaches', 'attached' etc) # - Don't include a period at the end of the title # - Always include a blank line before trailers # - More: https://github.com/keymanapp/keyman/wiki/Pull-Request-and-Commit-workflow-notes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tests for state transition matrix here is a spreadsheet with the tests and here is the original state transition matrix.
This keyboard is used for some tests. sil_euro_latin.zip
These test may require using the registry editor to get the state machine in the correct state for the test.
It may also require deleting the cache folder.
Resetting the
last updated
registry value.Finally starting keyman to test the "haskeyman" run OR logging out so Keyman is unloaded from memory.(keyman has not run)
Test names follow TEST _ STATE _ EVENT _ EXTRA _ DETAIL
Where STATE is usIdle, usUpdateAvailable... et al. The state of the update process
and _EVENT is the event that is occurring in this state.
User Testing
These tests are not complete end to end tests of updating Keyman. They are exercising a particualr code path, are particular scenario. Verifying it is working as designed.
IDLE
TEST_IDLE_KMSHELL_UPDATEDS_KEYMAN_ONLY
For this test, we need to make sure we have not already updated to the latest alpha build
To test KMSHELL and not the REQ_CHECK event the last 7 day date will prevent us. So we need to delete or reset the
last update check time
registry key. for these tests I will have one of each.update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Desktop
and Delete thelast update check time
keycd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has advanced to eitherusUpdateAvailable
orusDownloading
. Press F5 to refresh the view.TEST_IDLE_KMSHELL_UPDATEDS_KBDS_ONLY
For this test we need Keyman to be the latest alpha build
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Desktop
and Modifylast update check time
select each hex value and set it to zero see screenshot.cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has advanced to eitherusUpdateAvailable
orusDownloading
. Press F5 to refresh the view.TEST_IDLE_KMSHELL_UPDATEDS_KBDS_BOTH
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Desktop
and Delete thelast update check time
keycd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has advanced to eitherusUpdateAvailable
orusDownloading
. Press F5 to refresh the view.TEST_IDLE_KMSHELL_NO_UPDATES
WinR type regedit
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Desktop
and Delete thelast update check time
keycd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
should still beusIdle
. Press F5 to refresh the view.TEST_IDLE_REQCHECK_NO_UPDATES
WinR type regedit
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -buc
update state
should still beusIdle
. Press F5 to refresh the view.TEST_IDLE_REQCHECK_UPDATEDS_KBDS_BOTH
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -buc
update state
has advanced to eitherusUpdateAvailable
orusDownloading
. Press F5 to refresh the view.TEST_IDLE_APPLY_NOW
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -an
update state
should still beusIdle
. Press F5 to refresh the view.TEST_IDLE_CONFIGSWITCH
Follow steps 1-11 of TEST_IDLE_KMSHELL_UPDATEDS_KEYMAN_ONLY
12. Type
kmshell.exe -a
13. Verify the Keyman Configuration Window opens on the support tab
TEST_IDLE_FIRST_RUN
Follow steps 1-11 of TEST_IDLE_KMSHELL_NO_UPDATES
12. Type
kmshell.exe -firstrun=StartWithWindows,CheckForUpdates,AutomaticallyReportUsage,InstallDefaults, -defaultuilanguage=en
13. In the Regedit window verify the
update state
should still beusIdle
. Press F5 to refresh the view.TEST_IDLE_DOWNLOAD
Follow steps 1-11 of TEST_IDLE_KMSHELL_UPDATEDS_KEYMAN_ONLY
12. Type
kmshell.exe -bd
13. In the Regedit window verify the
update state
should still beusIdle
. Press F5 to refresh the view.Update Available
TEST_U_AVAILABLE_KMSHELL_NO_AUTO
For this test we need to make sure we have not already updated to the latest alpha build
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Desktop
and Delete thelast update check time
keycd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has advanced tousUpdateAvailable
and stays thereyou can continue on to TEST_U_AVAILABLE_KMSHELL_AUTO
TEST_U_AVAILABLE_KMSHELL_AUTO
Follow all steps in TEST_U_AVAILABLE_KMSHELL_NO_AUTO
type kmshell.exe -c
update state
has advancedusDownloading
TEST_U_AVAILABLE_REQCHECK
Follow all steps in TEST_U_AVAILABLE_KMSHELL_NO_AUTO
type kmshell.exe -buc
update state
has stayedusUpdateAvailable
Can continue if wanting to TEST_U_AVAILABLE_APPLY_NOW or TEST_U_AVAILABLE_CONFIGSWITCH
TEST_U_AVAILABLE_CONFIGSWITCH
Follow all steps in TEST_U_AVAILABLE_KMSHELL_NO_AUTO then complete the next two steps
type kmshell.exe -a
TEST_U_AVAILABLE_APPLY_NOW
Follow all steps in TEST_U_AVAILABLE_REQCHECK
update state
has advanced to eitherusDownloading
oruInstalling
. Press F5 to refresh the view.TEST_U_AVAILABLE_FIRST_RUN
Follow all steps in TEST_U_AVAILABLE_KMSHELL_NO_AUTO
kmshell.exe -firstrun=StartWithWindows,CheckForUpdates,AutomaticallyReportUsage,InstallDefaults, -defaultuilanguage=en
update state
is nowusIdle
TEST_U_AVAILABLE_DOWNLOAD
Follow all steps in TEST_U_AVAILABLE_KMSHELL_NO_AUTO
kmshell.exe -bd
update state
has advanced to eitherusDownloading
oruInstalling
. Press F5 to refresh the view.Downloading
The intent of all these tests is that they start with state being
usDownloading
before testing the event.TEST_DOWNLOADING_KMSHELL
Follow TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -c
update state
isusDownloading
Note it once the download has finished it will be in state
usWaitingRestart
the event from step 1 should have no effect on the processing. It will continue downloading.TEST_DOWNLOADING_REQCHECK
Follow all steps in TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -buc
update state
isusDownloading
Note it once the download has finished it will be in state
usWaitingRestart
the event from step 1 should have no effect on the processing. It will continue downloading.TEST_DOWNLOADING_APPLY_NOW
Follow all steps in TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -an
update state
isusDownloading
apply now
should now be present at found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).Note it once the download has finished it will be in state
usWaitingRestart
the event from step 1 should have no effect on the processing. It will continue downloading.TEST_DOWNLOADING_CONFIGSWITCH
Follow all steps in TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -a
update state
isusDownloading
Note it once the download has finished it will be in state
usWaitingRestart
the event from step 1 should have no effect on the processing. It will continue downloading.TEST_DOWNLOADING_DOWNLOAD
Basically a no-op
Follow all steps in TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -bd
update state
isdownloading
Note it once the download has finished it will be in state
usWaitingRestart
the event from step 1 should have no effect on the processing. It will continue downloading.TEST_DOWNLOADING_FIRST_RUN
Follow all steps in TEST_U_AVAILABLE_DOWNLOAD
kmshell.exe -firstrun=StartWithWindows,CheckForUpdates,AutomaticallyReportUsage,InstallDefaults, -defaultuilanguage=en
update state
isusIdle
WaitingRestart
TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).usIdle
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has advanced tousUpdateAvailable
, thenusDownloading
and finallyusWatingRestart
press F5 to refesh.type kmshell.exe -c
update state
has stayedusWatingRestart
TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_NOT_RUN
Follow all steps in TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
install
orclose
choose installTEST_WAITINGRESTART_REQCHECK
follow TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN steps 1-14
16. At the command prompt again
type kmshell.exe -buc
17. In the Regedit window verify the
update state
has stayedusWatingRestart
TEST_WAITINGRESTART_APPLY_NOW
follow TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN steps 1-14
15. At the command prompt again
type kmshell.exe -an
16. In the Regedit window verify the update goes to
usInstalling
thenusIdle
17. This will start installing and it may quickly go to
usIdle
once installed so theusInstalling
State is missed.TEST_WAITINGRESTART_CONFIGSWITCH
follow TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN steps 1-14
15. At the command prompt again
type kmshell.exe -c
16. In the Regedit window verify the
update state
has stayedusWatingRestart
19. Keyman Configuration opens
TEST_WAITINGRESTART_DOWNLOAD
follow TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN steps 1-14
15. At the command prompt again
type kmshell.exe -bd
16. In the Regedit window verify the
update state
has stayedusWatingRestart
TEST_WAITINGRESTART_FIRST_RUN
follow TEST_WAITINGRESTART_KMSHELL_KEYMAN_HAS_RUN steps 1-14
15. At the command prompt again
type kmshell.exe -firstrun=StartWithWindows,CheckForUpdates,AutomaticallyReportUsage,InstallDefaults, -defaultuilanguage=en
16. In the Regedit window verify the
update state
has goes tousIdle
18. "C:\Users"yourusername"\AppData\Local\Keyman\UpdateCache" folder. is now empty
Installing
For installing state we will basically just set the state to usInstalling
INSTALLING_STATE_SETUP_STEPS
update state
found at (Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine).If this currently the latest version of Keyman then the update state will end up back at usIdle.
Therefore, if not already
usInstalling
set the state tousInstalling
andmanually copy this file to "C:\Users"yourusername"\AppData\Local\Keyman\UpdateCache"
cache.json
TEST_INSTALLING_KMSHELL_SWITCHES
Note this is also ## TEST_INSTALLING_CONFIGSWITCH
Follow all steps in INSTALLING_STATE_SETUP_STEPS
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -c
update state
has goes tousIdle
TEST_INSTALLING_KMSHELL_INSTALL_TIP
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -install-tip 0C09 "tibetan_direct_input" "en-AU" ""
update state
stays atusInstalling
TEST_INSTALLING_REQCHECK
Follow all steps in INSTALLING_STATE_SETUP_STEPS
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -buc
(will be ignored)update state
stays atusInstalling
TEST_INSTALLING_APPLY_NOW
Follow all steps in INSTALLING_STATE_SETUP_STEPS
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -an
(will be ignored)update state
stays atusInstalling
TEST_INSTALLING_DOWNLOAD
Follow all steps in INSTALLING_STATE_SETUP_STEPS
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -bd
(will be ignored)update state
stays atusInstalling
TEST_INSTALLING_FIRST_RUN
Follow all steps in INSTALLING_STATE_SETUP_STEPS
cd "c:\Program Files (x86)\Keyman\Keyman Desktop"
kmshell.exe -firstrun=StartWithWindows,CheckForUpdates,AutomaticallyReportUsage,InstallDefaults, -defaultuilanguage=en
update state
stays atusIdle