-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QA test plan for: Drupal Phone Number Work #19116
Comments
@Agile6MSkinner @dsasser should this get pulled into Sprint 12? Not sure where we landed on that. I guess pending the destiny of the 3rd phone # ticket? |
@dsasser I took a pass at the How to test section, using the test cases we discussed. Feel free to edit however you want |
End of Sprint Update 9-30-24QA started today but in doing so we (Christian and I) noticed a static analysis error reported by PHPStan which I'm looking into. It appears to be a minimal required code change to remove some JetBrains class references. I expect to complete this QA work tomorrow. |
@dsasser I'm peeking at the audit report in tugboat as linked in the How to test section of the description. Most of the many, many numbers in this audit look solid! 🎉
A few other questions (and please forgive me if I'm forgetting the plan)
|
@mmiddaugh thank you for the quick review and subsequent questions. I'll do my best to respond to each, but there may be a product level answer to some of these:
Yes I believe the algorythm drops chracters that are not integers from the extension. I'm not sure there is a way to override that but I can look into it, or we can find a way to flag these odd cases and handle them manually. Interestingly, the FE is displaying the extension as 31 today.
The phone number was specified to have a standard of 10 digits only. I'm not sure of the origin but the "Toll Free: 844-501-8387" portion of the old number but it would not move to the new system in order to meet this requirement, and we would need to flag this for manual follow-up. That said, this is another case where the FE is displaying the number as specified under this ticket (ie: it is removing the toll free bits)
Yes this again was intentional to meet the requirements of this ticket, and the FE isn't displaying that text either:
That is my assumption as well but I haven't looked to be sure, but I'm sure @Agile6MSkinner could probably point us to those tickets if they exist. |
We don't have any tickets as part of this epic that would impact how the Main Number is entered or displayed. |
Only caveat is that we do have a placeholder ticket for considering whether we to ever migrate non-editable phone number fields to the Phone Number paragraph type. The VAMC facility main number is listed there as an example of such a type |
QA FindingsI found that almost all the numbers and extensions migrated as expected, with the following notable exceptions.
That these weren't picked up as valid phones makes sense to me, but we lack any logging/flagging mechanism, so we don't know every occurrence of this problem, only those that were picked up during QA. Adding logging would be a small lift and I think worth it so we understand the full scope of the problem. |
Noting from scrum: I am going to tear this to pieces. But waiting for @dsasser to give me a green light after stuff is rebuilt. I will also document my intentions before tearing it apart so we can make sure it is comprehensive @jilladams |
Missing Migrated Phone NumbersPer scrum we decided that for my QA results, we would do the following:
Nodes with non-migratable phone numbers:Staff Profile -> 38 nodes Total nodes affected = 62 Nodes Report Details
|
Nice to see that the majority of these are at least human parsable to determine intent. IMO we could do some bulk updates to clean a bunch of these up in Prod with a revision message like "Updated phone number formatting." For instance the first 10 of these are showing on prod as already clipped to the phone number, so there would be zero impact to Veterans if we chop off the trailing bits. I'm willing to go in and do some data cleanup tomorrow so we can whittle this list down to ones that will truly require editor outreach. @Agile6MSkinner @mmiddaugh let me know if I have a green light to do that |
@dsasser I assume you'll save that configuration somewhere so we can run that report again after launch? |
@dsasser definitely agree with not changing the migration. We can handle this manually and with guidance if we choose to do anything with this info. |
Notes on manual clean up@dsasser @mmiddaugh @Agile6MSkinner I have completed my manual data clean up Fixed, no follow up needed26 nodes
Possible follow up action24 nodes
Will be null after migration and that's okay!11 nodes
Needs editor outreach1 node
|
@davidmpickett According to https://en.wikipedia.org/wiki/North_American_Numbering_Plan, the Central office code, the "456" of 123-456-7890, has to start with numbers 2-9, rather than 0. |
Thanks @omahane. I updated my chart to reflect that insight. We'll definitely need the editor to fix the phone number. I wasn't able to find an obvious typo I could fix when comparing to other Cheyenne VAMC numbers |
Mid Sprint Update 10-10-24QA took a pause while we moved all the CMS work behind a feature toggle to allow for a smoother deployment in conjunction with the content-build changes. That work is in PR pending review. Once merged, it will trigger the (hopefully) final round of QA for the CMS side of things represented in this issue. |
I answered my question about what happens if there's a Draft Latest Revision with a change to the phone number before we run the migration. The migration uses the phone number from the Current Revision. This makes sense that we would give higher priority to the published content than draft content. There is also no data loss since the previous revision is still available in Drupal |
End of sprint update:
There will need to be additional QA of the FE output once those tickets are complete pending integration, but that's different from this ticket. @omahane @Agile6MSkinner @jilladams @dsasser let's discuss in 16th minute tomorrow |
As we discussed in scrum, there is no further QA that needs to happen, and we are free to move the integration PR into a mergable state, which requires code review from Facilities/PW, Platform CMS, and Platform CMS Devops (usually Platform CMS utilizes one person to fulfill both user groups. I'm getting the PR in shape for final review (removing DO NOT MERGE label, other cleanup) by our team first, and then Platform CMS once we have approvals on our end. |
This ticket is our acting artifact for getting the code merged, so: I'm gonna move it to Prod Verification just so we remember to double check things in prod after the PR merges and make sure the toggle is doing what we expect. |
You know what, no. QA is done, and we're gonna merge. THe actual original phone number tix are in COmplete Pending Integration, and those will track moving through to Prod verification > Done. Closing this. |
Testing context
Implementation ticket being tested:
#17860
#17861
#17862
What are we testing?
We are testing the migration of the data, which splits all existing phone numbers into phone and extension fields. We also looking at the editor interface to make sure that data entry meets ACs
Testing environment
Scope of changes: high level
Change from a field to a paragraph type for phone number entry in three content types: Billing & Insurance, Staff Profiles and VAMC System.
Daily Reset
Tugboat refreshes each night, and so each day the migrations need to be re-run prior to testing. Additionally, each day there is at least one change to the composer dependencies, and this makes a merge conflict with our branch. So we need engineering to "refresh" the branch each morning, followed then by someone running the below scripts:
Note: We'll need to have Platform CMS (or someone with prod shell access) execute these scripts for us when we go to prod
Running MIgrations
How to test migration
Original phone number information should be migrated following these rules:
Ext
fieldExamples:
ext
to denote extension:x
to denote extension:How to test the Content Types
User role to spoof
Content types to check
What to look for
Scenarios to test
A node that has a latest revision that differs from current revision pre-migrationAcceptance Criteria / Ownership
The text was updated successfully, but these errors were encountered: