SPIKE: Respond to Platform redis key request regarding Facilities keys

[jilladams]

Description or Additional Context

Platform needs product teams to verify the status of Redis keys that are not defined with an expiration in vets-api.

Facilities owns 3 such keys, apprarently, and we need to provide a high-confidence response to Platform on safety of expiring the keys or removing them altogether.

https://dsva.slack.com/archives/CE4304QPK/p1727205902397569?thread_ts=1727205902.397569&cid=CE4304QPK

We have noticed some Redis keys that are not defined in the redis.yml configuration file of the vets-api repository. Additionally, these keys have a TTL (Time to Live) of -1, meaning they are persisting indefinitely in the Redis store.
We would like your assistance in identifying where these keys are being set or defined within the vets-api codebase. Moreover, we are trying to understand why the TTL for these keys is set to -1, as this could lead to unnecessary persistence of data in Redis.
Could you please provide guidance on where these keys are located and the reasoning behind the TTL configuration?

Facilities related keys

1. facility-access-satisfaction
2. facility_dental_service
3. facility_mental_health

Patrick Vinograd provided some context around history: https://dsva.slack.com/archives/C0FQSS30V/p1727208432823589?thread_ts=1727206911.544229&cid=C0FQSS30V

In an earlier iteration of the facility locator (before LH provided an upstream API and when VA.gov was still etching facility data from an ArcGIS endpoint), we also merged in data from https://www.accesstocare.va.gov/ that was used to power the "veteran satisfaction with wait times" part of a facility detail page. We would periodically pull a bulk download from the access to care system and cache it in redis. There wasn't really any impetus to expire the data.

I believe the dental and mental health namespaces were a similar thing - the source of truth for those data points was different than that where we were getting the rest of the "specialties offered at VAMC" data, so they had to be separately fetched and merged in.

While the dental/mental health and patient satisfaction data is showing on the facility detail pages, my educated guess (please due your own due diligence) is that neither one is coming from vets-api (or if it is, it's included in the LH facilities API at this point), and so those cache entries could be removed entirely.

They would have been fetched by sidekiq jobs under vets-api/app/sidekiq/facilities - you can see the access_data_download.rb, dental_service_reload_job.rb and mental_health_reload_job.rb are still in source control - are they scheduled? If not, safe to clean up the code and the corresponding redis entries. Likewise if they are scheduled but nothing is consuming the data it would be a great tech debt cleanup. And you could potentially also clean up the forward proxy connection to the access to care API endpoint.

Related ticket

• Decommission obsolete ArcGIS/facilities code  va.gov-team#76315

Tasks

• Verify the notes above with @omahane and with Lighthouse, re: access to care, dental, and mental health services
• Sort out what we have to do short term to satisfy the Platform request vs. what we need to do long-term about the tech debt

Acceptance Criteria

• Response is provided on https://dsva.slack.com/archives/CE4304QPK/p1727205902397569?thread_ts=1727205902.397569&cid=CE4304QPK
• Ticket are cut for any next steps required

[jilladams]

@Agile6MSkinner FYI I don't think we'll be able to answer the platform request without a SPIKE / talking to @omahane and LH about the data migrations (?). I queued this for refinement tomorrow, but may wanna confirm with Michelle that we need to hop right on it, or if it can wait til Sprint 15+.

cc @mmiddaugh

[jilladams]

@jilladams I think Patrick Vinograd's PR fixed this. Need to revisit the Slack thread, and confirm if this is ok now or not.

[Agile6MSkinner]

This ticket will be in Sprint 19, but to be started after code freeze begins.

[mmiddaugh]

@Agile6MSkinner just noting this spike was deprioritized during Sprints 19 and 20. Let's make sure to pick it up during Sprint 21, if practical.

[SnowboardTechie]

From everything I am seeing playing catchup on previous tickets and slack threads it appears that we have already removed the code related to these keys and should be in a good place to update the thread okaying ours be removed. I will hold on that for now though.

@omahane I'll remember to bring this up at scrum tomorrow, we might be able to fit this in 16th minute. I would like to connect with you to verify I am not missing some important context here.

[SnowboardTechie]

16th minute led to discovering @eselkin is likely the best person to confirm my thoughts with here. @eselkin do you have any concerns or things I should be aware of with my thoughts above?

[eselkin]

Yes. The code is gone. The keys are all that remain and any data associated with the permanent (as permanent as memory is) keys.

Nothing should be updating values of those keys.

[SnowboardTechie]

Perfect, that lines up with my findings and leaves me very confident that we can pass along to platform that these keys can be safely wiped from Redis.

[SnowboardTechie]

I have provided the necessary update clearing platform to remove these keys here

Given the code related is already gone and we are unable to remove the keys without Platform, I don't see a need to cut any further tickets for this work.