[DOC] Add Playwright tests to grab relevant screenshots

[pavithraes]

Also, make sure we have descriptive image names

[kcpevey]

I'll organize the images into the directory structure {conda-store/conda-store-ui}/{diataxis section}/{descriptive filename}

[kcpevey]

playwright already exists and has a very basic integration test.

The playwright tests for the ui exist on the conda-store repo. This has some implications:

• The way that the conda-store tests are set up, it will pull the latest release of conda-store-ui to test on. Which means that the screen shots will not be updated until conda-store-ui has a release, and the release number has been bumped in the conda-store repo as well.
• There are no integration tests for conda-store-ui main or PRs. We are only aware of a failure once its been released and integrated at the conda-store level.

This leaves me with the questions:
Are we ok with the delay in screen shots updating? Or should the screen shot update happen on the conda-store-ui side (maybe with an auto-push to this repo?). In other words - what should trigger re-rendering of screen shots? it should be somewhat manual since they are binary and we don't want to overload the repo.

@pavithraes ?

[kcpevey]

The playwright test setup has been merged. This is now unblocked.

Where should the screenshots be stored? The documentation is in conda-store repo, but these images will be generated on the conda-store-ui repo. The easiest option is just to save them as artifacts and manually push them to the docs repo.

[costrouc]

Just noting here that the playwright test is flaky @dcmcand will open an issue.

[pavithraes]

Update - The flaky test is fixed, ref: conda-incubator/conda-store-ui#326, conda-incubator/conda-store-ui#321

[kcpevey]

• the workflow should be triggered manually or by commenting /regenerate images or something
• the workflow to generate the images should happen on the ui repo and store images as artifacts.
• The workflow will trigger opening a PR on the code repo

[kcpevey]

Still working on this. I had to create some test repos as a playground because some of the action triggers only work on main and I don't want to debug in the main repos.

I looked at the issue_comment trigger which is what we'd need to be able to run the "push to conda-store as PR" workflow. GH doesn't have a more tightly scoped pr_comment trigger, which means that EVERY comment on EVERY issue and PR in this repo will trigger this workflow. I believe that's a lot of wasted compute even if the job is "if pr event", its still spinning up machines. For that reason, my plan is a manual trigger or a trigger by label.

[trallard]

This comes from the fact that PRs are bundled under issues (so by proxy PR comments are also issue comments).

I looked at the issue_comment trigger, which is what we'd need to be able to run the "push to conda-store as PR" workflow. GH doesn't have a more tightly scoped pr_comment trigger, which means that EVERY comment on EVERY issue and PR in this repo will trigger this workflow.

This does not seem right - for example, from the action I shared only when we call it with bot, please update galata snapshots (ref: jupyterlab/jupyterlab#15386 (comment)) is when the workflow is triggered. So basically we should be checking for a specific comment and ensure that only that string triggers the workflow.

[kcpevey]

This does not seem right

I did code it that way, but then when I was reading the docs I guess I just misunderstood it so I reverted. I can switch it back to triggering on issue_comment.

The issue will still remain that the playwright test workflow will have to have already completed before this new workflow can be triggered. I couldn't find a way around that, but if you have a suggestion there, I'm all ears!

[trallard]

This has been open for so long and @gabalafou has already raised concerns about longer term toil and maintenance for this feature (with which I agree).

I am tempted to close this issue and the associated PR soon unless I hear compelling arguments against it and a mitigation plan for the maintenance burden and potential disruption to contribution workflows.

[soapy1]

I am tempted to close this issue and the associated PR soon unless I hear compelling arguments against it and a mitigation plan for the maintenance burden and potential disruption to contribution workflows.

+1 to closing the issue

[gabalafou]

The corresponding PR is already closed, so I'm closing this issue.