Ability to bulk-delete old incomplete submissions

[pmangahis]

Describe the problem you would like to solve
We've received 2 requests from hosted clients to have a multi-select option to remove incomplete submissions.

Describe the solution you'd like
Currently, JMs need to delete incomplete submissions from authors and QuickSubmit plugin.

Who is asking for this feature?
Journal Editors, Journal Manager

[NateWr]

Thanks @pmangahis. Can you say a little bit more about the use-case and the steps that editors are taking? I assume that editors/managers are identifying old incomplete submissions in some way before deciding what to remove. How do they determine which ones to remove? Why do they feel they need to remove them?

[pmangahis]

In one case the journal was doing a manual import from BePress using the QuickSubmit plugin. I think when there is an error during the submission process using QuickSubmit it still creates a record. They want to be able to remove duplicates.

In the other journal, it seems that the journal has multiple pages (over 10) of incomplete submissions from what I can tell were the authors created that the JM would like to remove.

[NateWr]

Thanks! I think the QuickSubmit issue is probably something that we should address at its source. With the incomplete submissions, do you know if these were done by a single malicious user or spam bot? Or had they just accumulated over many years?

[pmangahis]

I think it might be both cases for the other journal.

[carolinatanigushi]

We also have this issue on our installations.

Journals have previously let us know that there are a lot of abandoned submissions that they need to manually delete. It's an issue on our OPS installation as well, considering that out of our 357 currently active submissions, 239 have not been touched on the last six months.

[jnugent]

Just to add 0.02 to this from the systems side. We've been able to clean out incomplete submissions by using the following logic:

1. We identify old and stale incomplete submissions by looking at the submissions table and querying based on submission_progress > 0 and a date_last_activity that is older than a determined threshold with the client.
2. These submission ids are put into a text file and iterated over on the command line in a shell script and passed to the tools/deleteSubmissions.php tool.
3. In 3.3, before we can do this we need to comment out the code in the submission file service that logs file deletions because of this issue File logging still occurs when deleting a submission or context #7597

But this does seem to work for us. We've a few clients that are very old and have a long history with OJS 2 and now have upwards of a thousand incomplete submissions accumulated over the years, so it's nice to be able to clean these out.

Jason

[librariam]

In addition to Jason's script suggestion above, for a use case with a smaller number of incomplete submissions - I wonder if it would make sense to bring back the old submission "Delete" button for admin accounts only (before a submission is declined)? I understand the reason for removing it for JE/JM accounts, but having an admin delete submissions quickly would be very helpful.

[Devika008]

Hello,

Here are the designs for the the issue! The considerations taken for the designs were:

• Adding a checkbox next to each incomplete submission in the submission list.
• Adding a checkbox to select all checkboxes from Step 1.
• Adding a delete button to delete all selected items.
• Adding a confirmation modal

Also I do like @librariam 's suggestion of only including this option for the admin only

3.4

On selecting a checkbox, the delete button becomes enabled

You can select and deselect all after a checkbox has been selected

This needs to be an entirely intentional action and hence an extra layer of confirmation is necessary

3.5
For the 3.5 submission table, I confirmed with Israel that placing the button before, like in Gmail, is more intuitive. To make the table more accessible, the column heading can be visually hidden but still made available to screen reader users. When using a keyboard, there will be an "on focus" state. Developers can add an aria-label or a with a class to hide it. The text to be announced by the element should be "Select/deselect the submission." I think the suggestion fits extremely well with the current proposed submission lists dashboard

A confirmation message similar to 3.4 needs to be shown here as well.

[taslangraham]

Thanks @Devika008

[taslangraham]

Ready for review.

PRs

pkp-lib (main)
This introduces a new endpoint that accepts a list of submission IDs via a query param, performs the necessary validations, and deletes the corresponding submissions.

• pr: Allow bulk deletion of Incomplete Submissions #10258. @asmecher @ewhanson @jonasraoni

UI library (main)
Please see demo here

• pr: Allow bulk deletion of Incomplete Submissions ui-library#390. @jardakotesovec

submodule updates

• ojs: submodule update ojs#4387
• omp: submodule update omp#1665
• ops: submodule update ops#744

[jonasraoni]

@taslangraham I think you're tagging the wrong Erik, it should be @ewhanson 😁

[taslangraham]

@taslangraham I think you're tagging the wrong Erik, it should be @ewhanson 😁

ha! you're right. thanks

[jardakotesovec]

@taslangraham Hi Taslan,
this unfortunately needs to go on hold for some time, at least the UI part.

Thing is that for version 3.5 (main branch) - the submission list panel is being replaced with new UI, which is not using ListPanel anymore (#7495). At this addition can be considered once the new UI is more settled. Therefore the changes you introduced to submission list panel would work only for version 3.4 - but at this point we are not adding new functionality to 3.4 as we need to focus on finishing things off for 3.5.

Sorry about that, this is a mistake on our part.

[jardakotesovec]

@taslangraham Thanks, ui-library also merged.

@Tribunal33 Would you mind do some QAing on this one?

To summarise what we have intended to implement, feature is ability to delete incomplete submissions:

• as author I can delete my incomplete submissions
• as admin or journal manager I can delete incomplete submissions in the journal I have such roles

[Tribunal33]

Taking a look now @jardakotesovec and @taslangraham and will write up some test cases for it.

[Tribunal33]

This looks good. @taslangraham can we add some scope creep? I am not seeing a convenient way to cancel the process once the "Delete Incomplete Submissions" button has been engaged. I have a solution. Treat it like a filter with incomplete stage as the option and have the filter be deletable like other filters? This would solve two problems. First one is for those journals with pages and pages of submissions with intersparsed incompletes submissions it would bring them all to the top of the list. Second, there is already a clear filter button that can be applied to remove the filter. Here is a mockup of what I'm talking about. Let me know if there are any questions.

[jardakotesovec]

@Tribunal33 Or 'Cancel' button can show up next to Delete Incomplete Submissions, which was my suggestion. But I think its minor concern as its infrequent action - thats probably why @Devika008 kept it simple. But I will let her respond to it.

[Tribunal33]

After talking with @Devika008 and @taslangraham we will go with an additional Cancel button beside the Delete Incomplete Submissions button to reset this action. We will ignore the filtering idea as that breaks a PKP rule for filters. I believe Devika has upgrade the Figma Files to reflect this change :

[jardakotesovec]

@Tribunal33 Oki, updated on main. Feel free to close this one if you don't see any other issue here.

[Tribunal33]

@taslangraham and @jardakotesovec I am sorry to do this to ya but there is one more issue. If you have incomplete submissions on multiple pages that have been selected and click on the delete incomplete submissions button it only deletes from the page that you initialially selected. To test this:
precondition : you will need to create enough incomplete submissions to create another page

1. Then try initially starting on page 1 and selecting the three dots
2. Select Delete Incomplete submissions
3. Select an incomplete submission from page 1
4. Select Next Page
5. Select an incomplete submission from page 2
6. Select Delete Incomplete submissions button

Expected outcome: both incomplete submissions should have been removed from the filter list
Actual outcome: only page 1 (or initial pages) incomplete submissions will be removed.

[jardakotesovec]

Right.. from quick check the issue is not pagination. But the fact that our useFetch (which is using ofetch on background) is encoding array query param as /_submissions?ids=20&ids=21, which is not PHP backend friendly.. as it needs to be I think /_submissions?ids[]=20&ids[]=21. So we will need to adjust useFetch to encode arrays in php friendly way..

So I think the behaviour is that it just deletes the last item on that list..

Will address it after holidays.. no need for additional ticket.

[jardakotesovec]

Seems that they intend to address it in future versions by leveraging URLSearchParams

unjs/ufo#185
unjs/ufo#208
unjs/ofetch#440

@taslangraham Could you please try to find some workaround until this is resolved upstream? Ideally to address it somewhere within useFetch.js composable.

[taslangraham]

@jardakotesovec I'll take a look

[taslangraham]

@jardakotesovec I added a PR with the workaround here - pkp/ui-library#486. Please take a look

Approach I took was to rename the array query params to include [] before passing to ofetchInstance. So a query entry like {ids: [1,2,3]} would become {'ids[]':[1,2,3]}

[taslangraham]

hmm, tests are failing. Looking into it

[taslangraham]

hmm, tests are failing. Looking into it

fixed

[jardakotesovec]

@taslangraham That looks great, thank you! Feel free to merge it. And than we can ping @Tribunal33 if he can still find something :-).

[taslangraham]

Thanks for the review @jardakotesovec. Will update the submodules then @Tribunal33 can take a run at it again

[taslangraham]

Submodules updated. @Tribunal33 you can you can do another round of tests now

[Tribunal33]

@taslangraham and @jardakotesovec this passes QA. To test the remaining bug I needed to create enough incomplete submissions to see them on multiple pages in the submission list views.
Here are the High level test ideas:

1. Remove incomplete submission – single submission
2. Remove incomplete submission -- multi submission
3. Default button functionality
4. Button option visible from various roles
5. Remove incomplete submissions from different list pages
   a. Each submission list has 30 submissions before the next page of submissions