Implement pagination with 'continue' to load all linked pages

[bhushan354]

NOTE: Please review the pull request process before opening your first PR: https://github.com/WikiEducationFoundation/WikiEduDashboard/blob/master/CONTRIBUTING.md#pull-request-process

What this PR does

fixes #5588

Screenshots

Before:

After:
< add a screenshot of the UI after your change >

Open questions and concerns

< anything you learned that you want to share, or questions you're wondering about related to this PR >

[bhushan354]

• only have one slug editing-basics but this is the message i am getting.

also i have a few questions regarding the Wiki API and pagination:

1 - does the Wiki API always return a continue key when there are more than 500 linked pages? If not, how is pagination handled? This PR assumes that theres an api key continue.
2 - i used Postman to check for the presence of a continue key in the API response but was not able to get the output. how can I better inspect the API response or debug it? Are there any tools or methods you'd recommend for troubleshooting?
3 - Can you suggest any improvements for error handling or pagination logic in the current implementation?

Thanks !

[bhushan354]

@ragesoss can you review this ?

[ragesoss]

Thanks for the ping. It's on my todo list.

[ragesoss]

You'll definitely want to implement a test for this. I think the API always includes a continue param of some sort, if not all the results in included in the response. The limit won't necessarily be 500; that depends on a mediawiki setting that might change, as well as the identify of the client making the request (as making a request while logged in with some accounts will allow for larger limits). But I think 500 is the current limit for a logged-0ut request (which is what the dashboard uses).

I think the slides index currently has more than 500 links, so you can write test around that: https://meta.wikimedia.org/wiki/Training_modules/dashboard/slides

I suggest making a dedicated method to do the link fetching, so that you can test that in isolation, without needing to do anything with training records in the test.