Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove custom limiting on continued queries #74

Open
elifoster opened this issue Aug 30, 2019 · 1 comment
Open

Remove custom limiting on continued queries #74

elifoster opened this issue Aug 30, 2019 · 1 comment

Comments

@elifoster
Copy link
Member

elifoster commented Aug 30, 2019

For continued queries all limiting does is increase the number of requests made. We should remove the limit parameters on all continue query methods.

Example (with p response added to #post):

@mw.get_tags
{"batchcomplete"=>"", "limits"=>{"tags"=>5000}, "query"=>{"tags"=>[{"name"=>"Broken category"}, {"name"=>"Template:No summary"}, {"name"=>"abusefilter-condition-limit"}, {"name"=>"blanked page"}, {"name"=>"mobile edit"}, {"name"=>"mobile web edit"}, {"name"=>"mw-blank"}, {"name"=>"mw-changed-redirect-target"}, {"name"=>"mw-contentmodelchange"}, {"name"=>"mw-new-redirect"}, {"name"=>"mw-removed-redirect"}, {"name"=>"mw-replace"}, {"name"=>"mw-rollback"}, {"name"=>"mw-undo"}, {"name"=>"repeating characters"}, {"name"=>"visualeditor"}, {"name"=>"visualeditor-needcheck"}, {"name"=>"visualeditor-switched"}, {"name"=>"visualeditor-wikitext"}]}}
# => ["Broken category", "Template:No summary", "abusefilter-condition-limit", "blanked page", "mobile edit", "mobile web edit", "mw-blank", "mw-changed-redirect-target", "mw-contentmodelchange", "mw-new-redirect", "mw-removed-redirect", "mw-replace", "mw-rollback", "mw-undo", "repeating characters", "visualeditor", "visualeditor-needcheck", "visualeditor-switched", "visualeditor-wikitext"]

@mw.get_tags(1)
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"Template:No summary", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"Broken category"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"abusefilter-condition-limit", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"Template:No summary"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"blanked page", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"abusefilter-condition-limit"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mobile edit", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"blanked page"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mobile web edit", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mobile edit"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-blank", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mobile web edit"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-changed-redirect-target", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-blank"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-contentmodelchange", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-changed-redirect-target"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-new-redirect", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-contentmodelchange"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-removed-redirect", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-new-redirect"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-replace", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-removed-redirect"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-rollback", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-replace"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"mw-undo", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-rollback"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"repeating characters", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"mw-undo"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"visualeditor", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"repeating characters"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"visualeditor-needcheck", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"visualeditor"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"visualeditor-switched", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"visualeditor-needcheck"}]}}
{"batchcomplete"=>"", "continue"=>{"tgcontinue"=>"visualeditor-wikitext", "continue"=>"-||"}, "query"=>{"tags"=>[{"name"=>"visualeditor-switched"}]}}
{"batchcomplete"=>"", "query"=>{"tags"=>[{"name"=>"visualeditor-wikitext"}]}}
# => ["Broken category", "Template:No summary", "abusefilter-condition-limit", "blanked page", "mobile edit", "mobile web edit", "mw-blank", "mw-changed-redirect-target", "mw-contentmodelchange", "mw-new-redirect", "mw-removed-redirect", "mw-replace", "mw-rollback", "mw-undo", "repeating characters", "visualeditor", "visualeditor-needcheck", "visualeditor-switched", "visualeditor-wikitext"]

The output is the same but it submits more requests than needed. As far as I can tell, this serves no useful purpose and only complicates our code.

Proposals

I have a couple proposing, each increasing in severity of change:

  1. Remove limit parameters from all continued methods, use @query_limit_default always for continued methods
  2. Remove limit parameters from all query methods, use @query_limit_default always for all query methods
  3. Remove all limiting, use 'max' for all queries. Remove all limit parameters and @query_limit_default.

What are your thoughts, @xbony2?

@xbony2
Copy link
Member

xbony2 commented Aug 31, 2019

3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants