Listing calls & URL size limit of OpenML server

[janvanrijn]

The OpenML server has a URL size limit, which prevents us from doing listing calls with too many filters. For example, when I want to filter on 1000 task id's it is likely that the call will fail. It would be great if we can somehow automatically detect and catch this in the list all function. However, this is a though problem, as sometimes there are multiple filters (e.g., per task and per setup)

Extending this limit is not a legitimate fix, as the problem will re-occur when using bigger filters.

[janvanrijn]

Just realized that there is a straightforward fix for this, i.e., allowing also post variables for the listing functions (although POST means authentication)

@mfeurer what do you think?

[mfeurer]

Is the maximum URL length documented somewhere? Also, what's the server response? If the server response is something we can parse/display nicely, is there a reason to act upon this in addition?

Also, how would you handle this except raising an exception? Do you want to "parse" the query on the python side to chunk it? We could go back to post again, but we reverted that a while ago to not have to authenticate, so I don't think that's a great idea. Maybe we can catch a server exception 'URL too long' and then try a post request?

[janvanrijn]

I think this issue will mostly occur with setups, as these can not be filtered by other means (tasks, runs, etc)

I added a function to the openml contrib library:
https://github.com/openml/openml-python-contrib/blob/master/openmlcontrib/setups/functions.py

[mfeurer]

This can be reproduced with:

import openml

openml.datasets.list_datasets(
    data_id=list(range(10000))
)

@janvanrijn we can add a helper to either call the get or post request depending on the length of the URI. However, to do so we need to know the maximal length the server allows.