Support for scorer objects in model_selection optimizers

[janvanrijn]

Traceback (most recent call last):
  File "/home/vanrijn/projects/openml-pimp/openmlpimp/generatedata/run_optimizer.py", line 67, in <module>
    run = openml.runs.run_model_on_task(task, optimizer)
  File "/home/vanrijn/projects/openml-python/openml/runs/functions.py", line 36, in run_model_on_task
    flow = sklearn_to_flow(model)
  File "/home/vanrijn/projects/openml-python/openml/flows/sklearn_converter.py", line 42, in sklearn_to_flow
    rval = _serialize_model(o)
  File "/home/vanrijn/projects/openml-python/openml/flows/sklearn_converter.py", line 173, in _serialize_model
    _extract_information_from_model(model)
  File "/home/vanrijn/projects/openml-python/openml/flows/sklearn_converter.py", line 281, in _extract_information_from_model
    rval = sklearn_to_flow(v, model)
  File "/home/vanrijn/projects/openml-python/openml/flows/sklearn_converter.py", line 81, in sklearn_to_flow
    raise TypeError(o, type(o))
TypeError: (<bound method ObservableScorer.score of <openmlpimp.sklearn.beam_search.ObservableScorer object at 0x7f1b77062e48>>, <class 'method'>)

where openmlpimp.sklearn.beam_search.ObservableScorer is a plain object, but this is kind of supported by sklearn

[mfeurer]

You're trying to upload a method of an instance and not the instance itself. Shouldn't a scorer class be passed to the converter?

[amueller]

minimum example to reproduce?