[v2] Refactor text tasks to use DataLoader

[Samoed]

Ref #1606

Now models will receive encode function Dataloader

{
   "text": [...],  # default text
   "image": [...], 
   "audio": [...], 
   "body: [...], # models are allowed to construct the text from the body + title if they wish
   "title: [...],
}

Code Quality

• Code Formatted: Format the code using make lint to maintain consistent style.

[Samoed]

Right now it is very much a quick wrapper. Wouldn’t we prefer directly working with the dataset for datasets? (I know that this is more code to write)

It's not easy because datasets have different column names and most datasets require encoding two columns, and I don’t have a clear solution for handling that. Also in most tasks list of sentences passed to evaluators and there datasets can't be used for now, but we can change that. Additionally, some datasets return a dictionary instead of a dataset, and Pair classification expects all data to be in the first row (as I recall). I could pass the dataset directly and select columns, but that would be a similar approach to using a wrapper. (edited)

[KennethEnevoldsen]

So I would really like to see how a Dataloader native abstask would look like. Can we try to do it with just Classification?

I am also afraid of how much this influences throughput - can we do a quick test e.g. using minishlab models?

It is a bit annoying that we have to convert everything in the encode functions (it might be the right solution). We could consider whether it better to just hand of the Dataset object to the model? (but I assume that does not work for images?)

[KennethEnevoldsen]

Hmm don't we want to standardize everything?

[Samoed]

We want, but I still don't know what to do with them, because we don't have implementation for any model #1330

[KennethEnevoldsen]

I pinged him. Can't we just convert it to text and keep the "conversation in a column as well??

[Samoed]

Yes, can change like that

[Samoed]

I've updated clustering and classification tasks to use Dataset more natively