Create script for generating insights of AI Search performance

[miguelgrc]

Description

Requires #716, #717

Create a script (can be an actual script or some LangChain function) that will extract from the AI queries DB the rows between <start_date> and <end_date> and, using a (potentially llama3.1:8B, could be an even smaller one for this use case, we need to test it) model, extract some general insights from all the feedback received between those dates. The objective is to be able to understand how the model is performing and what users think of the responses produced.

Due to context window constraints, it might be a good idea to split it into multiple batches, and then make a final request with the summaries for all batches to generate a final, global summary. We are expecting to limit user comments to 500 characters = 504 bytes in postgres, this should help with the calculation of how many comments we can send in one request depending on the model we choose.

Work involved

Implement the script:

• Read comments from DB (allow optional filtering by positive/negative rating and allow providing the start and end dates)
• Split comment list appropriately and make one request to the LLM per batch
• Try several LLMs and choose the best one. For summarization a small one should be fine, maybe try llama3.1:8b and llama3.2:3b, or any other model if these don't perform well enough. Set a default but allow providing the model to be used.
• Output the brief, LLM-generated summary of the user's comments.

Acceptance criteria

We have a script which allows providing positive/negative rating filter, start and end dates, and model to use for summarization, and which returns some insights about the performance of the current AI Search.