feat: add question generator with llamaindex #193
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
ColabDog
reviewed
Oct 9, 2023
| Args: | ||
| num_questions (int): The number of questions to be generated. | ||
| directory_path (str): The path to the directory containing the document. | ||
| bad_questions (list, optional): A list of custom 'bad' questions to be appended to the generated questions. |
There was a problem hiding this comment.
Thanks for the PR!
For bad_questions, I feel like could be renamed to 'additional_questions' as I can see users supplying additional specific good or bad questions unless there's a a plan for generator subclasses to use 'bad_questions' in the future.
ColabDog
reviewed
Oct 10, 2023
ColabDog
reviewed
Oct 10, 2023
| @@ -0,0 +1,68 @@ | |||
| import os | |||
| from llama_index import SimpleDirectoryReader, ServiceContext | |||
There was a problem hiding this comment.
To manage complex external dependencies, can you please move these inside the QuestionGenerator abstraction?
|
Following up - this PR is good to merge in! On my side, I will need to restructure the tests to make them all pass and fix the infrastructure. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces the QuestionGenerator class that leverages the llama_index library to automatically generate questions from a given document. This enhancement aims to streamline the question generation process by utilizing the capabilities of the llama_index and providing an easy-to-use interface for generating questions, including custom 'bad' questions if needed.
Changes
QuestionGenerator Class:
Issue Addressed:
This PR addresses issue #191