Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support showing image in chat box #481

Merged
merged 3 commits into from
Dec 9, 2024
Merged

feat: support showing image in chat box #481

merged 3 commits into from
Dec 9, 2024

Conversation

NingLu
Copy link
Collaborator

@NingLu NingLu commented Dec 9, 2024

Fixes #

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request includes updates to the prompt and response handling utilities, as well as improvements to the Retrieval Augmented Generation (RAG) tool integration in the LangChain framework. Additionally, minor changes have been made to the ChatBot component in the portal application.

The changes in prompt_utils.py and response_utils.py aim to enhance the handling of user prompts and the formatting of responses, respectively. These improvements will provide a better user experience and more accurate results.

The modifications to tools/__init__.py and rag.py focus on optimizing the RAG tool integration within LangChain. This will enable more efficient retrieval and generation of relevant information, leading to improved response quality.

Finally, the updates to ChatBot.tsx in the portal application are related to the user interface and may include bug fixes or minor enhancements to the chatbot experience.

No additional dependencies are required for these changes.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

File Stats Summary

File number involved in this PR: 5, unfold to see the details:

The file changes summary is as follows:

Files
Changes
Change Summary
source/lambda/online/common_logic/common_utils/response_utils.py 2 added, 2 removed The code change limits the maximum number of reference figures included in the WebSocket message to 2, instead of potentially including all available figures.
source/lambda/online/common_logic/common_utils/prompt_utils.py 15 added, 4 removed This code update adds instructions for citing references from the provided documents, verifying the formatting of reference tags, and correcting any incorrectly formatted tags before finalizing the response.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/rag.py 65 added, 41 removed The code changes implement a function filter_response that filters out reference tags (<reference> and </reference>) from the LLM response, extracts the reference document numbers, and stores them along with the corresponding reference documents and figures in the state dictionary. It also removes duplicate figures from the list of reference figures.
source/lambda/online/common_logic/langchain_integration/tools/init.py 40 added, 60 removed The code changes involve refactoring and improving the ToolManager class for managing and registering tools in a structured way. It includes methods to convert tool definitions to Pydantic models, register tools from functions or AWS Lambda, and get registered tools. It also introduces a lazy loading mechanism for tool modules using decorators.
source/portal/src/pages/chatbot/ChatBot.tsx 8 added, 8 removed The code change enables the display of images in the AI response by iterating over the figure array in the ddb_additional_kwargs object and appending image URLs to the currentAIMessage state.

Copy link
Contributor

@IcyKallen IcyKallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IcyKallen IcyKallen merged commit e85e1a0 into dev Dec 9, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants