feat: support Nova Pro

[NingLu]

Fixes #

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request introduces several improvements and bug fixes to the codebase. The primary focus is on enhancing the functionality of the LangChain integration, which includes updates to various chains, chat models, and tools. Additionally, there are modifications to the source code for the online lambda function, the model ETL process, and the chatbot user interface.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 19, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/lambda/online/common_logic/common_utils/constant.py	2 added, 0 removed	The code change adds two new constants, NOVA_PRO (commented out) and NOVA_PRO (with a different value), to the LLMModelType class.
source/portal/src/pages/chatbot/components/Message.css	11 added, 0 removed	The code changes introduce styles for markdown table images with centered alignment and object-fit, and responsive markdown images with maximum width and auto height.
source/lambda/online/common_logic/langchain_integration/chat_models/init.py	3 added, 1 removed	This code change imports the NovaPro model from the bedrock_models module and adds support for loading it in the _load_module function.
source/lambda/online/common_logic/langchain_integration/chains/conversation_summary_chain.py	4 added, 0 removed	The code adds two new classes, GLM4Chat9BConversationSummaryChain and NovaProConversationSummaryChain, which inherit from Claude2ConversationSummaryChain and set different model IDs.
source/lambda/online/common_logic/common_utils/response_utils.py	5 added, 3 removed	The code changes modify the handling of figures in the stream_response function, removing the figure variable and instead extracting it from the extra_response dictionary, and conditionally including a truncated version of the first figure in the ddb_additional_kwargs of the context message.
source/lambda/online/common_logic/langchain_integration/chains/chat_chain.py	4 added, 0 removed	The code changes introduce two new classes, ChatGPT4oChatChain and NovaProChatChain, which extend existing chat chain classes and specify different language model types.
source/lambda/online/common_logic/langchain_integration/chains/rag_chain.py	5 added, 0 removed	The code changes introduce a new class NovaProRAGLLMChain that extends Claude2RagLLMChain and sets the model_id to LLMModelType.NOVA_PRO.
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_api.py	4 added, 0 removed	The code adds two new classes, CohereCommandRPlusToolCallingChain and NovaProToolCallingChain, which extend the Claude2ToolCallingChain class and specify different language model types.
source/lambda/online/common_logic/langchain_integration/chains/hyde_chain.py	4 added, 0 removed	The code adds two new classes, Internlm2Chat20BHydeChain and NovaProHydeChain, which extend existing classes and specify model IDs for LLM models.
source/lambda/online/common_logic/langchain_integration/chains/query_rewrite_chain.py	6 added, 1 removed	This code change introduces new classes for different language models (Claude3.5 Sonnet, Claude3.5 SonnetV2, InternLM2Chat20B, NovaPro) to be used in a query rewrite chain, and fixes a typo in the Claude3.5Sonnet class.
source/model/etl/code/sm_predictor.py	12 added, 9 removed	The code changes involve importing modules in a different order, adding lambda_return function, handling invalid request bodies, and exposing two Flask routes: /ping for health check and /invocations for processing PDF files from an S3 bucket.
source/portal/src/pages/chatbot/components/Message.tsx	9 added, 1 removed	The code changes add a custom component for rendering images in Markdown with a maximum width of 150px and auto height, while maintaining existing custom components for headings, tables, and table cells.
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_claude_xml.py	5 added, 0 removed	The code adds two new classes, Claude35SonnetToolCallingChain and NovaProToolCallingChain, which inherit from Claude2ToolCallingChain and specify different model IDs.
source/portal/src/utils/const.ts	1 added, 0 removed	The code change adds a new model 'us.amazon.nova-pro-v1:0' to the LLM_BOT_COMMON_MODEL_LIST array.
source/lambda/online/common_logic/common_utils/prompt_utils.py	56 added, 12 removed	This code change updates the list of supported LLM model types, adds a new response rule for handling images in the RAG prompt template, modifies the formatting and instructions for the RAG and conversation summary prompt templates, and includes the new NOVA_PRO model in various prompt template registrations for different task types like RAG, conversation summary, and tool-calling API.
source/lambda/online/common_logic/langchain_integration/chains/init.py	13 added, 6 removed	The code changes add support for NovaProChatChain, NovaProConversationSummaryChain, NovaProRAGLLMChain, NovaProHydeChain, NovaProQueryRewriteChain, and NovaProToolCallingChain across various import functions.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/init.py	28 added, 27 removed	The code changes involve modifying the PythonREPL tool from LangChain to adapt to the AWS Lambda execution environment, where /dev/shm is not available. It also includes error handling for when the tool does not produce a result.
source/lambda/online/common_logic/langchain_integration/chat_models/bedrock_models.py	48 added, 0 removed	This code adds a new class called NovaPro that inherits from Claude2. It sets the model_id to LLMModelType.NOVA_PRO and disables enable_auto_tool_choice and enable_prefill. Additionally, there is a commented-out create_model method that sets up the AWS Bedrock inference profile for the NovaPro model.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/rag.py	51 added, 7 removed	This code change introduces several new features and modifications, including:

1. Logging setup for better debugging.
2. A commented-out function llm_stream_helper for processing LLM output and extracting references and associated documents/figures.
3. Retrieval of documents from the RAG tool and storing them in the state.
4. Removal of duplicate figure handling.
5. Retrieval of LLM config and prompt templates from DynamoDB based on group, model, task type, and chatbot ID.
6. Invocation of the LLMChain with the retrieved context and prompt. |

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request introduces improvements and bug fixes to the language model integration and the user interface components of the chatbot application. The changes include updates to the LangChain integration, which handles the conversation flow, query rewriting, and response generation. Additionally, modifications have been made to the front-end components responsible for displaying messages in the chatbot interface.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 19, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/lambda/online/common_logic/langchain_integration/chains/rag_chain.py	5 added, 0 removed	The code adds a new class NovaProRAGLLMChain that extends Claude2RagLLMChain and sets the model_id to LLMModelType.NOVA_PRO.
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_claude_xml.py	5 added, 0 removed	The code adds two new classes, Claude35SonnetToolCallingChain and NovaProToolCallingChain, which inherit from Claude2ToolCallingChain and define different model IDs.
source/lambda/online/common_logic/langchain_integration/chains/conversation_summary_chain.py	4 added, 0 removed	The code adds two new classes, GLM4Chat9BConversationSummaryChain and NovaProConversationSummaryChain, which inherit from Claude2ConversationSummaryChain and specify different LLM model types.
source/lambda/online/common_logic/common_utils/constant.py	2 added, 0 removed	The code changes add two new model types, NOVA_PRO, to the LLMModelType class, with one line commented out.
source/lambda/online/common_logic/langchain_integration/chains/chat_chain.py	4 added, 0 removed	The code adds two new classes, ChatGPT4oChatChain and NovaProChatChain, which extend existing chat chain classes for different language models, likely ChatGPT-4 and Anthropic's Nova Pro.
source/lambda/online/common_logic/langchain_integration/chains/hyde_chain.py	4 added, 0 removed	The code adds two new classes, Internlm2Chat20BHydeChain and NovaProHydeChain, which inherit from existing classes and specify different model IDs.
source/lambda/online/common_logic/common_utils/response_utils.py	5 added, 3 removed	This code change removes the figure variable assignment, adds a ddb_additional_kwargs dictionary to the context message, and conditionally assigns a truncated reference figure to ddb_additional_kwargs["figure"] if available.
source/lambda/online/common_logic/langchain_integration/chains/query_rewrite_chain.py	6 added, 1 removed	This code change introduces new classes for different language models (Claude 3.5 Sonnet, Claude 3.5 Sonnet V2, Intern LM 2 Chat 20B, and Nova Pro) to handle query rewriting, extending the existing Claude2QueryRewriteChain class.
source/lambda/online/common_logic/langchain_integration/chat_models/bedrock_models.py	11 added, 3 removed	The code changes add support for using AWS credentials from environment variables, add a new LLM model class called NovaPro that inherits from Claude2 with specific configurations, and make formatting changes.
source/portal/src/pages/chatbot/components/Message.tsx	9 added, 1 removed	The code changes add a custom component for rendering images in Markdown with a maximum width of 150px and automatic height adjustment, while also applying custom styles to various HTML elements rendered by the ReactMarkdown component.
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_api.py	4 added, 0 removed	The code introduces two new classes, CohereCommandRPlusToolCallingChain and NovaProToolCallingChain, which extend Claude2ToolCallingChain and specify different LLM model types.
source/lambda/online/common_logic/langchain_integration/chains/init.py	13 added, 6 removed	The code changes add support for a new language model called "NovaProChatChain" across various chains like chat, conversation summary, RAG, Hyde, query rewrite, and tool calling chains.
source/lambda/online/common_logic/langchain_integration/chat_models/init.py	3 added, 1 removed	The code changes add support for importing the NovaPro model from the bedrock_models module and loading it using the LLMModelType.NOVA_PRO identifier.
source/lambda/online/common_logic/common_utils/prompt_utils.py	56 added, 12 removed	The code changes provide updated guidelines and formatting rules for the AI assistant's responses when using the RAG (Retrieval-Augmented Generation) and conversation summary templates. It also adds support for the NOVA_PRO model type across various task types.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/init.py	28 added, 27 removed	This code change adds new tools to the ToolManager for getting weather, handling missing parameters, providing final responses, casual chat, retrieving information from a knowledge base, and executing Python code within a REPL environment.
source/portal/src/utils/const.ts	1 added, 0 removed	The code change adds a new model 'us.amazon.nova-pro-v1:0' to the LLM_BOT_COMMON_MODEL_LIST array and creates a new LLM_BOT_RETAIL_MODEL_LIST array.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/rag.py	51 added, 7 removed	This code change adds logging functionality, defines a helper function to process references and figures from the LLM output stream, updates the retrieval of LLM configuration and prompt templates, and modifies the handling of duplicate figures. It also comments out some code related to removing duplicate figures and streaming the LLM output.
source/portal/src/pages/chatbot/components/Message.css	11 added, 0 removed	The code changes introduce styles for rendering images within Markdown tables, ensuring they fit properly, and for scaling regular Markdown images while maintaining aspect ratio.
source/model/etl/code/sm_predictor.py	12 added, 9 removed	The code changes include importing lambda_return from aikits_utils and process_pdf_pipeline from main, reordering imports, adding whitespace for readability, and handling invalid request bodies in the handler function.

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request includes several updates and improvements to the codebase, primarily focused on enhancing the functionality of the LangChain integration and the chatbot interface. The changes span multiple files and directories, addressing various aspects of the project.

Key updates include:

1. Modifications to the LangChain integration, specifically the chains, chat models, and tools. These changes aim to improve the overall performance, efficiency, and functionality of the language model integration.

2. Updates to the chatbot interface components, such as Message.css and Message.tsx. These modifications likely enhance the user experience and visual representation of the chatbot.

3. Changes to utility files like constant.py, prompt_utils.py, and response_utils.py, which may involve improvements to constant definitions, prompt handling, and response formatting.

4. Modifications to the sm_predictor.py file, which could potentially impact the model prediction functionality.

5. Updates to the const.ts file in the portal directory, suggesting changes to constant values or configurations used throughout the application.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 20, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_api.py	4 added, 0 removed	The code adds two new classes, CohereCommandRPlusToolCallingChain and NovaProToolCallingChain, which inherit from Claude2ToolCallingChain and specify different language model types.
source/portal/src/utils/const.ts	1 added, 0 removed	The code change adds a new model 'us.amazon.nova-pro-v1:0' to the list of common models available for LLM bots.
source/lambda/online/common_logic/langchain_integration/chains/tool_calling_chain_claude_xml.py	5 added, 0 removed	The code adds new classes for using different AI models: Claude3HaikuToolCallingChain, Claude35SonnetToolCallingChain, and NovaProToolCallingChain, each with a specific model ID.
source/lambda/online/common_logic/langchain_integration/chains/query_rewrite_chain.py	6 added, 1 removed	This code change introduces new classes for different language models (Claude3.5Sonnet, Claude3.5SonnetV2, Internlm2Chat20B, and NovaPro) to handle query rewriting, extending the existing Claude2QueryRewriteChain class.
source/lambda/online/common_logic/langchain_integration/chains/hyde_chain.py	4 added, 0 removed	The code changes introduce two new classes: Internlm2Chat20BHydeChain and NovaProHydeChain, which inherit from existing classes and define specific model types.
source/portal/src/pages/chatbot/components/Message.css	11 added, 0 removed	The code changes add styles for displaying images within Markdown tables and regular Markdown content, ensuring proper sizing and alignment.
source/lambda/online/common_logic/langchain_integration/chat_models/init.py	3 added, 1 removed	The code changes import a new language model called NovaPro from the bedrock_models module and add support for loading it in the _load_module function.
source/lambda/online/common_logic/common_utils/constant.py	2 added, 0 removed	The code change adds two new constants, NOVA_PRO and NOVA_PRO with a different value, to the LLMModelType class.
source/lambda/online/common_logic/langchain_integration/chains/rag_chain.py	5 added, 0 removed	The code adds a new class NovaProRAGLLMChain that extends Claude2RagLLMChain and sets the model_id attribute to LLMModelType.NOVA_PRO.
source/portal/src/pages/chatbot/components/Message.tsx	9 added, 1 removed	The code changes add a custom component for rendering images in Markdown with a maximum width of 150px and automatic height adjustment, while maintaining existing custom styles for headings, tables, and table cells.
source/lambda/online/common_logic/common_utils/response_utils.py	5 added, 3 removed	This code change removes the figure extraction from the response dictionary and instead extracts the first two characters of the first element from the 'ref_figures' list in the 'extra_response' dictionary, assigning it to the 'ddb_additional_kwargs' dictionary under the 'figure' key.
source/model/etl/code/sm_predictor.py	12 added, 9 removed	The code changes involve reorganizing import statements, adding a lambda_return function from aikits_utils, restructuring the handler function to handle missing or invalid parameters, and adding Flask routes for /ping and /invocations endpoints to handle GET and POST requests respectively.
source/lambda/online/common_logic/langchain_integration/chat_models/bedrock_models.py	11 added, 3 removed	The code changes add support for using AWS credentials from environment variables, add a new LLM model class called NovaPro with specific configurations, and make minor formatting changes.
.viperlightignore	1 added, 0 removed	This commit updates utility files, test cases, and documentation across various parts of the codebase, including the portal, lambda functions, and data preparation scripts.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/rag.py	51 added, 7 removed	This code change adds logging functionality, defines a helper function for streaming LLM responses and processing references, updates the retrieval of LLM configuration and prompt templates from DynamoDB, and comments out some code related to removing duplicate figures.
source/lambda/online/common_logic/langchain_integration/chains/conversation_summary_chain.py	4 added, 0 removed	The code adds two new classes, GLM4Chat9BConversationSummaryChain and NovaProConversationSummaryChain, which inherit from Claude2ConversationSummaryChain and specify different LLM model types.
source/lambda/online/common_logic/langchain_integration/chains/chat_chain.py	4 added, 0 removed	The provided code changes introduce two new chat chain classes, ChatGPT4oChatChain and NovaProChatChain, for utilizing the ChatGPT 4.0 and Nova Pro language models, respectively.
source/lambda/online/common_logic/langchain_integration/chains/init.py	13 added, 6 removed	The code changes import several new chains and components related to the NovaProModel, including NovaProChatChain, NovaProConversationSummaryChain, NovaProRAGLLMChain, NovaProHydeChain, NovaProQueryRewriteChain, and NovaProToolCallingChain.
source/lambda/online/common_logic/common_utils/prompt_utils.py	56 added, 12 removed	This code change adds a new LLM model type (LLMModelType.NOVA_PRO) to various lists of supported model types across different prompt templates for tasks like RAG, conversation summary, and tool-calling API. It also updates the RAG system prompt with more detailed response rules, including guidelines for handling images, language adaptation, reference citation, and formatting.
source/lambda/online/common_logic/langchain_integration/tools/common_tools/init.py	28 added, 27 removed	The code changes involve adding and modifying Python functions for loading various tools, including tools for getting weather, handling rhetorical questions, providing final responses, chatting, retrieving information from a knowledge base, and executing Python code in a REPL environment.