feat: support image understanding from html

[IcyKallen]

Fixes #

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request includes modifications to the llm_bot_dep package, which is a dependency for the job lambda function. The changes involve updates to the figure_llm.py module, as well as the docx.py and html.py modules in the loaders subpackage.

The figure_llm.py module has been updated to improve the handling of large language model (LLM) responses and enhance the overall performance of the system. The modifications to the docx.py and html.py modules aim to enhance the loading and processing of documents in these formats, ensuring better compatibility and improved accuracy in text extraction.

Additionally, there was an issue with the llm_bot_dep-0.1.0-py3-none-any.whl package, which has been marked as an error. This issue will need to be addressed separately.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 3, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/lambda/job/dep/llm_bot_dep/loaders/html.py	0 added, 1 removed	This code change removes the print(file_content) line, which likely printed the processed file content to the console.
source/lambda/job/dep/llm_bot_dep/loaders/docx.py	0 added, 16 removed	The code changes involve removing an unused import statement for the sys module, removing a commented-out line that appends a path to sys.path, and removing a commented-out block of code that imports boto3 and calls the process_doc function with hardcoded arguments.
source/lambda/job/dep/llm_bot_dep/figure_llm.py	123 added, 32 removed	This code change adds functionality to process images in Markdown content using a language model. It includes the ability to download images from URLs, check image size, process each image with an LLM, upload processed images to S3, and replace image references with the LLM's understanding. It also adds docstrings and comments for better code readability.

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request includes changes to the llm_bot_dep package, which is a dependency for the job lambda function. The primary changes are:

1. An error occurred while installing the llm_bot_dep-0.1.0-py3-none-any.whl package, which needs to be investigated and resolved.
2. Modifications have been made to the figure_llm.py file, which likely includes bug fixes or new features related to the figure generation functionality.
3. Updates to the docx.py and html.py files in the loaders module, which handle loading and processing of DOCX and HTML files, respectively. These changes could involve bug fixes, performance improvements, or added functionality for better document processing.

The motivation behind these changes is to improve the overall functionality, reliability, and performance of the llm_bot_dep package, which is a critical dependency for the job lambda function.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 3, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/lambda/job/dep/llm_bot_dep/loaders/docx.py	0 added, 16 removed	The code changes involve removing the import of the sys module, removing a commented-out line that appends a path to sys.path, and removing a commented-out block of code that imports boto3 and calls process_doc with specific arguments.
source/lambda/job/dep/llm_bot_dep/loaders/html.py	0 added, 1 removed	The code change removes the print(file_content) line and updates the file_content variable by processing markdown images with an LLM before creating a new Document object.
source/lambda/job/dep/llm_bot_dep/figure_llm.py	123 added, 32 removed	This code change adds functionality to process images in Markdown content using an LLM (Large Language Model). It includes the following key updates:

1. Imports modules for handling images, URLs, and temporary files.
2. Adds constants for minimum image size threshold.
3. Defines a class figureUnderstand to analyze images using LLM.
4. Implements methods to invoke LLM, upload images to S3, download images from URLs.
5. Adds a function process_markdown_images_with_llm to process all images, generate understandings, upload to S3, and update Markdown content. |