[Feature] Enhanced Agent in v1.3

[joshuayao]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

The umbrella feature request for all the enhancements of Agent in v1.3.

Components

• [Feature] update react_llama/rag_llama/sql_llama agents to include thought process in AIMessage.content for better streaming on UI #1183
• [Feature] improve agent's capability to process multi-turn messages and use history stored in memory/persistance #1184
• [Feature] survey code agent #1186

Examples: AgentQnA

• [Feature] AgentQnA uses assistants API and showcases multi-turn conversations GenAIExamples#1439
• [Feature] UI to support assistants API and streaming - show messages GenAIExamples#1440
• [Feature] Multimodal Agent example: PDF understanding and QnA, TAT-DQA as benchmark GenAIExamples#1442

Evaluation

• [Feature] add TAG-bench for agent_eval, publish TAG-bench results for sql agent llama GenAIEval#224
• [Feature] refactor HELMET: support OH, vllm-gaudi for agent related benchmark GenAIEval#225
• test AgentCompany benchmark: reproduce, run on gaudi