[Bug]: 使用自定义 OpenAI API 时，流式输出的最后一条会被丢弃，导致回答不完整

[jwang-paradise]

这个bug是否已存在现有issue了？

• 我确认没有已有issue，且已阅读常见问题。

错误表现

Bug现象

流式输出末尾不完整

控制台log

2024-09-24 10:40:59,532 [DEBUG] [OpenAIVision.py:160] [{'role': 'user', 'content': 'hi'}]
2024-09-24 10:41:00,515 [INFO] [base_model.py:703] 回答为：Hello! ... . How can I assis

末尾 assis 被截断。

流式 response 内容

我手动在OpenAIVisionClient._decode_chat_response() #L238 添加了 log:
logging.warning(f"***** {chunk['choices'][0]}")

2024-09-24 10:41:00,432 [DEBUG] [OpenAIVision.py:246] ***** {'index': 0, 'delta': {'content': 'How '}, 'finish_reason': None}
2024-09-24 10:41:00,448 [DEBUG] [OpenAIVision.py:246] ***** {'index': 0, 'delta': {'content': 'can '}, 'finish_reason': None}
2024-09-24 10:41:00,469 [DEBUG] [OpenAIVision.py:246] ***** {'index': 0, 'delta': {'content': 'I assi'}, 'finish_reason': None}
2024-09-24 10:41:00,482 [DEBUG] [OpenAIVision.py:246] ***** {'index': 0, 'delta': {'content': 's'}, 'finish_reason': None}
2024-09-24 10:41:00,515 [DEBUG] [OpenAIVision.py:246] ***** {'index': 0, 'delta': {'content': 't you today?'}, 'finish_reason': 'stop'}

可见，最后一条（包含 'finish_reason': 'stop' 的一条）response 的 delta.content 丢失了。

虽然 OpenAI 官方API中可能不会遇到此问题，但是建议可以考虑此情况，加强对 API 的兼容性

解决办法

我暂时的解决方法，在 OpenAIVision.py#L251 添加判断，如果有 content:

if finish_reason == "stop":
    if ("content" in chunk["choices"][0]["delta"] and chunk["choices"][0]["delta"]["content"] is not None):
        yield chunk["choices"][0]["delta"]["content"]

复现操作

1. 正常本地部署 (pip)
2. 通过 OpenAI API 连接本地大模型
3. 随意提问

错误日志

No response

运行环境

• OS: Windows 10
• Browser: /
• Gradio version: /
• Python version: 3.11

帮助解决

• 我愿意协助解决！

补充说明

No response

[hyyz17200]

我也有类似的问题

Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response Oct 21 22:23:33 python[493862]: finish_reason = chunk["finish_details"] Oct 21 22:23:33 python[493862]: ~~~~~^^^^^^^^^^^^^^^^^^ Oct 21 22:23:33 python[493862]: KeyError: 'finish_details' Oct 21 22:23:33 python[493862]: Traceback (most recent call last): Oct 21 22:23:33 python[493862]: File "/media/mydata/ChuanhuChatGPT/modules/models/OpenAIVision.py", line 250, in _decode_chat_response