Async streaming llamaindex openai agent example #71
Conversation
|
Hi this is great start! Thank you for your work and the PR :) I left some comments in the code for things to change, let me know if there's any questions. |
@Isaac-Flath Hi, where did you leave the comments? I do not see code reviews for some reason. |
|
@Kvit - I added them inline comments in code. Here's what it looks like when looking at the files. Let me know if you can see them! 
|
|
@Isaac-Flath , no those I cannot see |
|
@Isaac-Flath perhaps you have to do review for me to see comments >/ |
| # add Routers https://docs.fastht.ml/ref/handlers.html#apirouter | ||
| # Markdown support for daisy ui chat https://isaac-flath.github.io/website/posts/boots/FasthtmlTutorial.html | ||
|
|
||
| from fasthtml.common import Div, Span, Body, P, Form, Input, Button, Script, Link, Label, Nav, Title, Template, Style, serve |
There was a problem hiding this comment.
We can do from fasthtml.common import * to match the coding style of the other apps on the gallery.
| # =============== Router =============== | ||
| # in this example we will use the APIRouter to create a chat application and then mount it to the main app | ||
|
|
||
| r_chat = APIRouter() |
There was a problem hiding this comment.
We can define the app at the using app, rt = fast_app and use @rt as the decorator. The main advantage of API Router is it lets you import routes from other files easier in mutli-file apps. But since this is a single file app we don't need API Router.
|
|
||
| # Init global variables for OpenAI agent | ||
| agent = None | ||
| api_key_set = False # flag to check if API key is set |
There was a problem hiding this comment.
Is there any conflict with multiple users. Is it a problem if these values are set by one user and used by another? Does this need to be separated out by user or by session?
|
|
||
| Raises: | ||
| AssertionError: If msg_idx is None. | ||
| """ |
There was a problem hiding this comment.
Instead of numpy style docstrings let's use fastcore's docments. Generally we want things to match the fastai style guidelines (https://docs.fast.ai/dev/style.html) and minimize some vertical space.
| send (callable): The function to send data back to the WebSocket client. | ||
| session (dict): The session data for the current WebSocket connection. | ||
| Workflow: | ||
| 1. Checks if the OpenAI API key is set. If not, sends an error message to the client. |
There was a problem hiding this comment.
Can these workflow steps be put as inline comments right next to the thing where it's doing that instead of in a longer docstring to explain the code as it goes instead of the explanation and code being separate?
| ), cls='container mx-auto w-full px-4'), | ||
|
|
||
| # Chat messages | ||
| Div( |
There was a problem hiding this comment.
Can you minimize the verical space a bit more to match the style described in the fastai style guide? https://docs.fast.ai/dev/style.html
| Script(type="module", src="https://cdn.jsdelivr.net/npm/zero-md@3?register") # zero-md for markdown rendering | ||
| ) | ||
| # Set up the app, including daisyui and tailwind for the chat component | ||
| app = FastHTMLWithLiveReload(hdrs=headers, exts='ws', debug=True, htmlkw=dict( |
There was a problem hiding this comment.
Can you use FastHMTL or fast_app instead? LIve reload and debug is great when developing, but generally we want to remove that when we deploy it.
| Exception: If there is an error during the initialization of the OpenAIAgent or querying the agent. | ||
| """ | ||
| global agent, api_key_set | ||
| agent = OpenAIAgent.from_tools(llm=OpenAI(model="gpt-4o", api_key=apikey)) |
There was a problem hiding this comment.
With agent being a global variable shared by all users, does this cause issues where there could be conflicts when multiple users are using the app at the same time?
| print(f"Hello from OpenAI: {hello}") | ||
|
|
||
| except Exception as e: | ||
| print(f"Error setting OpenAI API Key: {e}") |
There was a problem hiding this comment.
Can you catch the specific exception instead of all exceptions. This could be confusing if it fails for a different reason as it will print error setting OpenAI API key regardless of what the exception was for and won't give a stack trace
| line_buffer = "" | ||
| async for response in response_stream.async_response_gen(): | ||
| chunk_message = response | ||
| print(chunk_message, end='', flush=True) # print message to console, remove for production |
There was a problem hiding this comment.
Can you go through and remove the print messages for the PR? I think there's a couple throughout which is great for debugging but don't want them in prod.
|
🤦 @Kvit How about now? |
Example of async chat with OpenAI Agent wrapped with Lamaindex RAG
Handles input of OpenAI API key from the screen entry form