This first release of the code in this repository includes the following changes and new features:
- Major refactoring of the app and source code. Instead of defining an
OpenAIChatAgent
, there are now several "chat handlers", functions that take a message (as a string) and process it or parse it somehow.chat.handlers.openai
includes functions that wrap the OpenAI API and can be used for chat completion with ChatGPT (gpt-3.5-turbo
) ortext-davinci
, generating images with dall-e, transcribing audio, and more. - Now defines
ChatClient
objects to manage the reception and sending of messages. This way, we can integrate other message-handing clients besides Twilio (TwilioWhatsappClient
) in the future. - The code in
app
is now much simpler and readableapp/whatsapp/app.py
contains the code of the Flask app. I only implemented a few new handlers to work specifically with Whatsapp messages. The rest is just calling the respective handlers to make the app behave as it should.
What's left in the backlog?
- Explore Whisper Endpoint for voice messages: We could explore using OpenAI's new whisper endpoint for audio transcription instead of AssemblyAIs. I would like first to benchmark them and choose the one that is better suited for a live version of the app.
chat.handlers
already included two functions for audio transcriptionchat.handlers.openai.speech.voice_transcription
(whisper) andchat.handlers.assemblyai.transcribe_audio
. - Automatically save all messages and sessions to a local or remote database In the future, I would like to define the classes in
app/datamodels/
as sqlalchemy data models and connect them to (a local or hosted) SQL database. For testing use-cases, the app could create its own SQLite database and keep a record of user sessions and messages there, and for production scenarios, span a PostgreSQL DB in a separate container. - A database backend would be very well for scalability as it would add the possibility to host this with serverless functions instead of a web application. Sessions/Chats would be retrieved from the database and processed only when a new message request arrives.
- Once we start saving data to the database, it could also be cool to generate a metabase instance in its separate container to get live analytics and tracking of the messages that arrive in the app.