AI Assistant
The AI assistant application leverages large language models like Llama 2 and Mixtral to engage in dynamic and contextually rich conversations. Users can ask questions, seek information, or request the model to perform text-based tasks such as writing, editing, or brainstorming ideas. The model can comprehend the input query and generate responses in natural language, making the interaction smooth and user-friendly. In addition to accessing LLMs, an advantage of using Spell Book over a vanilla LLM chat client is its function calling and model routing abilities. To take advantage of these, two models are required to be running: Llama 7B (ExLlama) and T5 Large v2. These two models are used in conjunction with two Loras to provide the function calling services for Spell Book.
- #1 AI Assistant Link Primary way to access the AI assistant is with the main menu.
- #2 Digital Allies Link Digital allies are AI assistants who have preconfigured settings to make them more useful for specific tasks or role play character profiles.
- #3 Conversation Settings The conversations generation and routing settings can be changed for each conversation with the settings dialog box.
- #4 Saved Conversations Conversations are saved and can be accessed with the Conversations buttons. Each digital ally stores its own conversation list.
- #5 Delete Conversation Conversation are stored in the database and can be deleted with this button.
You can interact with the AI assistant with your microphone if one is detected and you have a ASR model like Whisper loaded by Spell Book. The feature is activated by pressing the microphone button next to the prompt input box.
Once 2 seconds of silence is detected or you press the stop button the audio data will be submitted to the server for processing.
If xTTS is running and the proper setting is selected the models response will be vocalized in short snippets and played by the speakers. If this is not desired the functionality can be disabled on a conversation on digital ally basis.
Any prompt in the conversation tree can be regenerated, this is useful if a conversation starts to go off the rails and you need to backup a few steps to fix a few prompting mistakes.
- #1 Generation Selector Selector for viewing other generations in the conversation tree.
- #2 Regenerated Regenerated from any point in the conversation if it starts to go off the rails.
- #3 Cancel Cancels the regeneration and disregards your text edits to the prompt.
You can delete the last message in a conversation, it's icon is only visible when hovering over the last message with your mouse.
- #1 Delete message Delete the message and it's response from the AI assistant.
- #2 Edit and Regenerate Allow for you to edit the prompt and send it back to the model for regerenation.
The conversation settings allow to set the system message, change generations settings, select a microphone, name the conversation and select other options like if function calling is enabled. All of the settings found here can also be set on a Digital Ally level, so they are consistent each time you open a new conversation. See the Digital Ally documentation for a list of all the settings that can be changed.
Shortcuts allow you to access chat abilities and language models when automatic routing is not sending the request to the correct model or chat ability. Shortcuts are emojis that are prepended to a chat request. There are also several pre-defined shortcuts to control the context that is sent to the model. If function and skill pinning are turned on, manual selection also extends the routing definitions, so similar requests will be routed to the chat ability or language model automatically. Below are some examples of how to use shortcuts.
Each chat ability can be accessed with the shortcut emoji that appears at the beginning of its definition. This emoji also appears below the selected skill icon in the chat session.
Example: 🌞 What is the current temp in Chicago?
You can also access language models with a shortcut. The emoji that represents which language model a request can be sent to is found at the beginning of its name in the Online Skills section of the Servers configuration. This emoji is also output in the chat session in the left-hand column under the circular icon.
Example: ⚡ How do I open port 9000 on a Linux box?
You can also control the number of past messages that are sent to the language model by using the 👉 and 👆 emojis. This is useful if only a small part of the conversation is relevant to the next chat round and you are trying to save money on API calls to OpenAI. In the two examples below, the first will send only the one message to GPT-4. The second example sends the last 5 user/assistant responses to GPT-3.5.
Example: 👉✨ What is wrong with this code block?
Example: 👆5️⃣⚡ Summarize our conversation so far.
The Chat Ability and Skill pinning is functionality that allows the user to extend the definitions for routing. When pinning is enabled and a manual route is selected with an emoji, the function calling system will save the definition of the guessed function or knowledge domain to the database. Future similar chat requests will automatically be routed based on these extended definitions, making the function calling features work better with more use and manually selecting routes.