Develop a chatbot using models available on Intel Developer Cloud (OpenLlama, Camel, Zephyr, Intel Neural Chat, and Llama2 variants) or Prediction Guard’s LLM API (running models on IDC Gaudi2). Your task is to use this chatbot to creatively tell a story. The evaluation will focus on session transcripts that demonstrate the chatbot's responsiveness and creativity.
Learning Objective: Prompt engineering for LLMs, Finetuning, using LLM API services (Prediction Guard), memory management of chat interfaces (check the Simple LLM inference notebook), instruction prompts etc
- Story Development: Develop a storyline that your chatbot will narrate or participate in. This could be a scripted story where the chatbot plays a character, or an interactive story where the chatbot responds creatively to user inputs. Prepare a series of prompts that can guide the chatbot through the different stages of your story.
- Technical details: Please ensure the efficiency of your model is a priority (smaller the model, the better). If you're utilizing locally available models, I recommend reviewing the LLM finetuning notebook. Pay special attention to the BASE_MODELS section to understand the range of models accessible and the methodology for loading them via Intel's BigDL library, particularly for 4-bit model loading. If you are using Prediction Guard LLM APIs, checkout the documentation and also the library of models available with the Prediction Guard API.
- Creativity and Responsiveness: Focus on the chatbot's ability to be creative in its storytelling. This could be through imaginative responses, the ability to develop a narrative, or engaging interactions. Responsiveness is key – the chatbot should be able to handle unexpected inputs gracefully while staying in character or within the story's theme.
- Finetuning vs Prompt Engineering: Fine-tuning might offer more tailored responses but is time-consuming, while prompt engineering is quicker but requires creative and strategic prompt design. If opting for fine-tuning, select a small dataset that aligns closely with your desired narrative theme to expedite the training process.
- Notebooks: You can use the LLM inference notebook and the LLM finetuning notebook, both available on IDC under GenAI Essentials as the baseline for local models. If you are using Prediction Guard LLM APIs, we have a dedicated section for chat, check it out. We are certain it would be useful!
Technical setup: Emphasis on using small models, loaded in 4 bits if using Intel bigdl-llm library to load a local model, or access models over an API, loaded via Prediction Guard if LLM API is being used. Check out the inference section of LLM finetuning notebook on how to efficiently load models in 4bit on Intel GPUs and CPUs and this link in the resources on how to utilize LLMs with Prediction Guard.
Documentation: Document your development process, including the choice of model, the storyline, chatbot responses, and any challenges faced. Include session transcripts that highlight the chatbot's storytelling abilities and responsiveness.
Difficulty: Intermediate