diff --git a/meeting_notes/Reginald 24_08_23.md b/meeting_notes/Reginald 24_08_23.md new file mode 100644 index 00000000..dceb49cd --- /dev/null +++ b/meeting_notes/Reginald 24_08_23.md @@ -0,0 +1,23 @@ +# Reginald 24/08/23 + +## Notes +- Quick overview of existing Reginald Work + - Reginald (Handbook): fine-tune a Q&A LLM using the REG handbook + - Reginald (Llama): use something called llama-index + - Quick recap of the type of questions it answered +- Discussion of work to do this month + - Mainly focus on model development + - Getting onto Turing slack would be nice, but important to get the model working well first + - Not the end of the world with the bot existing in another Slack workspace for now + - Time better spent to learn about LLMs and playing around with libraries rather than looking into technicalities of slack and IT policy + - Models to look at + - RAG + - Using LLama2 rather than OpenAI API + +## Actions +- Start playing around with llama-index again and read up on alternative models +- Look into using an open-source model rather than OpenAI +- Admin for Ryan: + - Get Yi-Ling added to Reginald repo + - Add Rosie to Azure subscription + - Create regular calendar invite diff --git a/meeting_notes/Reginald 31_08_23.md b/meeting_notes/Reginald 31_08_23.md new file mode 100644 index 00000000..98e23242 --- /dev/null +++ b/meeting_notes/Reginald 31_08_23.md @@ -0,0 +1,35 @@ +# Reginald 31/08/23 + +## Notes +- Rosie been working with using llama-index with LLama2 (currently running locally with use of [llama-cpp](https://github.com/abetlen/llama-cpp-python)) + - Went through Rosie's development notebook +- Also had a play around with llama-index chat engine + - Went through some examples + - With the chat engine, the model will try to determine if it should query the database (i.e. construct a prompt with some text from the database) or should it just send something to the LLM + - Can we investigate more about _how_ it does this? +- Is it possible to obtain the prompt to the LLM + - We know that llama-index does some prompt engineering, but would it be possible to look at that? + - Also in cases when it tries to refine the answer - does it actually give much benefit? +- Yi-Ling has been looking at llama-index and ideas of how to evaluate the models to see if it's using the knowledge base effectively + - We should compile some "model" question and answer pairs that we hope our model to get right + - Start to think about being able to systematically compare models and approaches +- Discussion about documentation + - Start uploading meeting notes + - Start documenting the models we try and our experiences with them + +## Actions +- Continue with chat engine hacking + - There are several chat engine choices given by llama-index + - Compare different approaches and figure out what is the most appropriate + - Try to figure out when the model is querying and when it is + - Try to figure out how it makes the decision of whether or not to just have a conversation or not + - How does this work with small context lengths? + - How is it remembering/tracking conversation history? +- Maybe start working on Azure rather than local + - Could be able to run a larger model + - Might not need GPU if only inference and only for dev + - Maybe GPU long term? +- Admin: + - Start meeting note uploads (Ryan) + - Start model documentation (all) + - Start project board (Rosie, Levan)